From owner-freebsd-hackers@freebsd.org Sun Dec 6 00:23:04 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F3DB9A3FD19; Sun, 6 Dec 2015 00:23:03 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id C12B61E53; Sun, 6 Dec 2015 00:23:03 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [IPv6:2001:470:923f:2:e562:40a5:b2ac:6733] (unknown [IPv6:2001:470:923f:2:e562:40a5:b2ac:6733]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPSA id A16AD6E0; Sun, 6 Dec 2015 03:22:57 +0300 (MSK) Message-ID: <56637FE3.1020702@FreeBSD.org> Date: Sun, 06 Dec 2015 03:22:59 +0300 From: Lev Serebryakov Reply-To: lev@FreeBSD.org Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: freebsd-emulation@freebsd.org, freebsd-hackers@freebsd.org Subject: New way to rub FreeBSD on OS X -- xhyve, port of bhyve to OS X Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Dec 2015 00:23:04 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 I know, that many FreeBSD developers use MacBooks as their laptops. I've found port of bhyve to OS X: https://github.com/mist64/xhyve It should run FreeBSD better than Virtual Box :) - -- // Lev Serebryakov AKA Black Lion -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQJ8BAEBCgBmBQJWY3/iXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRGOTZEMUNBMEI1RjQzMThCNjc0QjMzMEFF QUIwM0M1OEJGREM0NzhGAAoJEOqwPFi/3EeP3nYP/jfHN1fw69om/BQTqCjMjJmY 20+ACnSX3Dgh04gSRBwCY3A/Oz34fCqFFAKgv6o8R9NZcw/RlDKnxzR7cBULPL1i 8Bb8O3dDf4izkU7p118/UIm9CpEndGsaoging3r0RGMo3svOUUpiQqelcSaXG25v 4eMMXcZ88j2YBDi6G3mNuXjqT91WrljKYQiwLRivUwvAQEsCC3q60aH8qbvSoewE kqq9LTIIbfrWauF+MrEEdRElawJFQ3GHRYDP0JvmSPKuMG+fgfk1jUuW6KFQaXTT x87MsmsiRi1Ln5I4LZpoFERPvhNNq7nCJglj5Wadh08VroJokRf5PoEPV6xIPDgH vKoogoXDI4HQbN3aVEc6ZJ++F+7RH/WPpIT5S4BDZtgXRf6oAIui8eaCWwhjJ34d /L5+5IGypdRmCm5cnnAo3tr0rwsiBXjge/yxR6JcN6WFlnlpKaQtu04+2+Sli8q2 NsGln42iGeX9HeWGa0SmCD1/qWr6YoEvbnD7yeu8/VBianV3fM3k617w/W5ir8oA 20OU+KegTd0A6rx9LSbLDE/H36nZmA5d9oPpbrHr9GogQpZlNM3cd+ndlGOkE4W9 WBgvaV60JD5lWxTKW9LAQxBQaidh5IutfZ3JRLLU1hQl2qoTZWlK+0fOZNAx9wf9 r11VordeV2amGmQuU3tQ =Wyy0 -----END PGP SIGNATURE----- From owner-freebsd-hackers@freebsd.org Sun Dec 6 00:53:58 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9DA42A403F4 for ; Sun, 6 Dec 2015 00:53:58 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 54CEE1D5F for ; Sun, 6 Dec 2015 00:53:58 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: by qgcc31 with SMTP id c31so118621350qgc.3 for ; Sat, 05 Dec 2015 16:53:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=longcount-org.20150623.gappssmtp.com; s=20150623; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=Ov7ywEDvu2wy7hxoGebSa49oEIW+aiIrDkIAvqDadow=; b=kPUA8kYhPN0v1Mq9UO/J3GggCQ8ff3jpOqjTL9l2QKv6+kmmvQv68uq2VAlPuRb3fc Hjmaqh/oQotyhX4SdqMyfOZwSL0CrNxJOUa799JJVeu3MxeEtXd+qTsBZiPmoyIa+n45 JtIv2QSATkSTcp0o97ebdbSMnZCcgrBUpK/fidosUxc5BEOEt2x7QzFspJDFPPbRsS3L miGbqhn0rAUd4vC78NfXahkunqhOb2c6z/YY+tacPeE7IdWUAmviaRzhgn1jA35gqdbf mAunZZ6zGFo8+dqgcb8iustlCczfq4PQmYZGWXYMWcqScwiJO2sLJtGJkB+TkGpy6yMY n2dA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=Ov7ywEDvu2wy7hxoGebSa49oEIW+aiIrDkIAvqDadow=; b=Ho1ay1U4mjrRZVVILTQtcE/qPlhi1BgfkE65jhg4wtyS0umCjFYeUsL8eUrMDS0rYe 42VtcUg7rZqlo/UrGdFNTAVaDEARPl3ynOUn+5nBvEuNZCH+ABB0pxQLf+IyAvbLh8rc 9rpUmI1ZUKbpIxbUZiX03PN1kPxQZ4qo73DDyJ7IlHcxHR6KrVvpDduA7NSgxFJriwlK UdeqAMymXUl903aaxpTzzeQNoEzdaakjaxRr2bSBV6T8uEj1bmaIvEQK/Jw39nSJCUJ9 H7X1mQTIz/NFuqaej3EZjeXp1CzqqS9yrVb4EkLhuEPMXpIE688JOJkbDqm4OnWkcWDi 0XBQ== X-Gm-Message-State: ALoCoQlhl1nK+eiGXB6fpQwdkEOlMjvtuzc2jCYD3tk53PVR/Gdjr+g9pUmR2Jdx2pJ6RGtJCJYH X-Received: by 10.140.94.243 with SMTP id g106mr28520189qge.57.1449363237254; Sat, 05 Dec 2015 16:53:57 -0800 (PST) Received: from [100.119.145.64] (121.sub-70-208-75.myvzw.com. [70.208.75.121]) by smtp.gmail.com with ESMTPSA id 66sm8658026qhu.10.2015.12.05.16.53.56 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 05 Dec 2015 16:53:56 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: New way to rub FreeBSD on OS X -- xhyve, port of bhyve to OS X From: Mark Saad X-Mailer: iPhone Mail (13B143) In-Reply-To: <56637FE3.1020702@FreeBSD.org> Date: Sat, 5 Dec 2015 19:53:55 -0500 Cc: freebsd-emulation@freebsd.org, freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <90B19B1D-1078-49BF-B2EC-ED2F00345800@longcount.org> References: <56637FE3.1020702@FreeBSD.org> To: lev@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Dec 2015 00:53:58 -0000 Lev When I looked at this a while back it was very Linux specific. Has there b= een any work on making it more generic ?=20 --- Mark Saad | nonesuch@longcount.org > On Dec 5, 2015, at 7:22 PM, Lev Serebryakov wrote: >=20 > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 >=20 >=20 > I know, that many FreeBSD developers use MacBooks as their laptops. >=20 > I've found port of bhyve to OS X: https://github.com/mist64/xhyve >=20 > It should run FreeBSD better than Virtual Box :) >=20 > - --=20 > // Lev Serebryakov AKA Black Lion > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.22 (MingW32) >=20 > iQJ8BAEBCgBmBQJWY3/iXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w > ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRGOTZEMUNBMEI1RjQzMThCNjc0QjMzMEFF > QUIwM0M1OEJGREM0NzhGAAoJEOqwPFi/3EeP3nYP/jfHN1fw69om/BQTqCjMjJmY > 20+ACnSX3Dgh04gSRBwCY3A/Oz34fCqFFAKgv6o8R9NZcw/RlDKnxzR7cBULPL1i > 8Bb8O3dDf4izkU7p118/UIm9CpEndGsaoging3r0RGMo3svOUUpiQqelcSaXG25v > 4eMMXcZ88j2YBDi6G3mNuXjqT91WrljKYQiwLRivUwvAQEsCC3q60aH8qbvSoewE > kqq9LTIIbfrWauF+MrEEdRElawJFQ3GHRYDP0JvmSPKuMG+fgfk1jUuW6KFQaXTT > x87MsmsiRi1Ln5I4LZpoFERPvhNNq7nCJglj5Wadh08VroJokRf5PoEPV6xIPDgH > vKoogoXDI4HQbN3aVEc6ZJ++F+7RH/WPpIT5S4BDZtgXRf6oAIui8eaCWwhjJ34d > /L5+5IGypdRmCm5cnnAo3tr0rwsiBXjge/yxR6JcN6WFlnlpKaQtu04+2+Sli8q2 > NsGln42iGeX9HeWGa0SmCD1/qWr6YoEvbnD7yeu8/VBianV3fM3k617w/W5ir8oA > 20OU+KegTd0A6rx9LSbLDE/H36nZmA5d9oPpbrHr9GogQpZlNM3cd+ndlGOkE4W9 > WBgvaV60JD5lWxTKW9LAQxBQaidh5IutfZ3JRLLU1hQl2qoTZWlK+0fOZNAx9wf9 > r11VordeV2amGmQuU3tQ > =3DWyy0 > -----END PGP SIGNATURE----- > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"= From owner-freebsd-hackers@freebsd.org Sun Dec 6 03:14:49 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8F92A3BFE2 for ; Sun, 6 Dec 2015 03:14:49 +0000 (UTC) (envelope-from jim@netgate.com) Received: from mail-ob0-x22c.google.com (mail-ob0-x22c.google.com [IPv6:2607:f8b0:4003:c01::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A15F418AF for ; Sun, 6 Dec 2015 03:14:49 +0000 (UTC) (envelope-from jim@netgate.com) Received: by obcse5 with SMTP id se5so92862079obc.3 for ; Sat, 05 Dec 2015 19:14:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netgate.com; s=google; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=TwKwLgLHdbLxSI1udVPJOlVcu2mvbakVlofzsy0U2c8=; b=NicSy5lfldfy0E6dvj4/h8lLuGwh5dTvUQsupZzWzERjCXHuP5hYXr7NTdlQlm8RaS B10NpZIsDfjwP2qfTglD5ZUZDmkvSqc0umf3Yb9IKI2StFLuqXI9DRFR2FZAzLyB4V2W FtYR+CLA6xg8IsD0Mjc+iBpT6mvqHBOedGZ5k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=TwKwLgLHdbLxSI1udVPJOlVcu2mvbakVlofzsy0U2c8=; b=GsKOGVMIvxKEWO4E6A5FhvammWwzZiALI7UCk/7ejDhI1umUuiI2cw20JyTJqDghXR UvifgXuecPaus8hrCCB23i4/nVUtaiMCRVOMLO6BsSy+p7cQIM8oAYccZsQgVQCWBA/9 tdrM9J4AXHYNPEYpNTE23ye+Na4Op3900NfN6mhvAWVyvMCxvLUpfgBa2bcpbSU8dnP4 jkYDUE6YCPQD/9dXVgLCpSJm79aqX1+/G1x1fMhYQcs9ERKO29Xyi92ER132zHnAdCzo jablsjv6Eg4t/eTSzCgtlqjAXFJop6eolG7lDUL/rqF9y8mkAWIaxkiyXRdqnjq54/Gy 1OdA== X-Gm-Message-State: ALoCoQmVnRN69dfnQW178arLDXYwkRbzj/gh7sCRnXd/IAZpENveq8y5iy9TiHUjSegT4tfT+gWV X-Received: by 10.60.174.201 with SMTP id bu9mr10217411oec.11.1449371688140; Sat, 05 Dec 2015 19:14:48 -0800 (PST) Received: from ?IPv6:2001:470:1f0f:281:4f2:807c:4b17:e33f? ([2001:470:1f0f:281:4f2:807c:4b17:e33f]) by smtp.gmail.com with ESMTPSA id wr6sm9112229obb.21.2015.12.05.19.14.44 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 05 Dec 2015 19:14:44 -0800 (PST) Mime-Version: 1.0 (1.0) Subject: Re: New way to rub FreeBSD on OS X -- xhyve, port of bhyve to OS X From: Jim Thompson X-Mailer: iPhone Mail (13B143) In-Reply-To: <90B19B1D-1078-49BF-B2EC-ED2F00345800@longcount.org> Date: Sat, 5 Dec 2015 21:14:43 -0600 Cc: freebsd-hackers@freebsd.org, freebsd-emulation@freebsd.org Message-Id: <46C2382D-DE25-492F-A737-2E43D2F5851B@netgate.com> References: <56637FE3.1020702@FreeBSD.org> <90B19B1D-1078-49BF-B2EC-ED2F00345800@longcount.org> To: Mark Saad , lev@FreeBSD.org Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Dec 2015 03:14:50 -0000 Lev, it's not new: https://github.com/mist64/xhyve/pull/30 Mark, enjoy: https://github.com/gvnn3/xhyve/commit/2d400d0eeaa6f23bc91db658dabe4afec431d6= 2d -- Jim > On Dec 5, 2015, at 6:53 PM, Mark Saad wrote: >=20 > Lev > When I looked at this a while back it was very Linux specific. Has there b= een any work on making it more generic ?=20 >=20 > --- > Mark Saad | nonesuch@longcount.org >=20 >> On Dec 5, 2015, at 7:22 PM, Lev Serebryakov wrote: >>=20 >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA512 >>=20 >>=20 >> I know, that many FreeBSD developers use MacBooks as their laptops. >>=20 >> I've found port of bhyve to OS X: https://github.com/mist64/xhyve >>=20 >> It should run FreeBSD better than Virtual Box :) >>=20 >> - --=20 >> // Lev Serebryakov AKA Black Lion >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v2.0.22 (MingW32) >>=20 >> iQJ8BAEBCgBmBQJWY3/iXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w >> ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRGOTZEMUNBMEI1RjQzMThCNjc0QjMzMEFF >> QUIwM0M1OEJGREM0NzhGAAoJEOqwPFi/3EeP3nYP/jfHN1fw69om/BQTqCjMjJmY >> 20+ACnSX3Dgh04gSRBwCY3A/Oz34fCqFFAKgv6o8R9NZcw/RlDKnxzR7cBULPL1i >> 8Bb8O3dDf4izkU7p118/UIm9CpEndGsaoging3r0RGMo3svOUUpiQqelcSaXG25v >> 4eMMXcZ88j2YBDi6G3mNuXjqT91WrljKYQiwLRivUwvAQEsCC3q60aH8qbvSoewE >> kqq9LTIIbfrWauF+MrEEdRElawJFQ3GHRYDP0JvmSPKuMG+fgfk1jUuW6KFQaXTT >> x87MsmsiRi1Ln5I4LZpoFERPvhNNq7nCJglj5Wadh08VroJokRf5PoEPV6xIPDgH >> vKoogoXDI4HQbN3aVEc6ZJ++F+7RH/WPpIT5S4BDZtgXRf6oAIui8eaCWwhjJ34d >> /L5+5IGypdRmCm5cnnAo3tr0rwsiBXjge/yxR6JcN6WFlnlpKaQtu04+2+Sli8q2 >> NsGln42iGeX9HeWGa0SmCD1/qWr6YoEvbnD7yeu8/VBianV3fM3k617w/W5ir8oA >> 20OU+KegTd0A6rx9LSbLDE/H36nZmA5d9oPpbrHr9GogQpZlNM3cd+ndlGOkE4W9 >> WBgvaV60JD5lWxTKW9LAQxBQaidh5IutfZ3JRLLU1hQl2qoTZWlK+0fOZNAx9wf9 >> r11VordeV2amGmQuU3tQ >> =3DWyy0 >> -----END PGP SIGNATURE----- >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"= From owner-freebsd-hackers@freebsd.org Sun Dec 6 03:17:23 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7C601A3B0B2 for ; Sun, 6 Dec 2015 03:17:23 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-wm0-x22d.google.com (mail-wm0-x22d.google.com [IPv6:2a00:1450:400c:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B60A1A30 for ; Sun, 6 Dec 2015 03:17:23 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: by wmvv187 with SMTP id v187so123330662wmv.1 for ; Sat, 05 Dec 2015 19:17:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=jClvSs7UwWJAglVnTVshmTBnbtHD+pdplgIXVB/CP4o=; b=pf93EdzVbsegCId7UrzFZu4sG1poxhCAi0C9hZLnZZNbISGnqVMbClGCWjmwgZD/Wi jF0EkKDgkjK6sOgqJawbczn/mZbiMcWXvmybUL0CrkhtahlR4L+WG0rIxwXT5SzoS0xl OJXnRxRdERoNjm9dUr95jpmsrRM1GLvlPdswE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=jClvSs7UwWJAglVnTVshmTBnbtHD+pdplgIXVB/CP4o=; b=ir8hnEh7HjJTVIwIHSw/iPcs5wwrly3El1qsm0MnBgIpvtq/igNe/9KU3n+/41ixrA 3gmOk2MEo8TK5ZcTgnVhBBuOdfULu8m6F5OMWEtIciWFRrjinbAKl8knGCCEcBRf2Aff A7yVOGXYhor/143QgWHL/uXkvdDVRrabXlYYnp9JJvaK6QjiQcQrJSTzriMUcSDjcacK 7MsolBour0IpZBwkLc8vpeM699xH6uMRRQ7dULySN2lCOiMc/2NYYapZ57qFvSBjY5CS CuGq11OWZDJeALYSyjNzUTKMVz/3+MDkcfRqZsb8mx4YYZMEKx8Vlm73piAja8T3WdgL rXeQ== X-Gm-Message-State: ALoCoQkIbn8zy02olA8MXzkEmr1Mm3uwV+UT2wb7md19nvbLL6nWvHvVRq4e4MH6fZVIZ3fegBDl X-Received: by 10.28.128.210 with SMTP id b201mr12724149wmd.69.1449371841085; Sat, 05 Dec 2015 19:17:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.28.43.1 with HTTP; Sat, 5 Dec 2015 19:16:51 -0800 (PST) In-Reply-To: References: From: Eitan Adler Date: Sat, 5 Dec 2015 19:16:51 -0800 Message-ID: Subject: Re: DELETE support in the VOP_STRATEGY(9)? To: Maxim Sobolev , FreeBSD Hackers , Pawel Jakub Dawidek Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Dec 2015 03:17:23 -0000 + correct mailing list - non-technical list On 4 December 2015 at 21:38, Maxim Sobolev wrote: > Hi folks, do we support delete operation in the vnode layer? This comes from > observation that md(4) converts from DELETE into sector-size zero buffer > write before it feeds it to the vnode. It's shame that a trim operation is > getting lost when you run md(4)+ufs atop a vnode. With the magnetic spinning > storage joining home particle accelerators, aka CRTs in the hall of fame > soon, we better make sure it's turtles all the way down in either direction. > By which I mean make sure that trim can not only propagate from VFS to BIO > but also the other way around . I believe having this would also allow > "punch-hole" API in the userland to be implemented trivially, syscall that > VM's out there would love to have. > > -Max -- Eitan Adler From owner-freebsd-hackers@freebsd.org Mon Dec 7 10:20:14 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 712C19B95E8 for ; Mon, 7 Dec 2015 10:20:14 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 3AA6F13D5; Mon, 7 Dec 2015 10:20:13 +0000 (UTC) (envelope-from des@des.no) Received: from desk.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 48F6DDA73; Mon, 7 Dec 2015 10:20:07 +0000 (UTC) Received: by desk.des.no (Postfix, from userid 1001) id 5EDDF48179; Mon, 7 Dec 2015 11:19:59 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Maxim Sobolev , FreeBSD Hackers , Pawel Jakub Dawidek , Eitan Adler Subject: Re: DELETE support in the VOP_STRATEGY(9)? References: Date: Mon, 07 Dec 2015 11:19:59 +0100 In-Reply-To: (Eitan Adler's message of "Sat, 5 Dec 2015 19:16:51 -0800") Message-ID: <86twnur25s.fsf@desk.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Dec 2015 10:20:14 -0000 Maxim Sobolev writes: > Hi folks, do we support delete operation in the vnode layer? This > comes from observation that md(4) converts from DELETE into > sector-size zero buffer write before it feeds it to the vnode. As the person who implemented this (because I needed a reliable way to test fsck_ffs -E): it is currently the only possible action, other than to ignore the request - which is actually fine since BIO_DELETE is not guaranteed to do anything whatsoever, except not corrupt data you didn't want deleted. I'm not opposed to the idea of VOP_DELETE or similar, but don't assume that "punching through" is always the correct semantic, and don't assume that it will be easy to implement - and it will have to be implemented correctly at every layer, in every geom, in every storage driver etc. There are many details to work out and many opportunities for mistakes. Remember that BIO_DELETE is strictly advisory. Should VOP_DELETE also be advisory, or do we want to guarantee that a VOP_DELETEd region reads back as all zeroes? Remember that we can't guarantee that the data will be removed from the underlying medium, or verify that it was. How do we handle an unaligned VOP_DELETE? Should a VOP_DELETE create a hole if the filesystem support holes? Since you mentioned VMs, this could result in fragmentation of initially unfragmented (preallocated) disk images. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@freebsd.org Tue Dec 8 09:35:33 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C437A9D4E17 for ; Tue, 8 Dec 2015 09:35:33 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: from mail-wm0-x232.google.com (mail-wm0-x232.google.com [IPv6:2a00:1450:400c:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 809091D80 for ; Tue, 8 Dec 2015 09:35:33 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: by wmec201 with SMTP id c201so203804838wme.0 for ; Tue, 08 Dec 2015 01:35:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sippysoft-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=uWVUNsO9fQOhywnJRteV1Fp3z0rXOKPNynYP70LqNtA=; b=DnugOvXbLv2+3nDiUIMvtFWn7gNg9mac1B+4qwCzCYbyPQdRHlTlqZnjr0smwMN4jm IUqiUb0yb82P700WiH5OTaZU1e0Br0ZJCld/caWWKEUB9R+UHx2ZEVEv4aoTw7xk6Ttp 5QbLIDBJ/tDxsE3RKLsTa+h7q77HtzYYOotxzhaoSvU+EXqKshdPnYEv06l4JBFMub41 NwDb7Y4FVRsNpTb1hGuK2ARXnNYCDMm9+UA7rqfwUKPRPpnaptx4lzVot3jBMNFSwzxm T8s7HmED9m9/+cqFwAufjY/uaZM9lirko6sqsGDgTD7GazSNWNNZ/Lxd4zI1mbRjJW7E q9Wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:date:message-id:subject:from :to:cc:content-type; bh=uWVUNsO9fQOhywnJRteV1Fp3z0rXOKPNynYP70LqNtA=; b=H2+XI9Fn6DWjeiXl/S9zTJ4Wcbfqj5TMz/WyPX9z8kuEHJGN1FmJQfds6uypHwsy0A DQ7hjGn3c0BHZ8d7fUXl+/6TDVnkF0Kl4PW6CR/oRCbUhdZwi6jC6Qg+2tCLq5gawKn9 4kBkBemjCdUhMWqwbnmNZxLwiZksE53X5ePQSExONANtC0QkH1bulU0oN+YKC0OH3h00 2+rvUUqCPfdJkSAge7VFDhgyiaDoDi6L/zyWbn2vX1aAO/pURFwJqB8s+lU/9PO9mBJ2 hHlnRmIv9XMLyu95OcPkPUyAiCaXvyleaR99xnSuzfua69m4yc5BhbPSsq5YMy1Xr41f pZTw== X-Gm-Message-State: ALoCoQn7xNbEX/Tfl0oPKkrhIXhIRK54j2G4FEaLPxUH7VFyELjaqx5RMbg/QvPY0S7W7F9IOpPLV0R8LIWT5vAFZ3bMKlgetJn10gmJY2q8Swh27KqtsDY= MIME-Version: 1.0 X-Received: by 10.194.172.2 with SMTP id ay2mr2557252wjc.137.1449567331781; Tue, 08 Dec 2015 01:35:31 -0800 (PST) Sender: sobomax@sippysoft.com Received: by 10.27.39.135 with HTTP; Tue, 8 Dec 2015 01:35:31 -0800 (PST) Date: Tue, 8 Dec 2015 01:35:31 -0800 X-Google-Sender-Auth: EqyANEPP8GE7pEsueXqy4Kn3ZZM Message-ID: Subject: posix_fallocate(2) && posix_fadvise(2) are somewhat broken From: Maxim Sobolev To: freebsd-current@freebsd.org, freebsd-hackers@freebsd.org Cc: kib@freebsd.org, Kirk McKusick Content-Type: multipart/mixed; boundary=089e013c6214e43b4c05265fb14e X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 09:35:34 -0000 --089e013c6214e43b4c05265fb14e Content-Type: text/plain; charset=UTF-8 Hi, while working on some unrelated feature I've noticed that at least those two system calls are not returning proper value (-1) on error. Instead actual errno value is returned from the syscall verbatim, i.e. posix_fadvise() returns 22 on EINVAL. Attached patch fixes that problem, however I am not sure if I need to assign td->td_retval[0] at all, those two operations by design never return anything but -1 on error and 0 on success. Can someone comment on this? Thanks! --089e013c6214e43b4c05265fb14e Content-Type: text/plain; charset=US-ASCII; name="vfs_syscalls.diff" Content-Disposition: attachment; filename="vfs_syscalls.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_ihx6qlod0 ZGlmZiAtLWdpdCBhL3N5cy9rZXJuL3Zmc19zeXNjYWxscy5jIGIvc3lzL2tlcm4vdmZzX3N5c2Nh bGxzLmMKaW5kZXggZTY3NWIwOS4uYmRiMTYzOSAxMDA2NDQKLS0tIGEvc3lzL2tlcm4vdmZzX3N5 c2NhbGxzLmMKKysrIGIvc3lzL2tlcm4vdmZzX3N5c2NhbGxzLmMKQEAgLTQ1MjgsNyArNDUyOCw3 IEBAIHN5c19wb3NpeF9mYWxsb2NhdGUoc3RydWN0IHRocmVhZCAqdGQsIHN0cnVjdCBwb3NpeF9m YWxsb2NhdGVfYXJncyAqdWFwKQogCiAJdGQtPnRkX3JldHZhbFswXSA9IGtlcm5fcG9zaXhfZmFs bG9jYXRlKHRkLCB1YXAtPmZkLCB1YXAtPm9mZnNldCwKIAkgICAgdWFwLT5sZW4pOwotCXJldHVy biAoMCk7CisJcmV0dXJuICh0ZC0+dGRfcmV0dmFsWzBdKTsKIH0KIAogLyoKQEAgLTQ2NjUsNSAr NDY2NSw1IEBAIHN5c19wb3NpeF9mYWR2aXNlKHN0cnVjdCB0aHJlYWQgKnRkLCBzdHJ1Y3QgcG9z aXhfZmFkdmlzZV9hcmdzICp1YXApCiAKIAl0ZC0+dGRfcmV0dmFsWzBdID0ga2Vybl9wb3NpeF9m YWR2aXNlKHRkLCB1YXAtPmZkLCB1YXAtPm9mZnNldCwKIAkgICAgdWFwLT5sZW4sIHVhcC0+YWR2 aWNlKTsKLQlyZXR1cm4gKDApOworCXJldHVybiAodGQtPnRkX3JldHZhbFswXSk7CiB9Cg== --089e013c6214e43b4c05265fb14e-- From owner-freebsd-hackers@freebsd.org Tue Dec 8 10:59:53 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9813C9D4007 for ; Tue, 8 Dec 2015 10:59:53 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from shell1.rawbw.com (shell1.rawbw.com [198.144.192.42]) by mx1.freebsd.org (Postfix) with ESMTP id 87C3E1EA8 for ; Tue, 8 Dec 2015 10:59:53 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from yuri.doctorlan.com (c-50-184-63-128.hsd1.ca.comcast.net [50.184.63.128]) (authenticated bits=0) by shell1.rawbw.com (8.15.1/8.15.1) with ESMTPSA id tB8Axqqc079559 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Tue, 8 Dec 2015 02:59:52 -0800 (PST) (envelope-from yuri@rawbw.com) X-Authentication-Warning: shell1.rawbw.com: Host c-50-184-63-128.hsd1.ca.comcast.net [50.184.63.128] claimed to be yuri.doctorlan.com To: Freebsd hackers list From: Yuri Subject: How to get the deterministic result for FreeBSD tar(1)? Message-ID: <5666B828.5000306@rawbw.com> Date: Tue, 8 Dec 2015 02:59:52 -0800 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 10:59:53 -0000 I have two identical directories (no diffs, all identical mtime attributes) compressed by this command: find dir -print0 | LC_ALL=C sort -z | tar cf archive.tgz --format=bsdtar --no-recursion --null -T - The results are different: 3 files out of 10,000 have pax attributes set that are different: - 27 ctime=1449566560.642715 +27 ctime=1449566903.167521 src/contrib/libarchive/archive_write_set_format_by_name.c suggests that format=bsdtar should force ARCHIVE_FORMAT_TAR_PAX_RESTRICTED format (no attributes), unless need_extension=1 is set on a per-file basis in archive_write_set_format_pax.c. need_extension=1 is triggered by these conditions: * too long or non-ASCII path * too long or non-ASCII link * too large file * too long GID or UID * too long or non-ASCII group name or user name * ACL entries and extended attributes * sparse info In my case file hierarchy is indeed very deep, and these three files also have the "path" attribute. I think this is a bug that in archive_write_set_format_pax.c ctime attribute is written in case one of the above conditions are satisfied, because ctime can't be controlled by the user, and will always cause the difference. So I have two questions: 1. How do I actually achieve the output determinism for tar(1)? 2. Is there an agreement that this is a bug that too long or non-ASCII path name triggers the leakage of ctime into a tar file? Yuri From owner-freebsd-hackers@freebsd.org Tue Dec 8 11:06:44 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EF5529D44EA for ; Tue, 8 Dec 2015 11:06:43 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 7EE581290; Tue, 8 Dec 2015 11:06:42 +0000 (UTC) (envelope-from des@des.no) Received: from desk.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id C6995DE6C; Tue, 8 Dec 2015 11:06:35 +0000 (UTC) Received: by desk.des.no (Postfix, from userid 1001) id 31C1348225; Tue, 8 Dec 2015 12:06:32 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Maxim Sobolev Cc: Kirk McKusick , Pawel Jakub Dawidek , Warner Losh , freebsd-hackers@freebsd.org Subject: Re: DELETE support in the VOP_STRATEGY(9)? References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> Date: Tue, 08 Dec 2015 12:06:32 +0100 In-Reply-To: (Maxim Sobolev's message of "Tue, 8 Dec 2015 00:53:48 -0800") Message-ID: <86fuzdqjwn.fsf@desk.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 11:06:44 -0000 Maxim Sobolev writes: > Dag-Erling Sm=C3=B8rgrav writes: > > 1) why did you take this off the list? > There was a complain from list admin about this being off-topic. Yes, and Eitan moved the discussion to hackers@. It should have stayed there. > > 2) why did you even bother to cc: me if you were going to competely > > ignore everything I said anyway? > I did not really ignore it, it just that I did not have much to reply > at that point. [...] Basically I don't think your concerns wrt DELETE > reliability/gurantees have much to do with this particular feature. > The reason being that BIO_DELETE essentially tells the storage layer > that whichever code "owns" the block in question (e.g. ZFS or UFS) > has moved it into the free pool and will NEVER ever want to read its > value back again (until it's written into again). No, it means that the contents of that block are no longer important and that the lower layers *may* reclaim it. It does not mean that nobody will ever try to read the block, nor does it guarantee that the block will actually be reclaimed or zeroed. We cannot rely on the lower layers to ensure that reading from a previously deleted block never returns data that may have belonged to a different file. BTW, I've encountered CF cards (including the SanDisk card in my home router) that freeze if issued a TRIM command. Furthermore, many CF, MMC and SD cards, especially those marketed for use in digital cameras, perform wear leveling "automagically" based on their own understanding of the filesystem layout, and will therefore work poorly with anything other than FAT (Kingston call it "optimized recording performance" in their marketing literature). > Technically speaking on 100% correctly working os/hardware attempt to > read block after it's been successfully BIO_DELETE'd could produce > exception of some sort without any ill effects. If that were the case, it would never be safe to do # dd if=3D/dev/da0 of=3D/dev/da1 bs=3D4096 conv=3Dsparse which I'm sure you'll agree is not acceptable. > [...] in this particular case of VOP_ALLOCATE(FALLOC_FL_PUNCH_HOLE), a > filesystem in question is responsible for making sure the range that > has been punched through reads 0, whether by making real logical hole > in the file and/or by padding it with zeroes as needed. Is it really? Here are a few of our options for implementing FALLOC_FL_PUNCH_HOLE: a) create a filesystem-level hole in the disk image; b) perform a), then issue a BIO_DELETE for the blocks that were released; c) perform a) or b), then zero the overspill if the requested range is unaligned; d) zero the entire range; e) perform d) followed by either a) or b); f) nothing at all. Now, consider the case of the guest OS in a VM issuing TRIM commands to the emulated storage controller, which the hypervisor translates into a FALLOC_FL_PUNCH_HOLE request for the corresponding range in the disk image. Discuss the advantages and drawbacks of each option I listed above for each of the 36 points in the space defined by the following axes: - The disk image is: - a preallocated file on a filesystem (or an md(4) device backed by a preallocated file) - a dynamically allocated file on a filesystem (or an md(4) device backed by an unallocated file) - a zvol - a device - The underlying storage's preferred block size is: - small (e.g. 4 kB sectors on an AF drive) - medium (e.g. 64 kB stripes on a RAID) - large (e.g. 1 MB erase blocks on an SSD) - The physical storage is: - volatile - solid-state - electromechanical If you think the answer is the same in all cases, you are deluded. DES -- Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@freebsd.org Tue Dec 8 15:43:35 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47B859D453D for ; Tue, 8 Dec 2015 15:43:35 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EECBC1482 for ; Tue, 8 Dec 2015 15:43:34 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by qgcc31 with SMTP id c31so21462425qgc.3 for ; Tue, 08 Dec 2015 07:43:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=XtrFqTMrE4RjoyBsdvti78NOB67bGaVhCPdf9wyN+Wk=; b=wqtzMFkfOtGoIO0zGfB4QGY/s31YSQmApwFxsZmxG8BvnR5Vdv3OCQRwRqmJQCrLZp IAd0CVWniVO/7izJ9ULuNsV4NdA/sftDY3i8L8BZClJjvo2aO5xKvGU6eTGN8cgqjVQZ NHWgcQSpTloLIygCyvdyI86SAPF8velLsPphzUSZN3S/rRwkzfJK/HXFa90+epKfqYTS u2M7GuufDlX3OFfXh/qKZX9FKMWVr5dAWmghVGbdPQpNylugTnyWPe2eKeWSjOwMXNHf cq7tX09j0rh1dq+ldJmyCTX1VoLO4qTkinGyA8wYdSzULWWCwf81/yzbX5V+O6UgG/QQ makA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=XtrFqTMrE4RjoyBsdvti78NOB67bGaVhCPdf9wyN+Wk=; b=Ib1VMo20ekm0Ph3dB4e5ueNiFu5/Zpcr6rYp3Rt0oukeTIAvqe4z7Q1gP9SIwkLF81 TxXtbc1QhOs6plRBazB0z08FSkZ4d+WwH2uE8AT3L3D95BuYNPxqyZVMK9Nvow5Cx1Ms 9g8hm++m85WVwZDKp5SOFW8N++gp10zvukFukm4A031o+Tez3XPqiAL6w5ovqo0hsFZl yJXTU23ju73+hhoHsjxVhIazWdayCT3LrelOpRWMl/pfdMmft389XRwRrUFSowHo3QT1 vRubx0gx7uoKNu8hNk2WLzgQ0dDtcthKwwEcKJJoP4Lb8Rk6VcPQiwz/DvnDhwjCe7rg mAEA== X-Gm-Message-State: ALoCoQnXbIcXEMQk3zseg0MrHedPqXOldkNFfW08JXFE/hOci8jhBag2YF4Z3oyVAyXgx2ydY8t4pkDupJ4TIg/FWXiEs4qflA== MIME-Version: 1.0 X-Received: by 10.55.81.11 with SMTP id f11mr326490qkb.10.1449589413218; Tue, 08 Dec 2015 07:43:33 -0800 (PST) Sender: wlosh@bsdimp.com Received: by 10.140.27.181 with HTTP; Tue, 8 Dec 2015 07:43:33 -0800 (PST) X-Originating-IP: [2601:280:4900:3700:4d3f:8eba:ea86:7700] In-Reply-To: References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> Date: Tue, 8 Dec 2015 08:43:33 -0700 X-Google-Sender-Auth: UiMUWGSwZOK9bOA8cEcJ52dXJ8Q Message-ID: Subject: Fwd: DELETE support in the VOP_STRATEGY(9)? From: Warner Losh To: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 15:43:35 -0000 [ forgot to cc hackers ] ---------- Forwarded message ---------- From: Warner Losh Date: Tue, Dec 8, 2015 at 8:41 AM Subject: Re: DELETE support in the VOP_STRATEGY(9)? To: Dag-Erling Sm=C3=B8rgrav On Tue, Dec 8, 2015 at 4:06 AM, Dag-Erling Sm=C3=B8rgrav wrote= : > Maxim Sobolev writes: > > Dag-Erling Sm=C3=B8rgrav writes: > > > 1) why did you take this off the list? > > There was a complain from list admin about this being off-topic. > > Yes, and Eitan moved the discussion to hackers@. It should have stayed > there. > > > > 2) why did you even bother to cc: me if you were going to competely > > > ignore everything I said anyway? > > I did not really ignore it, it just that I did not have much to reply > > at that point. [...] Basically I don't think your concerns wrt DELETE > > reliability/gurantees have much to do with this particular feature. > > The reason being that BIO_DELETE essentially tells the storage layer > > that whichever code "owns" the block in question (e.g. ZFS or UFS) > > has moved it into the free pool and will NEVER ever want to read its > > value back again (until it's written into again). > > No, it means that the contents of that block are no longer important and > that the lower layers *may* reclaim it. It does not mean that nobody > will ever try to read the block, nor does it guarantee that the block > will actually be reclaimed or zeroed. We cannot rely on the lower > layers to ensure that reading from a previously deleted block never > returns data that may have belonged to a different file. > > BTW, I've encountered CF cards (including the SanDisk card in my home > router) that freeze if issued a TRIM command. Furthermore, many CF, MMC > and SD cards, especially those marketed for use in digital cameras, > perform wear leveling "automagically" based on their own understanding > of the filesystem layout, and will therefore work poorly with anything > other than FAT (Kingston call it "optimized recording performance" in > their marketing literature). While these issues are relevant for BIO_DELETE, they aren't so much relevan= t for punching a hole in a file in a filesystem. The filesystem is the one that gets to decide whether and when to issue a BIO_DELETE (just as the lower layers get to decide what to do). A properly written filesystem will not issue a BIO_DELETE and then assume it will read back 0's. The whole point of the punch hole is to allow the filesystem to return the blocks to its free store. If that also happens to have the effect of causing a BIO_DELETE to go down, that's no different than deleting the file and having a BIO_DELETE go down for the resulting blocks that are freed. > > > Technically speaking on 100% correctly working os/hardware attempt to > > read block after it's been successfully BIO_DELETE'd could produce > > exception of some sort without any ill effects. > > If that were the case, it would never be safe to do > > # dd if=3D/dev/da0 of=3D/dev/da1 bs=3D4096 conv=3Dsparse > > which I'm sure you'll agree is not acceptable. > BIO_DELETE doesn't invalidate the LBA range, just its contents. LBAs are still required to read afterwards. This matches how the various standards dictate what the contents will be after whatever BIO_DELETE turns into. Maxim is simply wrong about this point, for this and many other reasons. > > [...] in this particular case of VOP_ALLOCATE(FALLOC_FL_PUNCH_HOLE), a > > filesystem in question is responsible for making sure the range that > > has been punched through reads 0, whether by making real logical hole > > in the file and/or by padding it with zeroes as needed. > > Is it really? > > Here are a few of our options for implementing FALLOC_FL_PUNCH_HOLE: > > a) create a filesystem-level hole in the disk image; > b) perform a), then issue a BIO_DELETE for the blocks that were > released; > c) perform a) or b), then zero the overspill if the requested range is > unaligned; > d) zero the entire range; > e) perform d) followed by either a) or b); > f) nothing at all. > I don't think f is an option. Unless it is OK to have random contents after creating a file and seeking some ways into and writing a byte. When you punch a hole in the file, you should get the same semantics as if you'd written up to just before the hole originally, then skipped to the end of the punched range and written the rest of the file. In Unix, that's well define= d to be 0's. It is undefined how those zeros are backed by the filesystem, or how much storage it takesup. A punch hole operation is a stronger statement about the contents after the fact than a BIO_DELETE operation. You are correct, though, that the decision to issue a BIO_DELETE is between the filesystem and the storage device. This makes a-e possible implementations, but some are stupider than others (which ones depend on the situation). Based on characteristics of both, the filesystem may return the blocks to its free store w/o doing anything further (if it frees them up at all). It could issue a BIO_DELETE on those blocks, if that is its policy. The device driver for th= e lower layers may return an error on the BIO_DELETE request or execute it faithfully. It cannot rely on it being a faster write zeros to the LBAs though. If it wants zeros, it has to write zeros. FreeBSD doesn't provide a way for the filesystem to know that the device implements BIO_DELETE as a guarnateed range of zeros after the operation completes, even if the device tells FreeBSD that information today as part of its IDENTIFY or INQUIRY data packets. > Now, consider the case of the guest OS in a VM issuing TRIM commands to > the emulated storage controller, which the hypervisor translates into a > FALLOC_FL_PUNCH_HOLE request for the corresponding range in the disk > image. Discuss the advantages and drawbacks of each option I listed > above for each of the 36 points in the space defined by the following > axes: > > - The disk image is: > - a preallocated file on a filesystem (or an md(4) device backed by a > preallocated file) > - a dynamically allocated file on a filesystem (or an md(4) device > backed by an unallocated file) > - a zvol > - a device > - The underlying storage's preferred block size is: > - small (e.g. 4 kB sectors on an AF drive) > - medium (e.g. 64 kB stripes on a RAID) > - large (e.g. 1 MB erase blocks on an SSD) > - The physical storage is: > - volatile > - solid-state > - electromechanical > > If you think the answer is the same in all cases, you are deluded. That's why these decisions are left to the stack. The only semantic that is required by the punch hole operation is that the filesystem return 0's on reads to that range. What the filesystem does to ensure this is up to the filesystem. As for md translating a BIO_DELETE into a PUNCH_HOLE, that's an acceptable thing for it to do (assuming we have a punch hole API). It is a stronger guarantee than is required by the BIO_DELETE API. However, PUNCH_HOLE should be implemented such that it is no slower than writes of zeros, and may be faster. Since md is doing writes of zeros today, this sounds like a possible win for those filesystems who implement the punch hole operation more efficiently than writing a block of zeros. And it may also allow the storage stack the chance to do an optimization that isn't present today. Warner From owner-freebsd-hackers@freebsd.org Tue Dec 8 15:44:57 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B35E39D46BF for ; Tue, 8 Dec 2015 15:44:57 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qg0-x22d.google.com (mail-qg0-x22d.google.com [IPv6:2607:f8b0:400d:c04::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 632291650 for ; Tue, 8 Dec 2015 15:44:57 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by qgcc31 with SMTP id c31so21528870qgc.3 for ; Tue, 08 Dec 2015 07:44:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=K3qiGpz0hWsri2eZI8zfD2m/S7xCNRHLR93sVRZqM9E=; b=OVvSyyW3EXNhP5iM1Y3rLUDg5QPQYGaUHeR2OUwBumpzwHEdOu5b5ye1EAzwim+KAO 4cUfc6RzHSAVHJFiLiQJFK2P20qelb8tzS7IjIQm7A36GIa8l1aB9QprR7kA/xzqNP+W QFJZufdJsLx49PSx3Ox42TaqJC4gKU3N6J45mrhlIZVl5Q1qBOpwMEuu1nIFXvXQ8ugV tiUYq3IqNExaLjX04Xk5IVkXjpGb02HSfIoTgZgN6RgGFjbNLFYrNGPyC13LwEnNMBQZ PVt4AlMtzVmt/qnXqz0OL3gTO1vHJIWFfdKs7Pf1gYJFzH25p1pojo00RIyFAnbX+H32 7Xuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=K3qiGpz0hWsri2eZI8zfD2m/S7xCNRHLR93sVRZqM9E=; b=JqL5NRpQk5YLhpF2uaQjyCmdnbZhzmjgmefAwE0STnXRe+4ji40qh6ER0wk8KlXmhs nngxQXgdxnd4stPYLZBeKha7QdkRMlRonDAKL4DQlhl0MTx4mvUrQypyRIG4ho9WoIuR O8zU4LRlI2KJmmIUL84Ad5OqZll2xcS6QVgx+ZlmkBJbp3SFuXBbCPRyDIFoAa16Nsuu 280K6p/ZgrJ5+qbjAZ0jadbfkEHay6UZbH4sFxSGQSjLksKXgKBGwriH1KQqSzy0FgO9 XXGhzbBA3n+mp2CfF+jOpa2n1GlbZ6OZLR03Ml1GQ3j+WeU/utXTrTWyDedVBH/yIPvd Hgjg== X-Gm-Message-State: ALoCoQkKmWti3+4w4xseKK/Pazh1ucYtzza7dxF4IepA3f+cejNuTi9CM5LdnUNZiAcJhFY1T0clDk34gpV4xLggh321dWxM0w== MIME-Version: 1.0 X-Received: by 10.140.99.86 with SMTP id p80mr210890qge.97.1449589496490; Tue, 08 Dec 2015 07:44:56 -0800 (PST) Sender: wlosh@bsdimp.com Received: by 10.140.27.181 with HTTP; Tue, 8 Dec 2015 07:44:56 -0800 (PST) X-Originating-IP: [2601:280:4900:3700:4d3f:8eba:ea86:7700] In-Reply-To: References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> Date: Tue, 8 Dec 2015 08:44:56 -0700 X-Google-Sender-Auth: VfmcLj4l2IODL3hjz4xPQqGLomQ Message-ID: Subject: Fwd: DELETE support in the VOP_STRATEGY(9)? From: Warner Losh To: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 15:44:57 -0000 [ forgot to cc hackers ] ---------- Forwarded message ---------- From: Warner Losh Date: Tue, Dec 8, 2015 at 8:22 AM Subject: Re: DELETE support in the VOP_STRATEGY(9)? To: Maxim Sobolev Cc: Dag-Erling Sm=C3=B8rgrav , Kirk McKusick , Pawel Jakub Dawidek On Tue, Dec 8, 2015 at 1:53 AM, Maxim Sobolev wrote: > 1. There was a complain from list admin about this being off-topic. I > think I've collected enough input from the interested parties so that I c= an > work on my patch before I feel like posting it publicly on some more > appropriate tech list for a wider review. > > 2. I did not really ignore it, it just that I did not have much to reply > at that point. But after being able to make some progress and looking at > the code in question I can probably comment on some of it now. Basically = I > don't think your concerns wrt DELETE reliability/gurantees have much to d= o > with this particular feature. The reason being that BIO_DELETE essentiall= y > tells the storage layer that whichever code "owns" the block in question > (e.g. ZFS or UFS) has moved it into the free pool and will NEVER ever wan= t > to read its value back again (until it's written into again). > Not quite true. It merely means the contents are no longer interesting. It doesn't mean they won't be read again. In FreeBSD BIO_DELETE has no post-condition always-true semantics. It might read back 0's. It might read back 1's. It might read back the data that was there before. All of these are allowed by various standards. FreeBSD even allows it to read back random data, though I'm not aware of any standards conforming hardware that would act that way. > Therefore, whatever geom driver does with that BIO_DELETE request truly > has no real effects on the file system operations, at least in theory, ev= en > if it effectively a NOP. > The filesystem cannot count on what happens with BIO_DELETE, or even if the underlying device supports it. > Technically speaking on 100% correctly working os/hardware attempt to rea= d > block after it's been successfully BIO_DELETE'd could produce exception o= f > some sort without any ill effects. > There's no basis for this assertion. BIO_DELETE doesn't destroy the LBA range. It merely advises the lower layers that the LBA range is no longer precious and can be discarded. Some lower layers implement this with a guarantee that it will read 0's or 1's until rewritten. Some say 'thank you for the hint' and provide no additional guarantees. > In reality, however, this is probably not a good idea to enforce that too > strictly except for debug/testing purposes. As for the alignment etc, in > this particular case of VOP_ALLOCATE(FALLOC_FL_PUNCH_HOLE), a filesystem = in > question is responsible for making sure the range that has been punched > through reads 0, whether by making real logical hole in the file and/or b= y > padding it with zeroes as needed. I've tested it with ZFS and it correctl= y > works on any range sizes/offsets, even when they aren't multiple of block > size. > At the filesystem level, it doesn't matter what happens in the main storage. Blocks that are no longer needed are returned to the free pool of the filesystem. Partial blocks are read -modifed- written for the appropriate ranges of zeros. A conforming implementation could even fail to return the blocks that were no longer needed and write zeros to them if it wanted. But what it does is filesystem dependent. If it does return them to the free pool, it could, if it wanted, issue a BIO_DELETE to the lower layers, assuming they have indicated they support it, and will have to deal with any error from doing so. Warner From owner-freebsd-hackers@freebsd.org Tue Dec 8 15:52:14 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 14A029D4D98; Tue, 8 Dec 2015 15:52:14 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id D31F21C5F; Tue, 8 Dec 2015 15:52:13 +0000 (UTC) (envelope-from des@des.no) Received: from desk.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 045CED20A; Tue, 8 Dec 2015 15:52:13 +0000 (UTC) Received: by desk.des.no (Postfix, from userid 1001) id 81C1D482CD; Tue, 8 Dec 2015 16:52:05 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Maxim Sobolev Cc: freebsd-current@freebsd.org, freebsd-hackers@freebsd.org, Kirk McKusick , kib@freebsd.org Subject: Re: posix_fallocate(2) && posix_fadvise(2) are somewhat broken References: Date: Tue, 08 Dec 2015 16:52:05 +0100 In-Reply-To: (Maxim Sobolev's message of "Tue, 8 Dec 2015 01:35:31 -0800") Message-ID: <868u55rl96.fsf@desk.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 15:52:14 -0000 Maxim Sobolev writes: > Hi, while working on some unrelated feature I've noticed that at least > those two system calls are not returning proper value (-1) on error. > Instead actual errno value is returned from the syscall verbatim, > i.e. posix_fadvise() returns 22 on EINVAL. That's how syscalls work. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@freebsd.org Tue Dec 8 16:43:04 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D11189D37DB for ; Tue, 8 Dec 2015 16:43:04 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 62D151DAC for ; Tue, 8 Dec 2015 16:43:04 +0000 (UTC) (envelope-from des@des.no) Received: from desk.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 167DED3A8; Tue, 8 Dec 2015 16:43:03 +0000 (UTC) Received: by desk.des.no (Postfix, from userid 1001) id 9B8F1482D4; Tue, 8 Dec 2015 17:42:58 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Warner Losh Cc: "freebsd-hackers\@freebsd.org" Subject: Re: Fwd: DELETE support in the VOP_STRATEGY(9)? References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> Date: Tue, 08 Dec 2015 17:42:58 +0100 In-Reply-To: (Warner Losh's message of "Tue, 8 Dec 2015 08:43:33 -0700") Message-ID: <864mfssxgt.fsf@desk.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 16:43:05 -0000 Warner Losh writes: > Dag-Erling Sm=C3=B8rgrav writes: > > Here are a few of our options for implementing FALLOC_FL_PUNCH_HOLE: > > > > a) create a filesystem-level hole in the disk image; > > b) perform a), then issue a BIO_DELETE for the blocks that were > > released; > > c) perform a) or b), then zero the overspill if the requested range is > > unaligned; > > d) zero the entire range; > > e) perform d) followed by either a) or b); > > f) nothing at all. > I don't think f is an option. Unless it is OK to have random contents > after creating a file and seeking some ways into and writing a > byte. When you punch a hole in the file, you should get the same > semantics as if you'd written up to just before the hole originally, > then skipped to the end of the punched range and written the rest of > the file. I didn't realize there was a spec, so I didn't know what the intended semantics were. > You are correct, though, that the decision to issue a BIO_DELETE is > between the filesystem and the storage device. This makes a-e possible > implementations, but some are stupider than others (which ones depend > on the situation). Each of them except f) is the optimal solution for at least one of the 36 cases I outlined, or 18 if you ignore the zvol and device points on the first axis. > > Discuss the advantages and drawbacks of each option I listed above > > for each of the 36 points in the space defined by the following > > axes: > > [...] > > If you think the answer is the same in all cases, you are deluded. > That's why these decisions are left to the stack. Define "stack". Do you mean the entire food chain from the hardware to the POSIX filesystem API? By design, no element in the stack has any knowledge of any other element, beyond the names and dimensions of its immediate consumers and suppliers (I find "producer" ambiguous). > The only semantic that is required by the punch hole operation is that > the filesystem return 0's on reads to that range. What the filesystem > does to ensure this is up to the filesystem. That's easy to say, but each option has advantages and disadvantages depending on information which is not necessarily available where it is needed. A filesystem-level hole results in fragmentation, which can have a huge performance impact on electromechanical storage but is negligible on solid-state storage. But the filesystem does not know whether the underlying storage is electromechanical or solid-state, nor does it know whether the user cares much about seek times (unless we introduce the heuristic "avoid creating holes unless the file already has them, in which case the userland probably does not care"). Then again, either the filesystem or the underlying storage *or both* may have copy-on-write semantics, in which case zeroing is worse than creating a hole. BTW, writing zeroes to NAND flash does not require erasing the block. I don't know whether SSDs take advantage of that to avoid unnecessarily reallocating or erasing a block, nor whether they automatically release and erase blocks that end up being completely zeroed. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@freebsd.org Tue Dec 8 17:08:05 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C57859D4D94 for ; Tue, 8 Dec 2015 17:08:05 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qg0-x229.google.com (mail-qg0-x229.google.com [IPv6:2607:f8b0:400d:c04::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 79E71175D for ; Tue, 8 Dec 2015 17:08:05 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by qgcc31 with SMTP id c31so25553557qgc.3 for ; Tue, 08 Dec 2015 09:08:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=9pd2dCJaKQLtecycxOQ0jUVTwzrYrN7IRxQCy0pBHRw=; b=dVWP/2Ji6Xgko/KSOxffvNm0uP3zEyQPSzAgKNKb3rpBCw2EQD2H+ccsNsJruHzNRk q8PnpNlPF5GSgWZOla+cArpLor5unilEvxgXxjSacZb8h5jY72jE6pVgYLd3/DXdrv5x XhSzven9iTAihCBkrFfo8yw46YyagxuJ7aJguOCCFCWjL36NRBQwoHZXaDjbmRfv+NQB 9Esrm01h30l2M2xMA32SrOL/iLIpGk92NiTYSh3Uc8H1tX+6R8zDayb/jhqoPhZ89C9t RrIoCgfqT+yiZDtHQ2Cngsh6t5Wa9/UUhXEHGuyTvEuDu5T5RuZw+kO34bnSu01mTfdJ smbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=9pd2dCJaKQLtecycxOQ0jUVTwzrYrN7IRxQCy0pBHRw=; b=GVJh0Yf0uugYFNS/M6bZB5oYV8PtvE+nEJtc8bGd6j5FKc1Rq0B0FQkBgfADs/1csT 8Z0nu9ulGRqp8o/jbg8B8p+ZqQTIyH+OHSwwv0djn5CCgUSuXc06J3YkQ7rXPnFZClhh 29ty55SPDL4yJsL1v7imYlZ621rr1zNG/OdpZzMPzx9hQOMId3FSoQTm7jJd7b4AM8cR Grz6q2H2NZUN9uO3Kn7Jvq4j+5hZB+bttS1BzjQxt0+1Du6GqWBVBy2W35pd8UErsbaQ B1alqrqBIK8w+Sk/8pf+s00T4oGKowogdmYbC46EawIlnZf3hnGps8u6M3dWBEoqaIyO +rYA== X-Gm-Message-State: ALoCoQnz5CGKH4ERge5mONZ31d8/D4t7+oFwvPhEZ1GBMYKlmAzTWhIQbeAqNlW1+eSKJFFq1MSkfZVDPKmctUitWh2cWafw5g== MIME-Version: 1.0 X-Received: by 10.140.40.38 with SMTP id w35mr903462qgw.52.1449594463066; Tue, 08 Dec 2015 09:07:43 -0800 (PST) Sender: wlosh@bsdimp.com Received: by 10.140.27.181 with HTTP; Tue, 8 Dec 2015 09:07:42 -0800 (PST) X-Originating-IP: [2601:280:4900:3700:4d3f:8eba:ea86:7700] In-Reply-To: <864mfssxgt.fsf@desk.des.no> References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> Date: Tue, 8 Dec 2015 10:07:42 -0700 X-Google-Sender-Auth: -UjaAoO15eSCCqfGcXqKn-VCNUY Message-ID: Subject: Re: Fwd: DELETE support in the VOP_STRATEGY(9)? From: Warner Losh To: =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 17:08:05 -0000 On Tue, Dec 8, 2015 at 9:42 AM, Dag-Erling Sm=C3=B8rgrav wrote= : > Warner Losh writes: > > Dag-Erling Sm=C3=B8rgrav writes: > > > Here are a few of our options for implementing FALLOC_FL_PUNCH_HOLE: > > > > > > a) create a filesystem-level hole in the disk image; > > > b) perform a), then issue a BIO_DELETE for the blocks that were > > > released; > > > c) perform a) or b), then zero the overspill if the requested range i= s > > > unaligned; > > > d) zero the entire range; > > > e) perform d) followed by either a) or b); > > > f) nothing at all. > > I don't think f is an option. Unless it is OK to have random contents > > after creating a file and seeking some ways into and writing a > > byte. When you punch a hole in the file, you should get the same > > semantics as if you'd written up to just before the hole originally, > > then skipped to the end of the punched range and written the rest of > > the file. > > I didn't realize there was a spec, so I didn't know what the intended > semantics were. I am assuming the semantics are the same as the Linux thing of the same name. > > You are correct, though, that the decision to issue a BIO_DELETE is > > between the filesystem and the storage device. This makes a-e possible > > implementations, but some are stupider than others (which ones depend > > on the situation). > > Each of them except f) is the optimal solution for at least one of the > 36 cases I outlined, or 18 if you ignore the zvol and device points on > the first axis. True. > > > > Discuss the advantages and drawbacks of each option I listed above > > > for each of the 36 points in the space defined by the following > > > axes: > > > [...] > > > If you think the answer is the same in all cases, you are deluded. > > That's why these decisions are left to the stack. > > Define "stack". Do you mean the entire food chain from the hardware to > the POSIX filesystem API? By design, no element in the stack has any > knowledge of any other element, beyond the names and dimensions of its > immediate consumers and suppliers (I find "producer" ambiguous). Also true. The stack makes the best choice it can at each level and passes the rest on down. There are administrative overrides, however, for the default actions (like sending down a BIO_DELETE). > > The only semantic that is required by the punch hole operation is that > > the filesystem return 0's on reads to that range. What the filesystem > > does to ensure this is up to the filesystem. > > That's easy to say, but each option has advantages and disadvantages > depending on information which is not necessarily available where it is > needed. A filesystem-level hole results in fragmentation, which can > have a huge performance impact on electromechanical storage but is > negligible on solid-state storage. It may result in fragmentation. UFS has techniques to cope, however. ZFS doesn't matter since it is log based and writing zeros would also produce a non-local copy. > But the filesystem does not know > whether the underlying storage is electromechanical or solid-state, nor > does it know whether the user cares much about seek times (unless we > introduce the heuristic "avoid creating holes unless the file already > has them, in which case the userland probably does not care"). Actually, the filesystem does know. Or has some knowledge of what is supported and what isn't. BIO_DELETE support is a strong indicator of a flash or other log-type system. Then > again, either the filesystem or the underlying storage *or both* may > have copy-on-write semantics, in which case zeroing is worse than > creating a hole. > It may have that implementation. That's what administrative controls are for. > BTW, writing zeroes to NAND flash does not require erasing the block. I > don't know whether SSDs take advantage of that to avoid unnecessarily > reallocating or erasing a block, nor whether they automatically release > and erase blocks that end up being completely zeroed. Turns out this hasn't been true for 4 or 5 generations of NAND. You cannot program pages in a block twice. That's not a supported operation, and canno= t possibly work for multi-level cell technology. Flash drives never use this fact because this fact simply won't work with anything other than first or secon= d generation single bit per cell technology. These days, NAND must be written from first page to last. And it is strongly advised that the DWELL time in the erase state be as short as possible. And it is also strongly advised that the writing be done as quickly as possible and that active counter measure be taken when you can't write it within a maximum amount of time (such as garbage collecting data forward to meet the timing constraints). You can violate these guidelines from time to time, but then the storage retention cannot be guaranteed to meet vendor specs. The firmware in recent generations of planar technologies makes certain assumptions to maximize life of the data and if you violate them (like giving the data time to decay (NAND cells are best thought of as tiny capacitors which lose charge over time) before the next cells are programmed (disturbing these cells in a predictable way), you'll get bit errors). And the new 3-d NAND throws a whole new set of constraints that are different from these constraints that plagued old planar NAND arrangements. SSDs have a log structure under the covers anyway. As you write data, the data gets laid down into a log. When you write new data, a note of the old obsolete data is made, but it isn't erased until either the entire block that held the data is invalidated, or the data is garbage collected forward to a new page of NAND in a new erase block. A BIO_DELETE turns into some flavor of TRIM or DELETE operation under the covers. The drive's FTL (Flash Translation Layer) then uses this to invalidate blocks in its internal tables. Maybe this will actually erase the data, maybe not, but that operation is decoupled from the TRIM completing. The invalidation occurs both in some kind of 'used mask' as well as in the LBA to physical mapping so when requests for the LBA come in later all 0's can be returned (for newer drives, complying with the latest standards). SD cards are a whole other domain of hacks and optimizations. Since I don't have direct experience making them and writing that firmware, I can't comment on what they are, other than to say that telling the drive the contents are no longer needed, or that you are about to write multiple blocks can help them out a lot. As technologies evolve, we should take advantage of them. The old assumptions, like writing 0's to NAND can be done without erasing, may prove to be unwise. This is why we let the drive tell us about their support for TRIMing technologies and let the filesystem decide when the best time to do that operation is. Sometimes these new technologies can map to existing facilities, other times they can't. Warner From owner-freebsd-hackers@freebsd.org Tue Dec 8 17:32:49 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47ACB9D438B; Tue, 8 Dec 2015 17:32:49 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-pa0-x232.google.com (mail-pa0-x232.google.com [IPv6:2607:f8b0:400e:c03::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1F8E01EF6; Tue, 8 Dec 2015 17:32:49 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by pacej9 with SMTP id ej9so15179357pac.2; Tue, 08 Dec 2015 09:32:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=X6WPJ2asx03m+jFClRzP+If50bXb3F0d3Hd1JQRq9vU=; b=iuXsHLcs4jdqWRezpF+HE3kSNN41W17v8D4KMZXL60YkjeFQ3OIJQBUcW4uNWgbFlv bfYJjSSd0YW0m8YIR/XRGqXbBrS3pUvh0+pRvlB9JAiSFC2+kPoCQUTq2TeE2TBoUsva UFDnA1J9HeP87HLWj02Fw4BVHx35KuXJyj2Qn1PBrM269UE1wispepIa07K9azNSUkRG S7aTjAq8T2EiC2wU4tQG9g0q/pdBFhHkNBU3B7qjouKaNlxALuPRSmbFNKvr0s+uirxI 6+8/qfS2P0rlGmke6PaDTnZwuxtarN7zew5px3oW3uoQqZ98k51MTTIywPL0iZDedxIq 2gxw== X-Received: by 10.66.224.165 with SMTP id rd5mr1508464pac.73.1449595968727; Tue, 08 Dec 2015 09:32:48 -0800 (PST) Received: from raichu ([104.232.114.184]) by smtp.gmail.com with ESMTPSA id 71sm6037676pfj.28.2015.12.08.09.32.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 08 Dec 2015 09:32:48 -0800 (PST) Sender: Mark Johnston Date: Tue, 8 Dec 2015 09:32:36 -0800 From: Mark Johnston To: Maxim Sobolev Cc: freebsd-current@freebsd.org, freebsd-hackers@freebsd.org, Kirk McKusick , kib@freebsd.org Subject: Re: posix_fallocate(2) && posix_fadvise(2) are somewhat broken Message-ID: <20151208173236.GA11078@raichu> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 17:32:49 -0000 On Tue, Dec 08, 2015 at 01:35:31AM -0800, Maxim Sobolev wrote: > Hi, while working on some unrelated feature I've noticed that at least > those two system calls are not returning proper value (-1) on error. > Instead actual errno value is returned from the syscall verbatim, > i.e. posix_fadvise() returns 22 on EINVAL. Attached patch fixes that > problem, however I am not sure if I need to assign td->td_retval[0] at all, > those two operations by design never return anything but -1 on error and 0 > on success. Can someone comment on this? Thanks! This behaviour is documented and specified by POSIX. I'm not sure why these syscalls are inconsistent with everything else, but the current implementation is correct. From owner-freebsd-hackers@freebsd.org Tue Dec 8 17:37:24 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FB319D487B for ; Tue, 8 Dec 2015 17:37:24 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: from mail-wm0-x234.google.com (mail-wm0-x234.google.com [IPv6:2a00:1450:400c:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 09720159E for ; Tue, 8 Dec 2015 17:37:24 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: by wmww144 with SMTP id w144so190320327wmw.1 for ; Tue, 08 Dec 2015 09:37:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sippysoft-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=aT2Mjvn64OpVhHB9XBny4RRZsVe/z+J5b+ycf24+Buk=; b=eV8nhDoWsNIWz29v3wffotBrApvqpI69V/vMM03gNPWIDcF0INYw9M+nI7vKXyL5GU Ac3+H4oMpMT9uMTlOw/UIhl6mKCMPMw4M3dX1Q2kHPn7Ge1S9sUGEotc/XUWk6GSX57Y 39jnYv7PoBtqm63M8PWaSUuCl8z46lrkKhyNnkVT0PcDbSv7EL2LtC9CVDgZTyQCovle 0HJFqUIIm8dggx3gXGPXikmz3S+Qh3zzol3lKn+m1IlpIUHSZujLKuST+wQMOmQV7/RP OqhPKvSt4ZDkjtm70RrH+AYWt5QV+KalXnLXwQTLm2CuIkKhc4Kt7lARvfEh+y0sSUhk +95A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=aT2Mjvn64OpVhHB9XBny4RRZsVe/z+J5b+ycf24+Buk=; b=Xtc8TXx7rKkqN1rnV7VHkAoRT8uJrUzbuGCSEz+l+ITUUTurqhSKbT/fcpwf1olBRS BLsTr370QD0F5IiwcMZJCDzCupTY4tWEVjZjfR50SclFXfwx2aBE22Vs8ZGaIGahI1eh iPIDvH596hxCQIgWWYwexz+fTJQCjdtnIgyhd+MMtgGlslYPh3QUSs6dX6BUcCiERFc4 EfDpZL7+2eFGKTC2SBH4VaJIKZjwyxYp1uCV1MzFYv17dkhwpT3y26IGphz3iakfRY9y sioJs5SJlrE1AhwcrtwIPJYMNE4bVhAkZTJOTGVa/yOqtFr0/N4IXGzWp1GrrMi6cB0Y Id1g== X-Gm-Message-State: ALoCoQlgiIic7JKLnKVl4qiysWYbvWOKuHRmlqD5OIww5ctfzLv/WHxaSfaNkWZBzSCjUAOWTt5On04a7E3XnOJpyPBK5KQRg7aSbJxO8zMkTos40KGYz1Y= MIME-Version: 1.0 X-Received: by 10.194.185.42 with SMTP id ez10mr962196wjc.82.1449596241214; Tue, 08 Dec 2015 09:37:21 -0800 (PST) Sender: sobomax@sippysoft.com Received: by 10.27.39.135 with HTTP; Tue, 8 Dec 2015 09:37:21 -0800 (PST) In-Reply-To: <201512081701.tB8H1ivY009763@hergotha.csail.mit.edu> References: <201512081701.tB8H1ivY009763@hergotha.csail.mit.edu> Date: Tue, 8 Dec 2015 09:37:21 -0800 X-Google-Sender-Auth: vwKGwcdtcjd5E8BYRUabTjnjh00 Message-ID: Subject: Re: posix_fallocate(2) && posix_fadvise(2) are somewhat broken From: Maxim Sobolev To: Garrett Wollman Cc: freebsd-current@freebsd.org, freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 17:37:24 -0000 Then it's documentation bug or maybe some discrepancy between SUS and POSIX. $ man posix_fadvise RETURN VALUES The posix_fadvise() function returns the value 0 if successful; otherwise the value -1 is returned and the global variable errno is set to indicate the error. STANDARDS The posix_fadvise() interface conforms to IEEE Std 1003.1-2001 (``POSIX.1''). HISTORY The posix_fadvise() system call first appeared in FreeBSD 9.1. On Tue, Dec 8, 2015 at 9:01 AM, Garrett Wollman < wollman@hergotha.csail.mit.edu> wrote: > In article NFGCy_w@mail.gmail.com> > sobomax@freebsd.org writes: > > >Hi, while working on some unrelated feature I've noticed that at least > >those two system calls are not returning proper value (-1) on error. > >Instead actual errno value is returned from the syscall verbatim, > > That is what the specification requires. > > RETURN VALUE > Upon successful completion, posix_fadvise( ) shall return > zero; otherwise, an error number shall be returned to > indicate the error. > > (Quote from SUSv7 p. 1410, lines 46221-46223.) > > -GAWollman > > From owner-freebsd-hackers@freebsd.org Tue Dec 8 17:42:49 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 202EE9D4EB8 for ; Tue, 8 Dec 2015 17:42:49 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay05.ispgateway.de (smtprelay05.ispgateway.de [80.67.18.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DC1101E39 for ; Tue, 8 Dec 2015 17:42:48 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from [78.35.183.85] (helo=fabiankeil.de) by smtprelay05.ispgateway.de with esmtpsa (TLSv1.2:AES128-GCM-SHA256:128) (Exim 4.84) (envelope-from ) id 1a6MHi-0000d5-Hy for freebsd-hackers@FreeBSD.org; Tue, 08 Dec 2015 18:42:42 +0100 Date: Tue, 8 Dec 2015 18:40:07 +0100 From: Fabian Keil To: Freebsd hackers list Subject: Re: How to get the deterministic result for FreeBSD tar(1)? Message-ID: <20151208184007.3080da7b@fabiankeil.de> In-Reply-To: <5666B828.5000306@rawbw.com> References: <5666B828.5000306@rawbw.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/_d.KNCs67RBAR5IAKg=XBE6"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 17:42:49 -0000 --Sig_/_d.KNCs67RBAR5IAKg=XBE6 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Yuri wrote: > I have two identical directories (no diffs, all identical mtime=20 > attributes) compressed by this command: > find dir -print0 | LC_ALL=3DC sort -z | tar cf archive.tgz --format=3Dbsd= tar=20 > --no-recursion --null -T - >=20 > The results are different: 3 files out of 10,000 have pax attributes set= =20 > that are different: > - 27 ctime=3D1449566560.642715 > +27 ctime=3D1449566903.167521 [...]=20 > So I have two questions: > 1. How do I actually achieve the output determinism for tar(1)? You can use an mtree spec to set fake timestamps etc. For an example see patch 12 in this set: https://www.fabiankeil.de/sourcecode/electrobsd/reproducible-build-goo-r291= 706-29246dc.diff Patch 5 contains a script to regenerate tar files with normalized timestamps (and some other attributes) but of course generating the files twice is a bit silly if it can be avoided. > 2. Is there an agreement that this is a bug that too long or non-ASCII=20 > path name triggers the leakage of ctime into a tar file? My general impression is that large parts of tar's behaviour are undefined (due to lack of documentation) and it's not obvious to me that this isn't one of them. Fabian --Sig_/_d.KNCs67RBAR5IAKg=XBE6 Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlZnFfgACgkQBYqIVf93VJ1JKwCgjigFJo/uCBufdBtn1syd4RK2 3rkAniiF5gYg/sts1h8L1lvkQVEguXsh =DN/n -----END PGP SIGNATURE----- --Sig_/_d.KNCs67RBAR5IAKg=XBE6-- From owner-freebsd-hackers@freebsd.org Tue Dec 8 17:43:11 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7B66A9D4F09; Tue, 8 Dec 2015 17:43:11 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 15F431F31; Tue, 8 Dec 2015 17:43:10 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id tB8Hgxo7057780 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 8 Dec 2015 19:43:00 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua tB8Hgxo7057780 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id tB8Hgx5T057779; Tue, 8 Dec 2015 19:42:59 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 8 Dec 2015 19:42:59 +0200 From: Konstantin Belousov To: Dag-Erling Sm??rgrav Cc: Maxim Sobolev , freebsd-current@freebsd.org, freebsd-hackers@freebsd.org, Kirk McKusick Subject: Re: posix_fallocate(2) && posix_fadvise(2) are somewhat broken Message-ID: <20151208174259.GA82577@kib.kiev.ua> References: <868u55rl96.fsf@desk.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <868u55rl96.fsf@desk.des.no> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 17:43:11 -0000 On Tue, Dec 08, 2015 at 04:52:05PM +0100, Dag-Erling Sm??rgrav wrote: > Maxim Sobolev writes: > > Hi, while working on some unrelated feature I've noticed that at least > > those two system calls are not returning proper value (-1) on error. > > Instead actual errno value is returned from the syscall verbatim, > > i.e. posix_fadvise() returns 22 on EINVAL. > > That's how syscalls work. No, this is not how typical syscalls work, but is how the posix_fallocate() and posix_fadvise() are specified by Posix. The patch is wrong, see also r261080 and r288640. From owner-freebsd-hackers@freebsd.org Tue Dec 8 17:54:21 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E73859D4968 for ; Tue, 8 Dec 2015 17:54:21 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 776441FFF for ; Tue, 8 Dec 2015 17:54:21 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id tB8HsEJ8060222 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 8 Dec 2015 19:54:15 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua tB8HsEJ8060222 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id tB8HsETs060221; Tue, 8 Dec 2015 19:54:14 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 8 Dec 2015 19:54:14 +0200 From: Konstantin Belousov To: Warner Losh Cc: "freebsd-hackers@freebsd.org" Subject: Re: Fwd: DELETE support in the VOP_STRATEGY(9)? Message-ID: <20151208175414.GB82577@kib.kiev.ua> References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 17:54:22 -0000 On Tue, Dec 08, 2015 at 08:43:33AM -0700, Warner Losh wrote: > [ forgot to cc hackers ] > ---------- Forwarded message ---------- > From: Warner Losh > Date: Tue, Dec 8, 2015 at 8:41 AM > Subject: Re: DELETE support in the VOP_STRATEGY(9)? > To: Dag-Erling Sm??rgrav > > > On Tue, Dec 8, 2015 at 4:06 AM, Dag-Erling Sm??rgrav wrote: > > > Maxim Sobolev writes: > > > Dag-Erling Sm??rgrav writes: > > > > 1) why did you take this off the list? > > > There was a complain from list admin about this being off-topic. > > > > Yes, and Eitan moved the discussion to hackers@. It should have stayed > > there. > > > > > > 2) why did you even bother to cc: me if you were going to competely > > > > ignore everything I said anyway? > > > I did not really ignore it, it just that I did not have much to reply > > > at that point. [...] Basically I don't think your concerns wrt DELETE > > > reliability/gurantees have much to do with this particular feature. > > > The reason being that BIO_DELETE essentially tells the storage layer > > > that whichever code "owns" the block in question (e.g. ZFS or UFS) > > > has moved it into the free pool and will NEVER ever want to read its > > > value back again (until it's written into again). > > > > No, it means that the contents of that block are no longer important and > > that the lower layers *may* reclaim it. It does not mean that nobody > > will ever try to read the block, nor does it guarantee that the block > > will actually be reclaimed or zeroed. We cannot rely on the lower > > layers to ensure that reading from a previously deleted block never > > returns data that may have belonged to a different file. > > > > BTW, I've encountered CF cards (including the SanDisk card in my home > > router) that freeze if issued a TRIM command. Furthermore, many CF, MMC > > and SD cards, especially those marketed for use in digital cameras, > > perform wear leveling "automagically" based on their own understanding > > of the filesystem layout, and will therefore work poorly with anything > > other than FAT (Kingston call it "optimized recording performance" in > > their marketing literature). > > > While these issues are relevant for BIO_DELETE, they aren't so much relevant > for punching a hole in a file in a filesystem. The filesystem is the one > that > gets to decide whether and when to issue a BIO_DELETE (just as the lower > layers get to decide what to do). A properly written filesystem will not > issue > a BIO_DELETE and then assume it will read back 0's. The whole point of > the punch hole is to allow the filesystem to return the blocks to its free > store. If that also happens to have the effect of causing a BIO_DELETE > to go down, that's no different than deleting the file and having a > BIO_DELETE > go down for the resulting blocks that are freed. Exactly. I completely agree with the statement above, and think that this is the only thing that should be implemented. There could be a request, most likely an filesystem-level ioctl, which punches holes in the file. Its effect on the file state should be the same as if the seek was done between writes, if the filesystem supports the ioctl. More, if supported, the lseek(SEEK_HOLE/SEEK_DATA) behaviour should be consistent with the request. The later might mean that we should restrict the interface to only accept ranges at block boundaries. Note that for UFS, it is automatically (due to the implementation at the lower levels) that BIO_DELETE might be issued for the fully freed blocks, when it is supported by the volume and safe from the PoV of the filesystem safety. In other words, the behaviour there should be the same as for the blocks freed due to the file truncation or final unlinking. UFS has a known quirk there, it does not allow hole to extend to the end of file, the last fragment or block must be allocated. I once did patched kernel and fsck to remove this restriction, but I probably lost the patch. From owner-freebsd-hackers@freebsd.org Tue Dec 8 17:59:52 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB2749D4DB3 for ; Tue, 8 Dec 2015 17:59:52 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: from mail-wm0-x230.google.com (mail-wm0-x230.google.com [IPv6:2a00:1450:400c:c09::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 534F314B2 for ; Tue, 8 Dec 2015 17:59:52 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: by wmec201 with SMTP id c201so40196636wme.1 for ; Tue, 08 Dec 2015 09:59:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sippysoft-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=QPLy10ZJP5pFmUQO/g2lwDAKV5nOfehBd2SRL0SqawY=; b=z59Nxo7RlQDBhIU69VZ4ls+e9OPJqHIDbG15uIu+Aj8Jk/biFMbMWADyzF1TnotrSi 68YkfDM//rPI4b/hucAKz4Ek1ttWKadd6qkWI/fGMkeV57RY8WW/mJbJBN4DSSvNeTCg nk+iqRIylsL8+Noy9gX/7B8CY3kbXLpHaUsXZKJ5cRRuZAC27bGU+LTua1BxuRKTrzYr mTHP0o03w5zsmRWMl2yrRrxIIp/3Hl3zXlulyClxf0ckDZBlJaQDkj/kx436d4t60tS6 Z9tWVqkYdoGuLcqPkTcDOc62WulD/aop4+LeSGPxeql1Aq1bqqK6Y1OPzEltmNYI6VD0 PEHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=QPLy10ZJP5pFmUQO/g2lwDAKV5nOfehBd2SRL0SqawY=; b=Riv6XmMF71PSfBrgWWvnqVw/VOwh0fvwuTZvoEndHilOfsTEfg+BCqXoxHEVYIDFbX nqCQR/gXGCFs+//fepYFxwf7htJjBms3IkK+Q7P8alIa2TyKDLTF0sdn75w7dGuj0VNg g6/XNRIwyM98VYOQEGOj8gCOdQKYkU9ZcCn8Th4eO4tvY/5eZzawZ7Qxwsdne2bldXMA mql8CxYFL/fRzhuZWPFocNeHP9Mclqey9RFnfKg62kvyaYe8DV4pr2p488dVvPQSmQNw V40WPAPRYsmhe7SYQPh9nDYk7c335SoIM2SRxoENgxT/j6bY+UAx8tZobBzLeSmCdDUL Rhvg== X-Gm-Message-State: ALoCoQl3QwmdvtvUTTBE+2OKFl9tOaZmf8arczkK/1r6gPX62Ms3zgaoODhje9u2eS640qJzONfmV7slB9qJwMPV6CVici0Ic+BuMS7uQ29OxXUBg25HL3c= MIME-Version: 1.0 X-Received: by 10.194.185.42 with SMTP id ez10mr1089745wjc.82.1449597590817; Tue, 08 Dec 2015 09:59:50 -0800 (PST) Sender: sobomax@sippysoft.com Received: by 10.27.39.135 with HTTP; Tue, 8 Dec 2015 09:59:50 -0800 (PST) In-Reply-To: <20151208174259.GA82577@kib.kiev.ua> References: <868u55rl96.fsf@desk.des.no> <20151208174259.GA82577@kib.kiev.ua> Date: Tue, 8 Dec 2015 09:59:50 -0800 X-Google-Sender-Auth: bMK13KdHMtYrxp7Jo9MAO3VtXQU Message-ID: Subject: Re: posix_fallocate(2) && posix_fadvise(2) are somewhat broken From: Maxim Sobolev To: Konstantin Belousov Cc: freebsd-current@freebsd.org, freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 17:59:52 -0000 Ah, ok, I see now. It's been broken and still broken in 9.x/10.x, already fixed in trunk and I been just reading wrong manpage. Thanks for the pointer, on a related note those fixes should probably be MFCed into 10.3 if it has not been already. On Tue, Dec 8, 2015 at 9:42 AM, Konstantin Belousov wrote: > On Tue, Dec 08, 2015 at 04:52:05PM +0100, Dag-Erling Sm??rgrav wrote: > > Maxim Sobolev writes: > > > Hi, while working on some unrelated feature I've noticed that at least > > > those two system calls are not returning proper value (-1) on error. > > > Instead actual errno value is returned from the syscall verbatim, > > > i.e. posix_fadvise() returns 22 on EINVAL. > > > > That's how syscalls work. > > No, this is not how typical syscalls work, but is how the posix_fallocate() > and posix_fadvise() are specified by Posix. The patch is wrong, see also > r261080 and r288640. > > From owner-freebsd-hackers@freebsd.org Tue Dec 8 18:44:44 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C2D69D42D6 for ; Tue, 8 Dec 2015 18:44:44 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 5184F1639 for ; Tue, 8 Dec 2015 18:44:44 +0000 (UTC) (envelope-from des@des.no) Received: from desk.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 50279D6C3; Tue, 8 Dec 2015 18:44:42 +0000 (UTC) Received: by desk.des.no (Postfix, from userid 1001) id E29D7482E3; Tue, 8 Dec 2015 19:44:38 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Warner Losh Cc: "freebsd-hackers\@freebsd.org" Subject: Re: Fwd: DELETE support in the VOP_STRATEGY(9)? References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> Date: Tue, 08 Dec 2015 19:44:38 +0100 In-Reply-To: (Warner Losh's message of "Tue, 8 Dec 2015 10:07:42 -0700") Message-ID: <86wpsord9l.fsf@desk.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 18:44:44 -0000 Warner Losh writes: > Dag-Erling Sm=C3=B8rgrav writes: > > But the filesystem does not know whether the underlying storage is > > electromechanical or solid-state, nor does it know whether the user > > cares much about seek times (unless we introduce the heuristic > > "avoid creating holes unless the file already has them, in which > > case the userland probably does not care"). > Actually, the filesystem does know. Or has some knowledge of what > is supported and what isn't. BIO_DELETE support is a strong indicator > of a flash or other log-type system. The filesystem can ask the layer below if BIO_DELETE is supported, but should not assume anything about what it means. For instance, I could write a gnop-like module that translates BIO_DELETE into an all-zeroes BIO_WRITE and passes everything else unmodified. It would provide a stronger guarantee than, say, SATA TRIM but would also have a completely different performance profile (even on SSDs, since it would do its work synchronously whereas TRIM works asynchronously). Anyway, my point is that Maxim needs to revise his assumptions. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@freebsd.org Tue Dec 8 18:52:28 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B67E79D316B for ; Tue, 8 Dec 2015 18:52:28 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 540B61529 for ; Tue, 8 Dec 2015 18:52:27 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by wmuu63 with SMTP id u63so192791246wmu.0 for ; Tue, 08 Dec 2015 10:52:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-type:content-transfer-encoding; bh=mU3kRmJ/3/sBJa2nYH+e/17ma1hDA/6FixHoARuewAY=; b=vX9yH5sE1QGSsd+rnyclKsx7cTyLBFoGnG5IqVwoPdW1rfb4pNGD+x1mj4T5zXCUOK X1U08S8Z3m/H/45NXzH8cP+yHPivUeipJbcR+s7EvHPg7gfwvcTE1GlevMC3SCY3uc9e lymEglsscqjpnjEbIktd19yrBBsqCOe1Gc9mVRKjxC0XckH+hqVsXp0egwq+mIVq8Ilx THQbKpNbMpfr+GLA8UyCYPsHJYYPZfP4BxIjzrI3ZCogPtd+d7nICHc7cIoOFlQY8Sz3 Ds8tHwv4+fHNtdMdYIA3YUBTZ1KkBNxuR1c/+c90zV44dSn8ESEiZywbz+TqZAqs3st8 AV5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=mU3kRmJ/3/sBJa2nYH+e/17ma1hDA/6FixHoARuewAY=; b=KLmKNijlcddc80b9z+pZlwmfZVFvCTpn2AKQOl4ze162sz7BSSf3ReeryX0N6Ymv6W 7ELFPkuhuA6OOSJrzQuzaOUpfTVbkRU39AxJ14FHsb3G+6C5Z1oI1vN08vAt5plbZ3au 0nogIEJzSaB9FxrfuczDqQu8pMSX/8VJX4FaFrT46X0zNzmeWgfXiW/fFaOri3EGytiB eh9GVBikpX0U50zu45F6a8gdsFFE0cWQZcPuzMrJJ8TF7/vmbewl/wT6BidKjLwtDAgT uDXgaYvbGiozGA3Ar8KHNZqcwPyGmPCff59ZkaB2jctbqb1i6LI9CoCImxZZS6Ldk40g IMIQ== X-Gm-Message-State: ALoCoQnwiuPN9ypvz/AvKhTi3gfstr57/aXOvJIjg/kfUd2tgY1J3VIxQhxz5Kzaz3UhO6PGVvAOIhWXAFQBJgil54qWHbvp8g== X-Received: by 10.194.134.102 with SMTP id pj6mr1307903wjb.23.1449600745047; Tue, 08 Dec 2015 10:52:25 -0800 (PST) Received: from [10.10.1.58] (liv3d.labs.multiplay.co.uk. [82.69.141.171]) by smtp.gmail.com with ESMTPSA id uz5sm4186446wjc.8.2015.12.08.10.52.23 for (version=TLSv1/SSLv3 cipher=OTHER); Tue, 08 Dec 2015 10:52:24 -0800 (PST) Subject: Re: Fwd: DELETE support in the VOP_STRATEGY(9)? To: freebsd-hackers@freebsd.org References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> From: Steven Hartland Message-ID: <566726ED.2010709@multiplay.co.uk> Date: Tue, 8 Dec 2015 18:52:29 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <86wpsord9l.fsf@desk.des.no> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 18:52:28 -0000 On 08/12/2015 18:44, Dag-Erling Smørgrav wrote: > Warner Losh writes: >> Dag-Erling Smørgrav writes: >>> But the filesystem does not know whether the underlying storage is >>> electromechanical or solid-state, nor does it know whether the user >>> cares much about seek times (unless we introduce the heuristic >>> "avoid creating holes unless the file already has them, in which >>> case the userland probably does not care"). >> Actually, the filesystem does know. Or has some knowledge of what >> is supported and what isn't. BIO_DELETE support is a strong indicator >> of a flash or other log-type system. > The filesystem can ask the layer below if BIO_DELETE is supported, but > should not assume anything about what it means. For instance, I could > write a gnop-like module that translates BIO_DELETE into an all-zeroes > BIO_WRITE and passes everything else unmodified. It would provide a > stronger guarantee than, say, SATA TRIM but would also have a completely > different performance profile (even on SSDs, since it would do its work > synchronously whereas TRIM works asynchronously). > > Anyway, my point is that Maxim needs to revise his assumptions. Just to clarify most consumer devices process TRIM synchronously, not asynchronously. Your example isn't actually just an example CAM scsi_da has a number of different ways it can process BIO_DELETE: * ATA TRIM * SCSI UMAP * Write Same 16 * Write Same 10 * Zero So you example is actually exists in practice in the FreeBSD code base ;-) Regards Steve From owner-freebsd-hackers@freebsd.org Tue Dec 8 18:54:14 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58E8D9D3382; Tue, 8 Dec 2015 18:54:14 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 239881871; Tue, 8 Dec 2015 18:54:13 +0000 (UTC) (envelope-from des@des.no) Received: from desk.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 2CEE5D6FD; Tue, 8 Dec 2015 18:54:13 +0000 (UTC) Received: by desk.des.no (Postfix, from userid 1001) id AA786482E5; Tue, 8 Dec 2015 19:54:06 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Konstantin Belousov Cc: Kirk McKusick , Maxim Sobolev , freebsd-current@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: posix_fallocate(2) && posix_fadvise(2) are somewhat broken References: <868u55rl96.fsf@desk.des.no> <20151208174259.GA82577@kib.kiev.ua> Date: Tue, 08 Dec 2015 19:54:06 +0100 In-Reply-To: <20151208174259.GA82577@kib.kiev.ua> (Konstantin Belousov's message of "Tue, 8 Dec 2015 19:42:59 +0200") Message-ID: <86poygrctt.fsf@desk.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 18:54:14 -0000 Konstantin Belousov writes: > Dag-Erling Sm=C3=B8rgrav writes: > > Maxim Sobolev writes: > > > Hi, while working on some unrelated feature I've noticed that at least > > > those two system calls are not returning proper value (-1) on error. > > > Instead actual errno value is returned from the syscall verbatim, > > > i.e. posix_fadvise() returns 22 on EINVAL. > > That's how syscalls work. > No, this is not how typical syscalls work, but is how the posix_fallocate= () > and posix_fadvise() are specified by Posix. The patch is wrong, see also > r261080 and r288640. Umm, I can't find the code ATM but syscalls store the actual return value in td_retval and return 0 or EWHATEVER and the syscall wrapper handles the translation. If that's not what Maxim was talking about, then please ignore me. Anyway, happy to hear that the X/Open group have found a new way to screw people over. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@freebsd.org Tue Dec 8 18:58:28 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 80EEF9D37D4 for ; Tue, 8 Dec 2015 18:58:28 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F249E1C95 for ; Tue, 8 Dec 2015 18:58:27 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id tB8IwJx3038121 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 8 Dec 2015 20:58:20 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua tB8IwJx3038121 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id tB8IwImh038095; Tue, 8 Dec 2015 20:58:18 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 8 Dec 2015 20:58:18 +0200 From: Konstantin Belousov To: Dag-Erling Sm??rgrav Cc: Warner Losh , "freebsd-hackers@freebsd.org" Subject: Re: Fwd: DELETE support in the VOP_STRATEGY(9)? Message-ID: <20151208185818.GD82577@kib.kiev.ua> References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86wpsord9l.fsf@desk.des.no> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 18:58:28 -0000 On Tue, Dec 08, 2015 at 07:44:38PM +0100, Dag-Erling Sm??rgrav wrote: > Warner Losh writes: > > Dag-Erling Sm??rgrav writes: > > > But the filesystem does not know whether the underlying storage is > > > electromechanical or solid-state, nor does it know whether the user > > > cares much about seek times (unless we introduce the heuristic > > > "avoid creating holes unless the file already has them, in which > > > case the userland probably does not care"). > > Actually, the filesystem does know. Or has some knowledge of what > > is supported and what isn't. BIO_DELETE support is a strong indicator > > of a flash or other log-type system. > > The filesystem can ask the layer below if BIO_DELETE is supported, but > should not assume anything about what it means. For instance, I could > write a gnop-like module that translates BIO_DELETE into an all-zeroes > BIO_WRITE and passes everything else unmodified. It would provide a > stronger guarantee than, say, SATA TRIM but would also have a completely > different performance profile (even on SSDs, since it would do its work > synchronously whereas TRIM works asynchronously). I again agree. This is how UFS issues TRIM. When the data block is freed and there are no dandling pointers in the inode copy on disk pointing to the block, BIO_DELETE is issued if volume reports it. Everything else is up to the geom stack and driver. > > Anyway, my point is that Maxim needs to revise his assumptions. Which does not invalidate the fact that a 'punch hole' interface is useful. From owner-freebsd-hackers@freebsd.org Tue Dec 8 19:02:40 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CDECC9D3C99 for ; Tue, 8 Dec 2015 19:02:40 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 9528C132E for ; Tue, 8 Dec 2015 19:02:40 +0000 (UTC) (envelope-from des@des.no) Received: from desk.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id BB59DD725; Tue, 8 Dec 2015 19:02:39 +0000 (UTC) Received: by desk.des.no (Postfix, from userid 1001) id 7ACC4482EC; Tue, 8 Dec 2015 20:02:30 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Konstantin Belousov Cc: Warner Losh , "freebsd-hackers\@freebsd.org" Subject: Re: Fwd: DELETE support in the VOP_STRATEGY(9)? References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <20151208175414.GB82577@kib.kiev.ua> Date: Tue, 08 Dec 2015 20:02:30 +0100 In-Reply-To: <20151208175414.GB82577@kib.kiev.ua> (Konstantin Belousov's message of "Tue, 8 Dec 2015 19:54:14 +0200") Message-ID: <86lh94rcft.fsf@desk.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:02:40 -0000 Konstantin Belousov writes: > Exactly. I completely agree with the statement above, and think that > this is the only thing that should be implemented. There could be a > request, most likely an filesystem-level ioctl, which punches holes in > the file. Its effect on the file state should be the same as if the > seek was done between writes, if the filesystem supports the ioctl. > More, if supported, the lseek(SEEK_HOLE/SEEK_DATA) behaviour should > be consistent with the request. The later might mean that we should > restrict the interface to only accept ranges at block boundaries. Having spent a couple of weeks recently trying to get code that uses SEEK_HOLE to work reliably, my advice is to burn it down and salt the earth. Wait, am I thinking out loud again? What I meant to say is that there is no such thing as consistency where SEEK_HOLE is concerned, and it might as well just be an alias for (SEEK_END, 0). DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@freebsd.org Tue Dec 8 19:03:17 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4AC9C9D3D5B for ; Tue, 8 Dec 2015 19:03:17 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-oi0-x235.google.com (mail-oi0-x235.google.com [IPv6:2607:f8b0:4003:c06::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0824D1493 for ; Tue, 8 Dec 2015 19:03:17 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: by oies6 with SMTP id s6so14648615oie.1 for ; Tue, 08 Dec 2015 11:03:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :message-id:references:to; bh=GMjzDPf8br8cyiNfA0oVdEriOpGdd0voEyO0Gcv6Ruk=; b=VObVRsK5Ho4Ea4xhjvuMWbE9nIDxJbppghgvNV1/3FcLej959nKhkAGIMzUcDX4c70 p8RhXoeR2qiu6qzxp3jrirm9eVqOZhrXCXaMUceiLMOK6qZNi/9Pebtv9JDYiyMfU8MP H/WCZyPukEA1F7OueRnGza1nlUlrFCoTX1lgh4Y6duYoqdZ0hkEssQsfwO3T/RaJafX6 Qd549H7nZTuAuNjRRfupKY7HtnBBSwaXEWygCc5+aahPrsqWr+1vr9P0KxGi3q+OpDwp ltbjWn3IOaLJyL+0OaN7r3FAvcc4Fdtq4o0DmmUSvUuvMB/PR5WbXupFAW5pNeT+/xJi FSaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:message-id:references:to; bh=GMjzDPf8br8cyiNfA0oVdEriOpGdd0voEyO0Gcv6Ruk=; b=OEkwErOCb8PrM0wHB1HMcDKIEea0ME9TrzvQe02Y7c4jtqGYezx7fZqlsK3YgG20Hb Ycl0cV2c7YC9XRWvmSU9BfvYDhpfKb5Y4Jc7uCn1LHjlXDrOO/l80gdopK9z5lQHow+7 O+QkNVtjeVGdbC1zlZaQ6MUJZPNWjGZyDAxbTC4ndaKMfvu+Nmi9FdJQuH/Msi9dWglP jo+kzkb141C/HbEq6aCq9FgQnCcOg0rz1TeuEXdSOwNQCwaMQ9dMaGS5aFZ95ws8iGkv CUQPCegalo19ac889g3eSIQz1/gKAIoY+i/RY5Fn+aES+bR9ZxQjKGk9V5XEHtNfX89S krKg== X-Gm-Message-State: ALoCoQk5B3ck1BhC6g7fF/LfmJz5ARWUWTDoeJqFwJkZSfbCtdZkVIOmQBmL6mcQJLwuaYlOpTz3OsRULSV9hYRSxHxBEKSoQw== X-Received: by 10.202.181.3 with SMTP id e3mr862163oif.67.1449601396181; Tue, 08 Dec 2015 11:03:16 -0800 (PST) Received: from ?IPv6:2601:280:4900:3700:4d3f:8eba:ea86:7700? ([2601:280:4900:3700:4d3f:8eba:ea86:7700]) by smtp.gmail.com with ESMTPSA id m206sm1914536oig.13.2015.12.08.11.03.15 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 08 Dec 2015 11:03:15 -0800 (PST) Sender: Warner Losh Subject: Re: DELETE support in the VOP_STRATEGY(9)? Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Content-Type: multipart/signed; boundary="Apple-Mail=_EC74369D-41B2-4A4A-B350-17C3E63CCCCA"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5.2 From: Warner Losh In-Reply-To: <566726ED.2010709@multiplay.co.uk> Date: Tue, 8 Dec 2015 12:03:11 -0700 Cc: freebsd-hackers@freebsd.org Message-Id: <0DB97CBA-4DC3-4D52-AE9D-54546292D66F@bsdimp.com> References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> <566726ED.2010709@multiplay.co.uk> To: Steven Hartland X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:03:17 -0000 --Apple-Mail=_EC74369D-41B2-4A4A-B350-17C3E63CCCCA Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Dec 8, 2015, at 11:52 AM, Steven Hartland = wrote: >=20 >=20 >=20 > On 08/12/2015 18:44, Dag-Erling Sm=C3=B8rgrav wrote: >> Warner Losh writes: >>> Dag-Erling Sm=C3=B8rgrav writes: >>>> But the filesystem does not know whether the underlying storage is >>>> electromechanical or solid-state, nor does it know whether the user >>>> cares much about seek times (unless we introduce the heuristic >>>> "avoid creating holes unless the file already has them, in which >>>> case the userland probably does not care"). >>> Actually, the filesystem does know. Or has some knowledge of what >>> is supported and what isn't. BIO_DELETE support is a strong = indicator >>> of a flash or other log-type system. >> The filesystem can ask the layer below if BIO_DELETE is supported, = but >> should not assume anything about what it means. For instance, I = could >> write a gnop-like module that translates BIO_DELETE into an = all-zeroes >> BIO_WRITE and passes everything else unmodified. It would provide a >> stronger guarantee than, say, SATA TRIM but would also have a = completely >> different performance profile (even on SSDs, since it would do its = work >> synchronously whereas TRIM works asynchronously). That ship has sailed. UFS, at least, assumes that if TRIM is supported = then relocating files to be contiguous is bad. But writing a gnop module that did the BIO_DELETE thing would be bogus. BIO_DELETE does not mean that blocks will read back as zeros. But = that=E2=80=99s not what BIO_DELETE means. So, sure you could invent a stupid thing that breaks the rules, and thus the assumptions of the other code, but why = would you want to do that? The SATA trims are actually synchronous (in the absence of power = failures). Once you TRIM The data, it is gone. And depending what bits are set in the identify response, you can count on different things. But to say = they happen asynchronously because of implementation details about when the = data is actually erased is missing the point. Also, your BIO_DELETE example wouldn=E2=80=99t guarantee the data is erased either. Writes to log = append devices (like SSDs) are like a TRIM followed by a write: the old LBA mapping is discarded and a new one replaces it. >> Anyway, my point is that Maxim needs to revise his assumptions. > Just to clarify most consumer devices process TRIM synchronously, not = asynchronously. It also depends on what you mean by =E2=80=98process=E2=80=99 here. > Your example isn't actually just an example CAM scsi_da has a number = of different ways it can process BIO_DELETE: > * ATA TRIM > * SCSI UMAP > * Write Same 16 > * Write Same 10 > * Zero >=20 > So you example is actually exists in practice in the FreeBSD code base = ;-) All these are effectively TRIM operations. The devices that implement = them use them as hints to optimize storage. DES=E2=80=99 BIO_DELETE -> WRITE = zero example doesn=E2=80=99t optimize storage at all, nor does it give the = lower layers any clue about how to optimize the storage. All the SCSI delete types do give that hint. Warner --Apple-Mail=_EC74369D-41B2-4A4A-B350-17C3E63CCCCA Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJWZylxAAoJEGwc0Sh9sBEAQVsQAMmXAMbT4ujIa1fwJHBI0NKK Mk/v2egPRzkqzp6WrXYFaK1h5XOpjFtnJt2R3SD9PbVdz80M3CHVhb0Qv8XaXWVr z6KYLaQpcrMCPe1Gdeym0gVNvS1loUTsotRisVAiW1EhbJnsx+0Wl7ad8O8bwx2+ 6zGX6hz+Kpcy2vHzzubRoJaNoRJnm8lY/lP+qsRdZWNsROdCwjCj/qwUDeGr25xA Z0/MS+NZrPoyEuPdr5vjCrChyK/mzl5+mv3GJWO3OP+JsAcTOzhSxEr0nMtEcXyN zHKeo/k79UcIPnXOd6bg2UM0/P+9+m/VkE3rCLF/MVY5wt2O6PiMDEblpbVGQCid 2q3MTBtQ3DDJ0IW6TveLlrfvk6XnlyO5NX+Th0sgB2HASCc6OmyyC2gu8T+gB37Q 2UsOU1pJvx4XBfE7cvPMpj0T40ax88zPWNamaTSyW5AhxZI8rGPst8BF3hCzHZqX yULRAlLn5lE1yeIUGvV86VLP+xFFnoRf7o8TB6uL6wIQ/O+s34EdPzQzmKpFVJMO MKr1dffdZLKtWsFnUY8yUXExhLxUR1W+zwRbNwetqwmgy6VJ4btu/ZES91t+wmVu gnDMGMaYv2HHhQHGZMicN3H60ywGIVLpA4PxLPK7s9vojMY6u9NNx/Bp3LRqkQ97 Vli8lviRtFAtZbNEr+4r =flRM -----END PGP SIGNATURE----- --Apple-Mail=_EC74369D-41B2-4A4A-B350-17C3E63CCCCA-- From owner-freebsd-hackers@freebsd.org Tue Dec 8 19:05:54 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8A3509D404A for ; Tue, 8 Dec 2015 19:05:54 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ob0-x234.google.com (mail-ob0-x234.google.com [IPv6:2607:f8b0:4003:c01::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A6411748 for ; Tue, 8 Dec 2015 19:05:54 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: by obcse5 with SMTP id se5so19391481obc.3 for ; Tue, 08 Dec 2015 11:05:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :message-id:references:to; bh=o0lHkCGi8L4y3ENCkALou+GK8dB9k2vSDCm6p11zUE0=; b=IyyIT5v4Py2GsHTsAFyyTPwLNKElG/QPOFXvbF/7Zy0Ux/XMB34zCJiQk7ybS58DjP rXxr6yq7zA98dFnDS0wU0hMAl8sOjE8DGkr+HP9w3VpK1D/w1MtcgVx87hrv9KlPUYmG rODoKJJGLPxzr4d61Nu5w7tttkee6PvUVoaZXQfWI/TEOOvxSJtqq7yUkHplsBow/78U wYCXtalD/w0ilecphhnF/AjhHDB4NMtA2Soi3LSphQNoirCc/sGDqokdqND0R1ydzaAX qSIiQurJeEKr+K9BZEAW7NRNN1ouGWVpk47dMP0WroQaaaUTeDdLPW/JbM4XinDxVIg3 19/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:message-id:references:to; bh=o0lHkCGi8L4y3ENCkALou+GK8dB9k2vSDCm6p11zUE0=; b=cZ+8hGmfeFHhloc3Zi0fWkEhkix5CxRFCacRKW58C+zLi01WbKMYuLV03BoHZZsU/R SPZWdF3jL15wJCGBlWnQ3CCNipFUn4yOWCtCuLRwd0/jq4Wb7mMcYn1DZtkzIIjzUnGI Ts7za0fbEfl+2Xb2OFOs7nUbd7v8hLcZRl5u6eCySAoS04pp8v/3uhEF2lRL4xTQWfuj vefsTomWzRHYVGJD7HvR7i/5uvBqxV4CgZf8ZKoOkb4M0Ije4Z4rq5Ea1ZbG52PmOOAz 11U7f6WiHGf3uV+5dkWdGpxUPA9n8z3uWygGaEzLNKaa8RyBSKOpCjqcNNKhCPAo4SGe 3IVg== X-Gm-Message-State: ALoCoQmWDJZ3FHmQiNPlLqnHdK0dLsZiI3qc8+AjXp7J/mUNWEI3hDHMNZ5tbxTnzeUgS58KVIry4XwmLlxXReAob9waDobJgQ== X-Received: by 10.182.58.35 with SMTP id n3mr1063767obq.46.1449601553432; Tue, 08 Dec 2015 11:05:53 -0800 (PST) Received: from ?IPv6:2601:280:4900:3700:4d3f:8eba:ea86:7700? ([2601:280:4900:3700:4d3f:8eba:ea86:7700]) by smtp.gmail.com with ESMTPSA id ct9sm1876772oec.15.2015.12.08.11.05.52 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 08 Dec 2015 11:05:52 -0800 (PST) Sender: Warner Losh Subject: Re: DELETE support in the VOP_STRATEGY(9)? Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Content-Type: multipart/signed; boundary="Apple-Mail=_76B38211-5D68-4DB3-812A-059CA44B4757"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5.2 From: Warner Losh In-Reply-To: <86twnur25s.fsf@desk.des.no> Date: Tue, 8 Dec 2015 12:05:51 -0700 Cc: Maxim Sobolev , FreeBSD Hackers , Pawel Jakub Dawidek , Eitan Adler Message-Id: References: <86twnur25s.fsf@desk.des.no> To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:05:54 -0000 --Apple-Mail=_76B38211-5D68-4DB3-812A-059CA44B4757 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Dec 7, 2015, at 3:19 AM, Dag-Erling Sm=C3=B8rgrav = wrote: >=20 > Maxim Sobolev writes: >> Hi folks, do we support delete operation in the vnode layer? This >> comes from observation that md(4) converts from DELETE into >> sector-size zero buffer write before it feeds it to the vnode. >=20 > As the person who implemented this (because I needed a reliable way to > test fsck_ffs -E): it is currently the only possible action, other = than > to ignore the request - which is actually fine since BIO_DELETE is not > guaranteed to do anything whatsoever, except not corrupt data you = didn't > want deleted. >=20 > I'm not opposed to the idea of VOP_DELETE or similar, but don't assume > that "punching through" is always the correct semantic, and don't = assume > that it will be easy to implement - and it will have to be implemented > correctly at every layer, in every geom, in every storage driver etc. > There are many details to work out and many opportunities for = mistakes. > Remember that BIO_DELETE is strictly advisory. Should VOP_DELETE also > be advisory, or do we want to guarantee that a VOP_DELETEd region = reads > back as all zeroes? Remember that we can't guarantee that the data = will > be removed from the underlying medium, or verify that it was. How do = we > handle an unaligned VOP_DELETE? Should a VOP_DELETE create a hole if > the filesystem support holes? Since you mentioned VMs, this could > result in fragmentation of initially unfragmented (preallocated) disk > images. http://man7.org/linux/man-pages/man2/fallocate.2.html would answer those questions. Warner --Apple-Mail=_76B38211-5D68-4DB3-812A-059CA44B4757 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJWZyoPAAoJEGwc0Sh9sBEAxUUQANrYKp/07D1peZqYseAGqdhj AhwT+5V0vaNyWiwH6CUVrlrqXfp5kM2Uu8AqiEW42zuUhdhl7nQDfAP1j4ipr9lA nOo+Oln4h8kChXyoz73o02te8FlWmnwPn2bt9k7U6cE1H0oMra+PDBsL5taOQmJO obj77YtjlNZ+Y7Z/kHFjePx58aathLms+E2AuvL4HKJUjLZNFhly7lK85HYEQQyL 0aUQ+ZtYKdCfGtH05BkDtvDRqM6Pklic9qp34e2hqCNZRAov5u40YUzOAFdeaRLz pUta/weAl1FvmuXxTEbGZdguBBYtYsCv5vCM5out6w8TmcyMkPGJMgHqrdQBNeDk 6kL+8iJZPVkD5itvMdb2rxJdl+TZRr+XBAd16LT83j7ZyPMzikFbUNMvPV/rTL6s ViZSzkbVmRbqmH4xRa5qIgcPx+ek1L0ZSnlA5PZAn74JTduR3KZQ+WbALYOW4Fre Aa1LGiD0A9sDDOoBzIe68vV0B1y120yxP3iWoNpo/pgsq/c0Jttl3laimb69FtP6 bWJ9tr9f7Y1CZlxYocgE5rE7DhzEje/KfhRJa49njPeeskvFnHlMo/Evlmlziihf 38eXw7NSmgQHCBPvxxTlx0Da4EQpyIg12Qylx9RCVTdsIQJ1V2eTQ0RCmxXEZToW mO7TMPAfcrSG6WZFfpB4 =h127 -----END PGP SIGNATURE----- --Apple-Mail=_76B38211-5D68-4DB3-812A-059CA44B4757-- From owner-freebsd-hackers@freebsd.org Tue Dec 8 19:06:28 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EB4429D40D0 for ; Tue, 8 Dec 2015 19:06:28 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 831F21866 for ; Tue, 8 Dec 2015 19:06:28 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id tB8J6Mmo090469 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 8 Dec 2015 21:06:22 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua tB8J6Mmo090469 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id tB8J6MdI090468; Tue, 8 Dec 2015 21:06:22 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 8 Dec 2015 21:06:22 +0200 From: Konstantin Belousov To: Dag-Erling Sm??rgrav Cc: "freebsd-hackers@freebsd.org" Subject: Re: Fwd: DELETE support in the VOP_STRATEGY(9)? Message-ID: <20151208190622.GE82577@kib.kiev.ua> References: <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> <20151208185818.GD82577@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151208185818.GD82577@kib.kiev.ua> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:06:29 -0000 On Tue, Dec 08, 2015 at 08:58:18PM +0200, Konstantin Belousov wrote: > On Tue, Dec 08, 2015 at 07:44:38PM +0100, Dag-Erling Sm??rgrav wrote: > > Warner Losh writes: > > > Dag-Erling Sm??rgrav writes: > > > > But the filesystem does not know whether the underlying storage is > > > > electromechanical or solid-state, nor does it know whether the user > > > > cares much about seek times (unless we introduce the heuristic > > > > "avoid creating holes unless the file already has them, in which > > > > case the userland probably does not care"). > > > Actually, the filesystem does know. Or has some knowledge of what > > > is supported and what isn't. BIO_DELETE support is a strong indicator > > > of a flash or other log-type system. > > > > The filesystem can ask the layer below if BIO_DELETE is supported, but > > should not assume anything about what it means. For instance, I could > > write a gnop-like module that translates BIO_DELETE into an all-zeroes > > BIO_WRITE and passes everything else unmodified. It would provide a > > stronger guarantee than, say, SATA TRIM but would also have a completely > > different performance profile (even on SSDs, since it would do its work > > synchronously whereas TRIM works asynchronously). > I again agree. This is how UFS issues TRIM. When the data block is freed > and there are no dandling pointers in the inode copy on disk pointing to > the block, BIO_DELETE is issued if volume reports it. Everything else > is up to the geom stack and driver. I am sorry for the followup mail, but I probably have to explain more. The freed block, for which BIO_DELETE is issued, is not marked as free in the bitmap, until the BIO_DELETE completion is reported. In other words, we do not reuse the freed block while TRIM command is possibly executed. From owner-freebsd-hackers@freebsd.org Tue Dec 8 19:13:35 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2BBC89D45B0; Tue, 8 Dec 2015 19:13:35 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9FA9A1C9C; Tue, 8 Dec 2015 19:13:34 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id tB8JDORI045632 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 8 Dec 2015 21:13:24 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua tB8JDORI045632 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id tB8JDOYr045631; Tue, 8 Dec 2015 21:13:24 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 8 Dec 2015 21:13:24 +0200 From: Konstantin Belousov To: Dag-Erling Sm??rgrav Cc: Kirk McKusick , Maxim Sobolev , freebsd-current@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: posix_fallocate(2) && posix_fadvise(2) are somewhat broken Message-ID: <20151208191324.GF82577@kib.kiev.ua> References: <868u55rl96.fsf@desk.des.no> <20151208174259.GA82577@kib.kiev.ua> <86poygrctt.fsf@desk.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86poygrctt.fsf@desk.des.no> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:13:35 -0000 On Tue, Dec 08, 2015 at 07:54:06PM +0100, Dag-Erling Sm??rgrav wrote: > Konstantin Belousov writes: > > Dag-Erling Sm??rgrav writes: > > > Maxim Sobolev writes: > > > > Hi, while working on some unrelated feature I've noticed that at least > > > > those two system calls are not returning proper value (-1) on error. > > > > Instead actual errno value is returned from the syscall verbatim, > > > > i.e. posix_fadvise() returns 22 on EINVAL. > > > That's how syscalls work. > > No, this is not how typical syscalls work, but is how the posix_fallocate() > > and posix_fadvise() are specified by Posix. The patch is wrong, see also > > r261080 and r288640. > > Umm, I can't find the code ATM but syscalls store the actual return > value in td_retval and return 0 or EWHATEVER and the syscall wrapper > handles the translation. If that's not what Maxim was talking about, > then please ignore me. I mean that typical syscall does not return error to usermode, it returns -1 and sets errno. But usermode conventions for the posix_f*e() are different, and I believe this is what tripped over Maxim and I reacted upon. Indeed kernel expects the syscall function from the sysentvec table to return error or zero. If zero is returned, then td_retval array is translated into return value for usermode by cpu_set_syscall_retval(). If non-zero is returned, typical kernel/libc interface returns the syscall function return value to usermode and additionally set flag (like PSL_C in the processor status word). Of course, there is an additional translation layer in usermode syscall trampolines. > > Anyway, happy to hear that the X/Open group have found a new way to > screw people over. It is the same as the pthread_* conventions. They are somewhat consistent. From owner-freebsd-hackers@freebsd.org Tue Dec 8 19:16:34 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 083679D4879 for ; Tue, 8 Dec 2015 19:16:34 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x232.google.com (mail-wm0-x232.google.com [IPv6:2a00:1450:400c:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F2A21F26 for ; Tue, 8 Dec 2015 19:16:33 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by wmvv187 with SMTP id v187so228079954wmv.1 for ; Tue, 08 Dec 2015 11:16:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-type:content-transfer-encoding; bh=/FSA3lFZDjIwyoTKvcRRRNB+QExwFDEVbtYVccICKeM=; b=F5iZlxlGBdQ4kUKFquFxDkR2uHpPJuhhI4NhMh8etsLudUnS7mNk9iphgKcfJawNFz uInRT5Vda0DWTslP2KHG/1FBfuwYWO/BpddlHE80LF/xB/5Gv0NQLbKWKdke2ZN9JVg4 KNVpb9P0jPTxqf+12+CWZ1X2GCY4N9I92z76FGeMEF7fc1N5TvqnvptuT5L5LsZMdXpL XegUdk4ZklATd22o+f2B7YI+eO5m1VsCAWeJ46v7j1bPrE0T60+KIRdHWhsbEwV8j0+c 0DE7g3Cm2nofdCJBnoO96q0s6UkjNQpSucgXkoN8d4C7TBKTLaVDCWbwkvYQgR6kn2CC E5NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=/FSA3lFZDjIwyoTKvcRRRNB+QExwFDEVbtYVccICKeM=; b=AoIqJsWtm+MVoN/jJ/ctOTPYwkO3+tF2uIEcAs9QJYn3dcVCErLqwbP0zJL7+W49h2 jAfKmyyDx1i/ao9cWe52GtnWXL/DlvJpb4RtsX5A/TtNWYgYFFSJFn9hfSUnupHjrWLC vdnOWrZl9v/vgH0USgaj217bXFh248Ac3ufMr8V8ISpHocYVSbO0/gkIJ0ob45QqZK5P sdBdXRCyceEL24JyHHkEMqpCGBErOgy0EO5ukPf7iuwGBRX7OtMldVbsy1qOeYPB3tmv 6P/fRntA4/Lfs3mleJer75g2LhA02WsGTTTotKheugznXGzCwdS7aCwNe2L9y3jgE7Z5 7XFg== X-Gm-Message-State: ALoCoQlUalCdgHtYqCkH/BI6Xl4as2Imiip9Meb+w+kexsUd/SaBsZ3WeoXdoDCbKkixkQ5xeJlz1kAPcB07LgcL0HF3WVGIdQ== X-Received: by 10.194.89.226 with SMTP id br2mr1491716wjb.22.1449602191874; Tue, 08 Dec 2015 11:16:31 -0800 (PST) Received: from [10.10.1.58] (liv3d.labs.multiplay.co.uk. [82.69.141.171]) by smtp.gmail.com with ESMTPSA id da10sm4243246wjb.22.2015.12.08.11.16.30 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 08 Dec 2015 11:16:30 -0800 (PST) Subject: Re: DELETE support in the VOP_STRATEGY(9)? To: Warner Losh References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> <566726ED.2010709@multiplay.co.uk> <0DB97CBA-4DC3-4D52-AE9D-54546292D66F@bsdimp.com> Cc: freebsd-hackers@freebsd.org From: Steven Hartland Message-ID: <56672C94.30404@multiplay.co.uk> Date: Tue, 8 Dec 2015 19:16:36 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <0DB97CBA-4DC3-4D52-AE9D-54546292D66F@bsdimp.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:16:34 -0000 On 08/12/2015 19:03, Warner Losh wrote: >> On Dec 8, 2015, at 11:52 AM, Steven Hartland wrote: >> >> >> >> On 08/12/2015 18:44, Dag-Erling Smørgrav wrote: >>> Warner Losh writes: >>>> Dag-Erling Smørgrav writes: >>>>> But the filesystem does not know whether the underlying storage is >>>>> electromechanical or solid-state, nor does it know whether the user >>>>> cares much about seek times (unless we introduce the heuristic >>>>> "avoid creating holes unless the file already has them, in which >>>>> case the userland probably does not care"). >>>> Actually, the filesystem does know. Or has some knowledge of what >>>> is supported and what isn't. BIO_DELETE support is a strong indicator >>>> of a flash or other log-type system. >>> The filesystem can ask the layer below if BIO_DELETE is supported, but >>> should not assume anything about what it means. For instance, I could >>> write a gnop-like module that translates BIO_DELETE into an all-zeroes >>> BIO_WRITE and passes everything else unmodified. It would provide a >>> stronger guarantee than, say, SATA TRIM but would also have a completely >>> different performance profile (even on SSDs, since it would do its work >>> synchronously whereas TRIM works asynchronously). > That ship has sailed. UFS, at least, assumes that if TRIM is supported then > relocating files to be contiguous is bad. > > But writing a gnop module that did the BIO_DELETE thing would be bogus. > BIO_DELETE does not mean that blocks will read back as zeros. But that’s > not what BIO_DELETE means. So, sure you could invent a stupid thing that > breaks the rules, and thus the assumptions of the other code, but why would > you want to do that? > > The SATA trims are actually synchronous (in the absence of power failures). > Once you TRIM The data, it is gone. And depending what bits are set in > the identify response, you can count on different things. But to say they > happen asynchronously because of implementation details about when the data > is actually erased is missing the point. Also, your BIO_DELETE example > wouldn’t guarantee the data is erased either. Writes to log append devices > (like SSDs) are like a TRIM followed by a write: the old LBA mapping is > discarded and a new one replaces it. Not all SATA TRIMs are synchronous , some FW does process them in the background. Saying once you TRIM data its gone is actually too strong I'm afraid, as its advisory, the FW can ignore you if it so chooses. There is the concept of DSM deterministic read which if set "should" result in returning the same values from read of a TRIMed sector every time, but even this is unreliable due to FW bugs (yes I've seen this). > >>> Anyway, my point is that Maxim needs to revise his assumptions. >> Just to clarify most consumer devices process TRIM synchronously, not asynchronously. > It also depends on what you mean by ‘process’ here. Indeed it does, here I mean when / if the data is removed from the media by the HW. > >> Your example isn't actually just an example CAM scsi_da has a number of different ways it can process BIO_DELETE: >> * ATA TRIM >> * SCSI UMAP >> * Write Same 16 >> * Write Same 10 >> * Zero >> >> So you example is actually exists in practice in the FreeBSD code base ;-) > All these are effectively TRIM operations. The devices that implement them > use them as hints to optimize storage. DES’ BIO_DELETE -> WRITE zero > example doesn’t optimize storage at all, nor does it give the lower layers > any clue about how to optimize the storage. All the SCSI delete types > do give that hint. This is true, just wanted to highlight that "TRIM" can mean very different things even at the CAM layer. From owner-freebsd-hackers@freebsd.org Tue Dec 8 19:17:18 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C47BC9D497A for ; Tue, 8 Dec 2015 19:17:18 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x236.google.com (mail-wm0-x236.google.com [IPv6:2a00:1450:400c:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 58C8A1068 for ; Tue, 8 Dec 2015 19:17:18 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by wmww144 with SMTP id w144so193596300wmw.1 for ; Tue, 08 Dec 2015 11:17:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-type:content-transfer-encoding; bh=XVvNtyW3bMRyYKACo6Fbj2SnwjD8iSeoKzrbj1z7FOE=; b=h6LkizQz8ShZ1CpOiWRFeFXnDDYZVDV2764I5M1bz26Wc12ZgEPU2M2GfCPY62Bigi pXfryHfR+O9NgAQsyal5kwKobsYJPFuQuDXFBCUIcrDfMYpHb8TkoYjSYF7T2Q78P5W0 NAC0wDPOgwrAv/RmLTh3kZj0WpY3ZxXjFtvrT6TkfAqAJMEqIxvQqDjGk6xpIrPPOWYK Faw9+NRCJSd18YIxuGPsIWzhGYI1lq+aODp9+/OlQdxYk87fVe+/iEBINpyDMFI692Pu 6bzZLZQ0AMzhkhdODAz7oCP3BT9PLUGQw9USjUifr9Tmr0yfvaW8fr6ZUIWIhBApEf+i xzIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=XVvNtyW3bMRyYKACo6Fbj2SnwjD8iSeoKzrbj1z7FOE=; b=HeICt3X6Mi0o5XPFlbnqEwR3HIqw2mETLnZfR1BbNdNi1iQhf/a1y4JD8pBK2CDCtj HzZyCCEdTjowiZ9NvNdQc22Io5J2ZZRBGLdT0BBhXK66BgaKRnyAcwj9FuL4Zshd9SsJ lgE2tJdJKyu9QNzpF78aUpqc3P5BpWr6vjoE7TpdSPI+YiFHrIQwrH5XMfLfVjV0oJMl HKVhhXYPZCLc/1BCq+3Z7nsUwCN/1083lY153YGSWSjxCXGLVMaxYK5BblNC07ojxzck jFSJeNbwrGqTUABW+jx5Z6bg0Gl7MPYAQRX3ZWn6kt4d+m/1q3ZrMjrKZbruHKB5dMIi ijYg== X-Gm-Message-State: ALoCoQmyhRTzMCDUjtRbEjRP2JdgxRlEYgorQxwstb6SE95uQhn9GjI/ncVXLT/wBRrCjwkepn1eJmFFNNd8hKJdC2/tQvUAEg== X-Received: by 10.28.172.129 with SMTP id v123mr6335101wme.47.1449602236621; Tue, 08 Dec 2015 11:17:16 -0800 (PST) Received: from [10.10.1.58] (liv3d.labs.multiplay.co.uk. [82.69.141.171]) by smtp.gmail.com with ESMTPSA id ft4sm4212150wjb.37.2015.12.08.11.17.15 for (version=TLSv1/SSLv3 cipher=OTHER); Tue, 08 Dec 2015 11:17:15 -0800 (PST) Subject: Re: Fwd: DELETE support in the VOP_STRATEGY(9)? To: freebsd-hackers@freebsd.org References: <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> <20151208185818.GD82577@kib.kiev.ua> <20151208190622.GE82577@kib.kiev.ua> From: Steven Hartland Message-ID: <56672CC1.7040809@multiplay.co.uk> Date: Tue, 8 Dec 2015 19:17:21 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <20151208190622.GE82577@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:17:18 -0000 On 08/12/2015 19:06, Konstantin Belousov wrote: > On Tue, Dec 08, 2015 at 08:58:18PM +0200, Konstantin Belousov wrote: >> On Tue, Dec 08, 2015 at 07:44:38PM +0100, Dag-Erling Sm??rgrav wrote: >>> Warner Losh writes: >>>> Dag-Erling Sm??rgrav writes: >>>>> But the filesystem does not know whether the underlying storage is >>>>> electromechanical or solid-state, nor does it know whether the user >>>>> cares much about seek times (unless we introduce the heuristic >>>>> "avoid creating holes unless the file already has them, in which >>>>> case the userland probably does not care"). >>>> Actually, the filesystem does know. Or has some knowledge of what >>>> is supported and what isn't. BIO_DELETE support is a strong indicator >>>> of a flash or other log-type system. >>> The filesystem can ask the layer below if BIO_DELETE is supported, but >>> should not assume anything about what it means. For instance, I could >>> write a gnop-like module that translates BIO_DELETE into an all-zeroes >>> BIO_WRITE and passes everything else unmodified. It would provide a >>> stronger guarantee than, say, SATA TRIM but would also have a completely >>> different performance profile (even on SSDs, since it would do its work >>> synchronously whereas TRIM works asynchronously). >> I again agree. This is how UFS issues TRIM. When the data block is freed >> and there are no dandling pointers in the inode copy on disk pointing to >> the block, BIO_DELETE is issued if volume reports it. Everything else >> is up to the geom stack and driver. > I am sorry for the followup mail, but I probably have to explain more. > The freed block, for which BIO_DELETE is issued, is not marked as free > in the bitmap, until the BIO_DELETE completion is reported. In other > words, we do not reuse the freed block while TRIM command is possibly > executed. This is the same for ZFS. From owner-freebsd-hackers@freebsd.org Tue Dec 8 19:19:15 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CFCD59D4B53 for ; Tue, 8 Dec 2015 19:19:15 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ob0-x233.google.com (mail-ob0-x233.google.com [IPv6:2607:f8b0:4003:c01::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D4B11298 for ; Tue, 8 Dec 2015 19:19:15 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: by obbww6 with SMTP id ww6so19754221obb.0 for ; Tue, 08 Dec 2015 11:19:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :message-id:references:to; bh=u6mUIa1S6P2XdrzxAlheaazRidNDBqR2RqSPJzZ2d8M=; b=0x5O/OBLyxLXtGFGur0mUlK11Jg/fNIhupKqXrNRPrmpr88ZSeubOAHY9x4R0a9K0n +G2owNopR7UVviENb/XfQuT191rO/chJelabHhDwP6IT/7JM9Y2Z1jcmI+gbqkPAWpQx 3JrCL48l2wNaDlQYRjod8GGuSihjZ1nay2LHn3oSjAWmfqStAvz8+ytAHjygTlf1NtFo IcPSH2PwvSm1wZNTtit/cZGu8dkOAI6s3C303Js3HBAvdkX/VFc88daL7hms40J6I4ws ZOT5gCyOHFfVMFYWcwPlc6rBhvY7vYvokF2Hh6d4sLOBuIqmf9L9ew6ozWQ12uoTWj+1 yqlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:message-id:references:to; bh=u6mUIa1S6P2XdrzxAlheaazRidNDBqR2RqSPJzZ2d8M=; b=cOzbhiwm6KbcJ7FyOGmJ6WQXLPmp0JVRc7iucHn/iLvyhYdfsQ67cggARLoeMlMv5W Ts1Bh9LakATQamBfOIXCdKWXVkANrBQJipepB82mAtTvKqBve4c9zTZmtrifHWeNmVMx lRULQ0AvCmva+8c2+OLGkcaVupZ65PGI/9je6tHdnc7HESacyV6bkWIxBNN9KAsq8/Tz MeQBOC5jtZSQ+77bhpNrXC5V54AK7Goozv85RfUl5Ku19bBBROgWGgjEWu+ahwGHmhoN i9nIuJ4Pe5ef+EBRnzgQ0XkiM7wf28BlXQXBfmtm0REHnE1GzL+Fk+hSxMNwesNqAoYj OFsQ== X-Gm-Message-State: ALoCoQlhaeUK8zy+zfcScqMiNplv7nPPzY7zoE2ZvyQ2I+4o8AYfDBOi2fREFHmR92tBXEDRSKC2W8E+NqU5qRFQYd78eXsDcA== X-Received: by 10.60.67.104 with SMTP id m8mr1039364oet.37.1449602354648; Tue, 08 Dec 2015 11:19:14 -0800 (PST) Received: from ?IPv6:2601:280:4900:3700:4d3f:8eba:ea86:7700? ([2601:280:4900:3700:4d3f:8eba:ea86:7700]) by smtp.gmail.com with ESMTPSA id x131sm1896904oix.27.2015.12.08.11.19.13 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 08 Dec 2015 11:19:13 -0800 (PST) Sender: Warner Losh Subject: Re: DELETE support in the VOP_STRATEGY(9)? Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Content-Type: multipart/signed; boundary="Apple-Mail=_BC57BC0A-AA3E-464E-BE53-94B0B1DD300A"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5.2 From: Warner Losh In-Reply-To: <20151208190622.GE82577@kib.kiev.ua> Date: Tue, 8 Dec 2015 12:19:11 -0700 Cc: Dag-Erling Sm??rgrav , "freebsd-hackers@freebsd.org" Message-Id: <0A599E1F-FF63-4F21-B765-44A4531968DB@bsdimp.com> References: <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> <20151208185818.GD82577@kib.kiev.ua> <20151208190622.GE82577@kib.kiev.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:19:16 -0000 --Apple-Mail=_BC57BC0A-AA3E-464E-BE53-94B0B1DD300A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Dec 8, 2015, at 12:06 PM, Konstantin Belousov = wrote: >=20 > On Tue, Dec 08, 2015 at 08:58:18PM +0200, Konstantin Belousov wrote: >> On Tue, Dec 08, 2015 at 07:44:38PM +0100, Dag-Erling Sm??rgrav wrote: >>> Warner Losh writes: >>>> Dag-Erling Sm??rgrav writes: >>>>> But the filesystem does not know whether the underlying storage is >>>>> electromechanical or solid-state, nor does it know whether the = user >>>>> cares much about seek times (unless we introduce the heuristic >>>>> "avoid creating holes unless the file already has them, in which >>>>> case the userland probably does not care"). >>>> Actually, the filesystem does know. Or has some knowledge of what >>>> is supported and what isn't. BIO_DELETE support is a strong = indicator >>>> of a flash or other log-type system. >>>=20 >>> The filesystem can ask the layer below if BIO_DELETE is supported, = but >>> should not assume anything about what it means. For instance, I = could >>> write a gnop-like module that translates BIO_DELETE into an = all-zeroes >>> BIO_WRITE and passes everything else unmodified. It would provide a >>> stronger guarantee than, say, SATA TRIM but would also have a = completely >>> different performance profile (even on SSDs, since it would do its = work >>> synchronously whereas TRIM works asynchronously). >> I again agree. This is how UFS issues TRIM. When the data block is = freed >> and there are no dandling pointers in the inode copy on disk pointing = to >> the block, BIO_DELETE is issued if volume reports it. Everything = else >> is up to the geom stack and driver. > I am sorry for the followup mail, but I probably have to explain more. > The freed block, for which BIO_DELETE is issued, is not marked as free > in the bitmap, until the BIO_DELETE completion is reported. In other > words, we do not reuse the freed block while TRIM command is possibly > executed. Since the BIO_DELETE queue up in the storage layer, and we make no = requirement on ordering[*] in the storage layer, this is prudent. A BIO_WRITE could = overtake the BIO_DELETE, which would be bad. Warner [*] Well, apart from BIO_ORDERED which isn=E2=80=99t used for BIO_DELETE = requests and generally is only needed in FreeBSD to ensure that writes are = flushed out (with BIO_FLUSH) in a particular order for power-fail recovery. --Apple-Mail=_BC57BC0A-AA3E-464E-BE53-94B0B1DD300A Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJWZy0wAAoJEGwc0Sh9sBEAbJEQAIC1gd3LTznddZhgW+zwheyV JPaStJjcQCHUSb3DRzs+s+r7nMqMmUCSzqTUZEIwVEet8MT3lhUt8vQ+V7eC8EjF pGihuaVXPfqSMBZA9wl4OG1R4szXbNVNtz56ZmZpWXLnLdTwoVdR0Qbbqg5KbtXm cRdYRyAfEP8RJ7FeTB8rmYjsAFrm93R9oqI5415j8jX/eIOUjpsZJOVKmJzNSAMZ DtYMTUtMrx/UHi97PEnh4DhUs8O2D/jGjIke3khQtsLc4oluizMlkuxSq+fOH7sc NOS1pHwSFrj8OK6hh5MrLytHqXYecnouKMSWylyTp0gPsA0QZ+4ua5uJYh0oyWqo w1RjUqTUQQxGQeK61F7/s1cTn2GgR2GyNIWB/XTdn8rUbDZU8439j742Vz7jc8XQ +kQD6SA2H/+7WM0VjvZ/E0EPSIYAu5r4MhwE7xd38HOsfTukBiLUGCELPdqyAlqI rWEQAt7Jbfndi7w6z6lnbIEg7t1tifmXRVgWVMmHBf3hMJD28LdmGBdLYRjACy+e +qg+entLA9sLK2PCv4wZOqCsTiSfM7dLIW+6i4Urau9yfoLhFBpH0d7AcXZTug/D mWxcbW/6DSqBWBXtedpGPnSi5/vv4MD47p9bf3zq1a0s5dALfmm4bgg8/Y/kUreY qlXgGo12PmihfIoLgYNt =qcyS -----END PGP SIGNATURE----- --Apple-Mail=_BC57BC0A-AA3E-464E-BE53-94B0B1DD300A-- From owner-freebsd-hackers@freebsd.org Tue Dec 8 19:20:46 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1E8999D4D87 for ; Tue, 8 Dec 2015 19:20:46 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id D8474178E for ; Tue, 8 Dec 2015 19:20:45 +0000 (UTC) (envelope-from des@des.no) Received: from desk.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id B9952D789; Tue, 8 Dec 2015 19:20:44 +0000 (UTC) Received: by desk.des.no (Postfix, from userid 1001) id 667CE482EF; Tue, 8 Dec 2015 20:20:39 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Konstantin Belousov Cc: "freebsd-hackers\@freebsd.org" Subject: Re: Fwd: DELETE support in the VOP_STRATEGY(9)? References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> <20151208185818.GD82577@kib.kiev.ua> Date: Tue, 08 Dec 2015 20:20:39 +0100 In-Reply-To: <20151208185818.GD82577@kib.kiev.ua> (Konstantin Belousov's message of "Tue, 8 Dec 2015 20:58:18 +0200") Message-ID: <86h9jsrblk.fsf@desk.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:20:46 -0000 Konstantin Belousov writes: > Dag-Erling Sm=C3=B8rgrav writes: > > The filesystem can ask the layer below if BIO_DELETE is supported, but > > should not assume anything about what it means. For instance, I could > > write a gnop-like module that translates BIO_DELETE into an all-zeroes > > BIO_WRITE and passes everything else unmodified. It would provide a > > stronger guarantee than, say, SATA TRIM but would also have a completely > > different performance profile (even on SSDs, since it would do its work > > synchronously whereas TRIM works asynchronously). > I again agree. This is how UFS issues TRIM. When the data block is freed > and there are no dandling pointers in the inode copy on disk pointing to > the block, BIO_DELETE is issued if volume reports it. Everything else > is up to the geom stack and driver. I'm not sure what point you're trying to make. I get the impression that you didn't really read what I wrote, just skimmed it and then included it as decoration, and that you're making the same mistake I'm trying (unsuccessfully, it seems) to prevent Maxim from making. Please go back and read everything I've written before. Yes, it's long(ish), but that's because it's a difficult problem which won't be solved by pretending it's easy and ignoring everybody who tells you it isn't. > > Anyway, my point is that Maxim needs to revise his assumptions. > Which does not invalidate the fact that a 'punch hole' interface is > useful. Allow me to quote my first email on the subject: | I'm not opposed to the idea of VOP_DELETE or similar, but don't assume | that "punching through" is always the correct semantic, and don't assume | that it will be easy to implement - and it will have to be implemented | correctly at every layer, in every geom, in every storage driver etc. | There are many details to work out and many opportunities for mistakes. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@freebsd.org Tue Dec 8 19:28:47 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D1FD99D542B for ; Tue, 8 Dec 2015 19:28:47 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qg0-x22a.google.com (mail-qg0-x22a.google.com [IPv6:2607:f8b0:400d:c04::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E60F1081 for ; Tue, 8 Dec 2015 19:28:47 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by qgec40 with SMTP id c40so32171576qge.2 for ; Tue, 08 Dec 2015 11:28:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=j7Xmo4KEBOb360BKFU02/0ks+poOkrVs8MB8wWQAFtM=; b=q050lqlZddxnXpmkl28VIJrEewm5tt6XTd9xR3vNaaTjA87kh/kUe1D/PBJHCyIkv/ eG3ZafLivCPl4oH8KHNggPhBUXSbmT53OvokJwaSwS/e+nHPM4zB/GOamBWNwlIbszdJ U7wsjI0SAXXS3YveauR5bjC8riIhcoyB/hesd+u3kJDaNdC9vcpcPfSkOGXcz1sdRKSM OFt8CbZ/aNuves9XDC05twN5n54GfFR7oBe0vggK4ORQkVxC35B5oEa+fAU9x+/USo2S jP4SEu8Swm/+c0rBGky+vw3TjQUqR/z4qYSs7V25I/0WvKIQVlmA/iA2IFHklYGctvFK CDbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=j7Xmo4KEBOb360BKFU02/0ks+poOkrVs8MB8wWQAFtM=; b=lZCtdGnj3BcQKCqbh/5V9kN+PWP/wAxgqDiRs+NH/i1rayIInxCIlSgMuq/piYs1jW 03EFCuKqQFMreyVOXCrmx7OwhTI17MR327VKxiJtJgwKqPxv8ctM+0UNr8TEh9wH37OX CkaC34B3YDwThrDH2AQEWBLnzAznTuV+1+eCSbt9opDdQnivwqbHVjVRvsPnmYojsLCC 5YUDjweKInpvJ9Pd35lGg3v9JTdoqC3UDVNsAt/+Ga0a0Lgj4dFZPEkgvPSOz2PqHjkZ haZvqzzU74vCcwM5msLS/LSzzwmM3buF4QRf6TcnG4kGR+9sg15pPZYi+fNzAKUP/yql LRyQ== X-Gm-Message-State: ALoCoQk+IVz40lrw5X+u4kJq4uphkY7Xu2y1C3Y+qP+41liZ3ditq6D6pt3hVKJcSD7fG8gTZ68AeL7Mqf/bl5auN6kO8lhNlg== MIME-Version: 1.0 X-Received: by 10.140.176.143 with SMTP id w137mr7717572qhw.20.1449602926551; Tue, 08 Dec 2015 11:28:46 -0800 (PST) Sender: wlosh@bsdimp.com Received: by 10.140.27.181 with HTTP; Tue, 8 Dec 2015 11:28:46 -0800 (PST) X-Originating-IP: [2601:280:4900:3700:4d3f:8eba:ea86:7700] In-Reply-To: <56672C94.30404@multiplay.co.uk> References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> <566726ED.2010709@multiplay.co.uk> <0DB97CBA-4DC3-4D52-AE9D-54546292D66F@bsdimp.com> <56672C94.30404@multiplay.co.uk> Date: Tue, 8 Dec 2015 12:28:46 -0700 X-Google-Sender-Auth: pAR-Ck3zjeSnWBEyRlIo1iWSzpA Message-ID: Subject: Re: DELETE support in the VOP_STRATEGY(9)? From: Warner Losh To: Steven Hartland Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:28:48 -0000 On Tue, Dec 8, 2015 at 12:16 PM, Steven Hartland wrote: > > > On 08/12/2015 19:03, Warner Losh wrote: > >> On Dec 8, 2015, at 11:52 AM, Steven Hartland >>> wrote: >>> >>> >>> >>> On 08/12/2015 18:44, Dag-Erling Sm=C3=B8rgrav wrote: >>> >>>> Warner Losh writes: >>>> >>>>> Dag-Erling Sm=C3=B8rgrav writes: >>>>> >>>>>> But the filesystem does not know whether the underlying storage is >>>>>> electromechanical or solid-state, nor does it know whether the user >>>>>> cares much about seek times (unless we introduce the heuristic >>>>>> "avoid creating holes unless the file already has them, in which >>>>>> case the userland probably does not care"). >>>>>> >>>>> Actually, the filesystem does know. Or has some knowledge of what >>>>> is supported and what isn't. BIO_DELETE support is a strong indicator >>>>> of a flash or other log-type system. >>>>> >>>> The filesystem can ask the layer below if BIO_DELETE is supported, but >>>> should not assume anything about what it means. For instance, I could >>>> write a gnop-like module that translates BIO_DELETE into an all-zeroes >>>> BIO_WRITE and passes everything else unmodified. It would provide a >>>> stronger guarantee than, say, SATA TRIM but would also have a complete= ly >>>> different performance profile (even on SSDs, since it would do its wor= k >>>> synchronously whereas TRIM works asynchronously). >>>> >>> That ship has sailed. UFS, at least, assumes that if TRIM is supported >> then >> relocating files to be contiguous is bad. >> >> But writing a gnop module that did the BIO_DELETE thing would be bogus. >> BIO_DELETE does not mean that blocks will read back as zeros. But that= =E2=80=99s >> not what BIO_DELETE means. So, sure you could invent a stupid thing that >> breaks the rules, and thus the assumptions of the other code, but why >> would >> you want to do that? >> >> The SATA trims are actually synchronous (in the absence of power >> failures). >> Once you TRIM The data, it is gone. And depending what bits are set in >> the identify response, you can count on different things. But to say the= y >> happen asynchronously because of implementation details about when the >> data >> is actually erased is missing the point. Also, your BIO_DELETE example >> wouldn=E2=80=99t guarantee the data is erased either. Writes to log appe= nd devices >> (like SSDs) are like a TRIM followed by a write: the old LBA mapping is >> discarded and a new one replaces it. >> > > Not all SATA TRIMs are synchronous , some FW does process them in the > background. > > Saying once you TRIM data its gone is actually too strong I'm afraid, as > its advisory, the FW can ignore you if it so chooses. > > There is the concept of DSM deterministic read which if set "should" > result in returning the same values from read of a TRIMed sector every > time, but even this is unreliable due to FW bugs (yes I've seen this). I guess I've been lucky. In FreeBSD we only depend that the data will read without error after a BIO_DELETE and that a subsequent BIO_WRITE will make BIO_READ deterministic again. But I was mostly trying to say that once you issue a TRIM to the drive, and it returns, the TRIM is done in the sense that there's not another TRIM_COMPLETED message that comes back from the drive. > Anyway, my point is that Maxim needs to revise his assumptions. >>>> >>> Just to clarify most consumer devices process TRIM synchronously, not >>> asynchronously. >>> >> It also depends on what you mean by =E2=80=98process=E2=80=99 here. >> > Indeed it does, here I mean when / if the data is removed from the media > by the HW. I agree. Most firmware is asynchronous in this sense. You have to do something called a SECURE ERASE to have the data be actually gone. The granularity of that command, though is the entire drive. > Your example isn't actually just an example CAM scsi_da has a number of >>> different ways it can process BIO_DELETE: >>> * ATA TRIM >>> * SCSI UMAP >>> * Write Same 16 >>> * Write Same 10 >>> * Zero >>> >>> So you example is actually exists in practice in the FreeBSD code base >>> ;-) >>> >> All these are effectively TRIM operations. The devices that implement th= em >> use them as hints to optimize storage. DES=E2=80=99 BIO_DELETE -> WRITE = zero >> example doesn=E2=80=99t optimize storage at all, nor does it give the lo= wer layers >> any clue about how to optimize the storage. All the SCSI delete types >> do give that hint. >> > This is true, just wanted to highlight that "TRIM" can mean very differen= t > things even at the CAM layer. > Agreed. There's many different ways to implement BIO_DELETE's rather loose semantics. This is one reason why we give people the knobs to turn it off if performance is hurt in their application. This is the ultimate escape hatch when the performance profile of BIO_DELETE in the actual drive doesn't match the upper layer's assumptions. Warner From owner-freebsd-hackers@freebsd.org Tue Dec 8 19:29:07 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6BBE39D5496 for ; Tue, 8 Dec 2015 19:29:07 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 3082C1170 for ; Tue, 8 Dec 2015 19:29:07 +0000 (UTC) (envelope-from des@des.no) Received: from desk.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 6FF3BD7A6; Tue, 8 Dec 2015 19:29:06 +0000 (UTC) Received: by desk.des.no (Postfix, from userid 1001) id 04828482F2; Tue, 8 Dec 2015 20:29:05 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Warner Losh Cc: Steven Hartland , freebsd-hackers@freebsd.org Subject: Re: DELETE support in the VOP_STRATEGY(9)? References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> <566726ED.2010709@multiplay.co.uk> <0DB97CBA-4DC3-4D52-AE9D-54546292D66F@bsdimp.com> Date: Tue, 08 Dec 2015 20:29:04 +0100 In-Reply-To: <0DB97CBA-4DC3-4D52-AE9D-54546292D66F@bsdimp.com> (Warner Losh's message of "Tue, 8 Dec 2015 12:03:11 -0700") Message-ID: <86d1ugrb7j.fsf@desk.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:29:07 -0000 Warner Losh writes: > Dag-Erling Sm=C3=B8rgrav writes: > > The filesystem can ask the layer below if BIO_DELETE is supported, but > > should not assume anything about what it means. For instance, I could > > write a gnop-like module that translates BIO_DELETE into an all-zeroes > > BIO_WRITE and passes everything else unmodified. It would provide a > > stronger guarantee than, say, SATA TRIM but would also have a completely > > different performance profile (even on SSDs, since it would do its work > > synchronously whereas TRIM works asynchronously). > That ship has sailed. UFS, at least, assumes that if TRIM is supported > then relocating files to be contiguous is bad. But writing a gnop > module that did the BIO_DELETE thing would be bogus. No less bogus than any other implementation of BIO_DELETE. I could write a gnop-like module that silently discards them. My point is that it's wrong to infer anything else from GEOM::candelete support than the fact that BIO_DELETE requests will be accepted and may or may not do something, somewhere, at some point. We can easily create a different GEOM attribute which indicates that seeks are essentially free, and FFS could use that instead of GEOM::candelete to disable relocation. > BIO_DELETE does not mean that blocks will read back as zeros. It doesn't mean they won't, either. > So, sure you could invent a stupid thing that breaks the rules, and > thus the assumptions of the other code, but why would you want to do > that? It wouldn't break any rules or valid assumptions. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@freebsd.org Tue Dec 8 19:34:19 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 61F669D596C for ; Tue, 8 Dec 2015 19:34:19 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qg0-x232.google.com (mail-qg0-x232.google.com [IPv6:2607:f8b0:400d:c04::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 16DE11A36 for ; Tue, 8 Dec 2015 19:34:19 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by qgea14 with SMTP id a14so32894472qge.0 for ; Tue, 08 Dec 2015 11:34:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Geih0qs8+eAA3NOc4DAgbLBl9st7wVqpt7uYgzrKBL4=; b=G1TrzZ7Qma25GJO/h2RN20nECRlqlLxear9cqZbseHb64YSX0nx7x+dRZofvTMSKah rBl7j7REh4r8YN9mYI49yG+R0cw5iX/Nlvf2tqKk/roGG5gS44WJsME6qyUgXRyfEpgV Gvh465147jQNHW6goI8Pko3Tgi2YxGpBCcDP4aKw7PeMKGi1Gj3hwUc2Xswzs4qdO0EP 3zRJQNloNN3d/l4OKcBct1sXk7l3T7xzGLQa8MvJNU4Ut9vaZz9wVByiWVlXw3QM0Z4I nXvRuWJhX9ugDKiLTTnQSmu62HcGdNO4moPKI2mbSSZu7D096ZE8y9yyB9mQrvgPDcTq 2bPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=Geih0qs8+eAA3NOc4DAgbLBl9st7wVqpt7uYgzrKBL4=; b=G2CLInr9zKXa+R2D3U/e299bWL6daGFDEj0nqD0toNPlpqJ0kfiH9dYL+WX4YMHFwc p21QRngbUYV2st1Piu8NyI4pAhB9CnwDTGU6dUspjJ6ljelSm1pbWFyc5mDnf2iKHLjA VIiF34PblHluU40gm/aqveIDuCEl6w8kF777IHswwVnWUBl3xBiVX6U1z9f9DBaJYD9P 5a47MmGOE0eobuk00VpmKhVDkzj69k1RE9fhYnjFRrc6zy9w/WW92A0B4wjHFbYPaO4o /0gtqAiQt7BObxUdl0isZj6F+GVPUMocyasq5p7XSJY5SFvBQztp7jCO+mF0nZY9HRrc /Tdw== X-Gm-Message-State: ALoCoQlQeg20Tw5BCLEIHq510gV/7WKAFYDRL/glokeaFWRREnmGmNXROcW53a8x2L4uH5nZypmkJuaOH5sx3+JeNMF6McxzlA== MIME-Version: 1.0 X-Received: by 10.140.19.78 with SMTP id 72mr1836090qgg.81.1449603256846; Tue, 08 Dec 2015 11:34:16 -0800 (PST) Sender: wlosh@bsdimp.com Received: by 10.140.27.181 with HTTP; Tue, 8 Dec 2015 11:34:16 -0800 (PST) X-Originating-IP: [2601:280:4900:3700:4d3f:8eba:ea86:7700] In-Reply-To: <86d1ugrb7j.fsf@desk.des.no> References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> <566726ED.2010709@multiplay.co.uk> <0DB97CBA-4DC3-4D52-AE9D-54546292D66F@bsdimp.com> <86d1ugrb7j.fsf@desk.des.no> Date: Tue, 8 Dec 2015 12:34:16 -0700 X-Google-Sender-Auth: UAzj1RRCaEsQO2nM372NHDSGq8g Message-ID: Subject: Re: DELETE support in the VOP_STRATEGY(9)? From: Warner Losh To: =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= Cc: Steven Hartland , "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:34:19 -0000 On Tue, Dec 8, 2015 at 12:29 PM, Dag-Erling Sm=C3=B8rgrav wrot= e: > Warner Losh writes: > > Dag-Erling Sm=C3=B8rgrav writes: > > > The filesystem can ask the layer below if BIO_DELETE is supported, bu= t > > > should not assume anything about what it means. For instance, I coul= d > > > write a gnop-like module that translates BIO_DELETE into an all-zeroe= s > > > BIO_WRITE and passes everything else unmodified. It would provide a > > > stronger guarantee than, say, SATA TRIM but would also have a > completely > > > different performance profile (even on SSDs, since it would do its wo= rk > > > synchronously whereas TRIM works asynchronously). > > That ship has sailed. UFS, at least, assumes that if TRIM is supported > > then relocating files to be contiguous is bad. But writing a gnop > > module that did the BIO_DELETE thing would be bogus. > > No less bogus than any other implementation of BIO_DELETE. I could > write a gnop-like module that silently discards them. My point is that > it's wrong to infer anything else from GEOM::candelete support than the > fact that BIO_DELETE requests will be accepted and may or may not do > something, somewhere, at some point. We can easily create a different > GEOM attribute which indicates that seeks are essentially free, and FFS > could use that instead of GEOM::candelete to disable relocation. When this was implemented, we thought about that. But we couldn't come up with any cases where you'd have one set and not the other. And the actual thing you'd want isn't that seeks are free, though that's a good clue. The actual thing you want is to know if there's a performance benefit to keeping files contiguous, and the extent size where that stops making sense. > > > BIO_DELETE does not mean that blocks will read back as zeros. > > It doesn't mean they won't, either. > > > So, sure you could invent a stupid thing that breaks the rules, and > > thus the assumptions of the other code, but why would you want to do > > that? > > It wouldn't break any rules or valid assumptions. Actually, I'm wrong here. You are correct. I got carried away a bit. Sorry. Warner From owner-freebsd-hackers@freebsd.org Tue Dec 8 19:46:53 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 25ECF9D425B for ; Tue, 8 Dec 2015 19:46:53 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id E0185162B for ; Tue, 8 Dec 2015 19:46:52 +0000 (UTC) (envelope-from des@des.no) Received: from desk.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 6B97AD7FA; Tue, 8 Dec 2015 19:46:51 +0000 (UTC) Received: by desk.des.no (Postfix, from userid 1001) id 0AFE1482F6; Tue, 8 Dec 2015 20:46:50 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Warner Losh Cc: "freebsd-hackers\@freebsd.org" , Steven Hartland Subject: Re: DELETE support in the VOP_STRATEGY(9)? References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> <566726ED.2010709@multiplay.co.uk> <0DB97CBA-4DC3-4D52-AE9D-54546292D66F@bsdimp.com> <86d1ugrb7j.fsf@desk.des.no> Date: Tue, 08 Dec 2015 20:46:50 +0100 In-Reply-To: (Warner Losh's message of "Tue, 8 Dec 2015 12:34:16 -0700") Message-ID: <868u54radx.fsf@desk.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:46:53 -0000 Warner Losh writes: > Dag-Erling Sm=C3=B8rgrav writes: > > My point is that it's wrong to infer anything else from > > GEOM::candelete than the fact that BIO_DELETE requests will be > > accepted and may or may not do something, somewhere, at some point. > > We can easily create a different GEOM attribute which indicates that > > seeks are essentially free, and FFS could use that instead of > > GEOM::candelete to disable relocation. > When this was implemented, we thought about that. But we couldn't come > up with any cases where you'd have one set and not the other. And the > actual thing you'd want isn't that seeks are free, though that's a > good clue. The actual thing you want is to know if there's a > performance benefit to keeping files contiguous, and the extent size > where that stops making sense. I'm having a hard time understanding how the fact that seeks are essentially free is *not* a good indication that there is no benefit to keeping files contiguous, since keeping files contiguous is something we do to avoid the cost of seeking. Support for deletion, on the other hand, is *completely* orthogonal. And my example was not taken entirely out of the blue: I'm sure there would be a huge market for storage devices, whether electromechanical or solid-state, which implemented this in hardware, along with guarantees that reallocated sectors are truly non-recoverable. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@freebsd.org Tue Dec 8 19:51:07 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4BF39D46C4 for ; Tue, 8 Dec 2015 19:51:07 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7E9B41B63 for ; Tue, 8 Dec 2015 19:51:07 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id tB8Jp1Y4001029 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 8 Dec 2015 21:51:02 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua tB8Jp1Y4001029 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id tB8Jp1tQ001026; Tue, 8 Dec 2015 21:51:01 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 8 Dec 2015 21:51:01 +0200 From: Konstantin Belousov To: Dag-Erling Sm??rgrav Cc: "freebsd-hackers@freebsd.org" Subject: Re: Fwd: DELETE support in the VOP_STRATEGY(9)? Message-ID: <20151208195101.GG82577@kib.kiev.ua> References: <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> <20151208185818.GD82577@kib.kiev.ua> <86h9jsrblk.fsf@desk.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86h9jsrblk.fsf@desk.des.no> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:51:08 -0000 On Tue, Dec 08, 2015 at 08:20:39PM +0100, Dag-Erling Sm??rgrav wrote: > Konstantin Belousov writes: > > Dag-Erling Sm??rgrav writes: > > > The filesystem can ask the layer below if BIO_DELETE is supported, but > > > should not assume anything about what it means. For instance, I could > > > write a gnop-like module that translates BIO_DELETE into an all-zeroes > > > BIO_WRITE and passes everything else unmodified. It would provide a > > > stronger guarantee than, say, SATA TRIM but would also have a completely > > > different performance profile (even on SSDs, since it would do its work > > > synchronously whereas TRIM works asynchronously). > > I again agree. This is how UFS issues TRIM. When the data block is freed > > and there are no dandling pointers in the inode copy on disk pointing to > > the block, BIO_DELETE is issued if volume reports it. Everything else > > is up to the geom stack and driver. > > I'm not sure what point you're trying to make. I get the impression > that you didn't really read what I wrote, just skimmed it and then > included it as decoration, and that you're making the same mistake I'm > trying (unsuccessfully, it seems) to prevent Maxim from making. Please > go back and read everything I've written before. Yes, it's long(ish), > but that's because it's a difficult problem which won't be solved by > pretending it's easy and ignoring everybody who tells you it isn't. > > > > Anyway, my point is that Maxim needs to revise his assumptions. > > Which does not invalidate the fact that a 'punch hole' interface is > > useful. > > Allow me to quote my first email on the subject: > > | I'm not opposed to the idea of VOP_DELETE or similar, but don't assume > | that "punching through" is always the correct semantic, and don't assume > | that it will be easy to implement - and it will have to be implemented > | correctly at every layer, in every geom, in every storage driver etc. > | There are many details to work out and many opportunities for mistakes. I wrote that VOP_DELETE() or any other interface should work at the level of data blocks consituing the file data, and its only action should be removing the blocks from the inode, same as is done unlink or truncate. I wrote that any BIO_DELETE bios issued by the filesystem, should be issued by the code which already acts on freeing the blocks. In other words, no new handling or new semantic for BIO_DELETE is required (at least in context of UFS). We already issue BIO_DELETE on UFS. If your argument about being hard or impossbile for VOP_DELETE() is valid, it must be valid for the current UFS code. But I do not see how. From owner-freebsd-hackers@freebsd.org Tue Dec 8 19:58:53 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DDC0D9D4B3E for ; Tue, 8 Dec 2015 19:58:53 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-pa0-x22b.google.com (mail-pa0-x22b.google.com [IPv6:2607:f8b0:400e:c03::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A58821EC2 for ; Tue, 8 Dec 2015 19:58:53 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: by pacdm15 with SMTP id dm15so16906418pac.3 for ; Tue, 08 Dec 2015 11:58:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :message-id:references:to; bh=5nkXaeQP2gLbNfQUV5ZPhSoSccsKFvPkxSHB6s9hBY8=; b=ZfRR6YwF0mo0AVkbTJND+gZbCvleoEGtgwLaDqAuC5XPklztpR2341SETLQnJp3Bng UaGeALzKNDDp1+u9yp3qd/fxbw+Fq+jFmi/7Lwo3ZL6uC2RtRHBne/wNJ7gTK5kfWSCV zjxp3Y6zsrfpPuGCiHMAqYIr4SyecsNRMgc1gunwajCEvsRfJh7mN0k6trkxYf9lQr1A XiNvHkbvMgz15wxbBXnsaBrADmFdSGUI5B+w+05q6/lnJ2t5zLcV2Oqr4Zzme66F3Yj4 cJbJXcIcF6wWq4sSGgzW15yVhJgxybGTGmH9Ct9g6yXZAREmmOJ7aYCJOxf9IPwOgGA9 1jXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:message-id:references:to; bh=5nkXaeQP2gLbNfQUV5ZPhSoSccsKFvPkxSHB6s9hBY8=; b=c8Ivi5DMCHOLd7gSIWRX6wUx2gkfqs56VrfulRTXiS6q/uBBJCN4v+Hht6EJaVBkLW 1KjTDA8wCqrN85N+StLabaNUUvkSfHL+ogjxNF2Y2btLgv/w5bFTGu9wwLO9d5b4bhz2 h7nfoAADeWKqmfwLh3Zi0eYdRbDsHaaRt74jMEQOUy2y20c/l6kFx5d0AQJ8Ys6xDnYZ RPJBb2094alA7UDa+0v3JZlHl899xNNyIvBWU4X8yv/KwO7B6KzuC4jseCd9S5D8mlyz 8TItLmnQnDUJ3GOFrHMGdm7CIAsi08Go+z+G2h2cmR5jVJI5DDEZWKW4fmRC5l69Lrm1 qsiw== X-Gm-Message-State: ALoCoQmMBd97632F+9ooKSXKAvE6aYD6W6nbct6X2uHkIn1CTKQhxhWkOVKOzviWqpE1ZyBt9xnLaprJnbh0vgurkpIfynr1gw== X-Received: by 10.66.248.106 with SMTP id yl10mr2471387pac.140.1449604733102; Tue, 08 Dec 2015 11:58:53 -0800 (PST) Received: from ?IPv6:2601:280:4900:3700:4d3f:8eba:ea86:7700? ([2601:280:4900:3700:4d3f:8eba:ea86:7700]) by smtp.gmail.com with ESMTPSA id e20sm6503645pfb.1.2015.12.08.11.58.51 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 08 Dec 2015 11:58:52 -0800 (PST) Sender: Warner Losh Subject: Re: DELETE support in the VOP_STRATEGY(9)? Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Content-Type: multipart/signed; boundary="Apple-Mail=_EB5456C4-B287-466A-AA13-FAEC4F84D4BB"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5.2 From: Warner Losh In-Reply-To: <868u54radx.fsf@desk.des.no> Date: Tue, 8 Dec 2015 12:58:49 -0700 Cc: "freebsd-hackers@freebsd.org" , Steven Hartland Message-Id: References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> <566726ED.2010709@multiplay.co.uk> <0DB97CBA-4DC3-4D52-AE9D-54546292D66F@bsdimp.com> <86d1ugrb7j.fsf@desk.des.no> <868u54radx.fsf@desk.des.no> To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:58:54 -0000 --Apple-Mail=_EB5456C4-B287-466A-AA13-FAEC4F84D4BB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Dec 8, 2015, at 12:46 PM, Dag-Erling Sm=C3=B8rgrav = wrote: >=20 > Warner Losh writes: >> Dag-Erling Sm=C3=B8rgrav writes: >>> My point is that it's wrong to infer anything else from >>> GEOM::candelete than the fact that BIO_DELETE requests will be >>> accepted and may or may not do something, somewhere, at some point. >>> We can easily create a different GEOM attribute which indicates that >>> seeks are essentially free, and FFS could use that instead of >>> GEOM::candelete to disable relocation. >> When this was implemented, we thought about that. But we couldn't = come >> up with any cases where you'd have one set and not the other. And = the >> actual thing you'd want isn't that seeks are free, though that's a >> good clue. The actual thing you want is to know if there's a >> performance benefit to keeping files contiguous, and the extent size >> where that stops making sense. >=20 > I'm having a hard time understanding how the fact that seeks are > essentially free is *not* a good indication that there is no benefit = to > keeping files contiguous, since keeping files contiguous is something = we > do to avoid the cost of seeking. Support for deletion, on the other > hand, is *completely* orthogonal. And my example was not taken = entirely > out of the blue: I'm sure there would be a huge market for storage > devices, whether electromechanical or solid-state, which implemented > this in hardware, along with guarantees that reallocated sectors are > truly non-recoverable. I=E2=80=99d say it=E2=80=99s not nuanced enough. Seeks may be = essentially from for SSDs, but there=E2=80=99s still some benefit to clustering writes. Only the = SSD=E2=80=99s firmware can know what units it would prefer to write things in so that it = spreads the sectors across however many banks / chips make up one unit internally that can be done in parallel. While not perfect, GEOM::candelete gives an indicate that the device = does storage management. In the vast majority of the cases in actual = hardware, this means that the actual physical media is obscured by at least one layer of indirection. Since there=E2=80=99s the layer of indirection, = assumptions about continuity are out the window. While there may be a tiny fraction where drives try to shoe-horn =E2=80=98reliable erasure=E2=80=99 into data set = management trim operations, or similar, I=E2=80=99d imagine that to get the assurances = they=E2=80=99d want from the OS and filesystems, they=E2=80=99d implement a new primitive or = attribute which would allow them to use a more robust command set to ensure when the feature is engaged it=E2=80=99s working as advertised. So using = GEOM::candelete is good enough until actual problems can be demonstrated. We can do the work then to solve the problem found with it rather than guess about = what the problems might be and design to that. And to be fair, having an additional property of =E2=80=98seeks are = nearly free=E2=80=99 would also be a good way to tell. I=E2=80=99m not convinced it is worth the = effort to add it to all the storage devices in the tree when GEOM::candelete is a good = proxy. I guess if we=E2=80=99re going to the effort, I=E2=80=99d like there to = be a richer set of data provided than just =E2=80=98seeks are nearly free=E2=80=99. Warner --Apple-Mail=_EB5456C4-B287-466A-AA13-FAEC4F84D4BB Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJWZzZ6AAoJEGwc0Sh9sBEARw8QAJ26d8PwzKSUzP9jiOl8NP4e EA9m9EJ4DaikBfB8sndtpq6sQBh9SYBXEr8Xg+FtVmVnJuU70pp1yyepIRCWDoiH Egz7XXkhkdPiTZu6TlwjZCJuj8E+kPBJyInLURx/hsF+TkJ3wzzY2wHeOuloHkKZ uujnllF4nASDmfVxFJmWYpvvP7wlzTA4bFdzKeiF1Fdja/sXIoo/Nm744Vwb56q7 o0bDNSkVZhZLozS1nCz9tGwFq5cyZF0r20J91NnPSD3DmJ/l8dkRJ3RKIoI28cYI A3NjNVHNM4pUQE3Hafx3/H+WkN1rLaSsMOLH6J0ZSVvT5b51BqNVF844LlvGaZpy YngtZnFRrhRoJl/LaMmfSvYUw/vzvrCMpU4uOatGZq9Cq6qNAqAn2EWhe/d1MOe3 ieaa89tIN6A8CR4hgpZl0Y1lUH5DS5fuGLV72aBfDJE6pPeQmepcRQ+qbiDITFk4 I/tW7KBOMko8HbqfSEJ2w3eVAzR5pl9WbL2ChRmhcS8TYvX0McJtQqbDmTW0moFa ZkyKCl1NpBL81OsWyrMEprEq3LEN8uFARGVvftY4zhCnLCbjKqhbCv/tjzpF3TEL ZosfubJFYa/IlpLVC8sp54jDtvxyuJzOXsqxbTsGqYKmIvadJSMxKgo5PU6Yxbfs Jcx6I3DznL0mHMuJ8AQW =3ORU -----END PGP SIGNATURE----- --Apple-Mail=_EB5456C4-B287-466A-AA13-FAEC4F84D4BB-- From owner-freebsd-hackers@freebsd.org Tue Dec 8 20:01:34 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD6D19D4EA3 for ; Tue, 8 Dec 2015 20:01:34 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 756E9112E for ; Tue, 8 Dec 2015 20:01:34 +0000 (UTC) (envelope-from des@des.no) Received: from desk.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id B10AAD822; Tue, 8 Dec 2015 20:01:33 +0000 (UTC) Received: by desk.des.no (Postfix, from userid 1001) id 73F88482FA; Tue, 8 Dec 2015 21:01:27 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Konstantin Belousov Cc: "freebsd-hackers\@freebsd.org" Subject: Re: Fwd: DELETE support in the VOP_STRATEGY(9)? References: <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> <20151208185818.GD82577@kib.kiev.ua> <86h9jsrblk.fsf@desk.des.no> <20151208195101.GG82577@kib.kiev.ua> Date: Tue, 08 Dec 2015 21:01:27 +0100 In-Reply-To: <20151208195101.GG82577@kib.kiev.ua> (Konstantin Belousov's message of "Tue, 8 Dec 2015 21:51:01 +0200") Message-ID: <861tawr9pk.fsf@desk.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 20:01:34 -0000 Konstantin Belousov writes: > If your argument about being hard or impossbile for VOP_DELETE() is > valid, it must be valid for the current UFS code. But I do not see > how. ...because you haven't read what I actually wrote. Please do. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@freebsd.org Tue Dec 8 20:13:12 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D73E99D3ABC for ; Tue, 8 Dec 2015 20:13:12 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 9C8D81D81 for ; Tue, 8 Dec 2015 20:13:12 +0000 (UTC) (envelope-from des@des.no) Received: from desk.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 97549D85B; Tue, 8 Dec 2015 20:13:11 +0000 (UTC) Received: by desk.des.no (Postfix, from userid 1001) id 856A5482FD; Tue, 8 Dec 2015 21:13:09 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Warner Losh Cc: "freebsd-hackers\@freebsd.org" , Steven Hartland Subject: Re: DELETE support in the VOP_STRATEGY(9)? References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> <566726ED.2010709@multiplay.co.uk> <0DB97CBA-4DC3-4D52-AE9D-54546292D66F@bsdimp.com> <86d1ugrb7j.fsf@desk.des.no> <868u54radx.fsf@desk.des.no> Date: Tue, 08 Dec 2015 21:13:09 +0100 In-Reply-To: (Warner Losh's message of "Tue, 8 Dec 2015 12:58:49 -0700") Message-ID: <86wpsopulm.fsf@desk.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 20:13:12 -0000 Warner Losh writes: > And to be fair, having an additional property of =E2=80=98seeks are nearly > free=E2=80=99 would also be a good way to tell. I=E2=80=99m not convinced= it is worth > the effort to add it to all the storage devices in the tree when > GEOM::candelete is a good proxy. I just provided you with an (admittedly fictional, but not unreasonable) example of a layer which implements BIO_DELETE on top of storage that may or may not have free seeks. The two are completely orthogonal; they just happen to be strongly correlated on currently available hardware. Note that my fictional example would guarantee that BIO_DELETEd space reads back as zeroes, even if the request doesn't align with physical block boundaries. Sounds pretty useful to me, even if it doesn't guarantee that the deleted data cannot be recovered from the physical media by a sufficiently determined attacker with access to liquid nitrogen and an electron microscope. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@freebsd.org Tue Dec 8 20:20:30 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D41E9D4006 for ; Tue, 8 Dec 2015 20:20:30 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qg0-x234.google.com (mail-qg0-x234.google.com [IPv6:2607:f8b0:400d:c04::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0C9AF1159 for ; Tue, 8 Dec 2015 20:20:30 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by qgea14 with SMTP id a14so35142954qge.0 for ; Tue, 08 Dec 2015 12:20:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=GJ2jNatMBS0ZJqS2GAinDf2YJJxuJC47TKhH/rAmFH8=; b=JzhvexfJV+yCWvTlqESz0pkwUJuax9ExhPubY+tLgLsGXJlYZdQvAtGMD/RpOStIhA cMyT7CNgUZUC5TzGxusWUqUQQZ24VdRrJZAz9a241EWE0dV25vOiTCGA8CPYVjJRIZJ+ KmvdIwA+q4car1aeaUzfYU1gLZPR3zkMRr04ilsO77tA96G/Z7xut4CmtBxGj13FQwpr j7tfsEvRmV7zoOgxcAL/R9h8f7dD9xVirqrJUTNJR/jXpkK1dsrF+8J1c7hFa1viIQfV Kzp9/9cnznPOq0+N2ci1UwyuucLniKYfaDs1Fde4JtW5qs18PxiyffS8z8xnT60Qxn0X 7vAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=GJ2jNatMBS0ZJqS2GAinDf2YJJxuJC47TKhH/rAmFH8=; b=J3t6uNh+ZTZ4nbpWZb9YgOnEYoTfXevnrp7CkKurhJhOkpHT3n5HKHWM4nJNbqRCmx Ep+XhCt0ewY60ikHCHIfQJLyKSPKviei4oufyB+0axF4pS7CLYUmlJagCttOeu7mGa86 A9CDiTrD0kFZ8nPwli8T4VfD1IeraQFgD6/8j3NRXsLHsBoJjuFsbbo5AIAxgQnCWxLt BDgCAwLw8WmPtUHin1awiPliG0vEUAjEoVSaa5/yEd0aUNAFTqb6ebgMi4OwB3ftZt97 +Y5gMktNHcYgPyRoY7JUMGALYr3S2QOpMRQP/2khhQSSd6xSQm7dfAzFHkV4ibNx/gAO booQ== X-Gm-Message-State: ALoCoQmOioCdmDkOux9sJdq/+apMFW2mP8mRsyPn1JY5bhSgLzXW3lkQBfflmqOkjCltNE5GKWdmuqMzZclUUFgTc/Ld7yQC+Q== MIME-Version: 1.0 X-Received: by 10.55.40.153 with SMTP id o25mr2123871qko.93.1449606028482; Tue, 08 Dec 2015 12:20:28 -0800 (PST) Sender: wlosh@bsdimp.com Received: by 10.140.27.181 with HTTP; Tue, 8 Dec 2015 12:20:28 -0800 (PST) X-Originating-IP: [2601:280:4900:3700:4d3f:8eba:ea86:7700] In-Reply-To: <86wpsopulm.fsf@desk.des.no> References: <201512052002.tB5K2ZEA026540@chez.mckusick.com> <86poyhqsdh.fsf@desk.des.no> <86fuzdqjwn.fsf@desk.des.no> <864mfssxgt.fsf@desk.des.no> <86wpsord9l.fsf@desk.des.no> <566726ED.2010709@multiplay.co.uk> <0DB97CBA-4DC3-4D52-AE9D-54546292D66F@bsdimp.com> <86d1ugrb7j.fsf@desk.des.no> <868u54radx.fsf@desk.des.no> <86wpsopulm.fsf@desk.des.no> Date: Tue, 8 Dec 2015 13:20:28 -0700 X-Google-Sender-Auth: gW0hxhKspDt0jkTB2EYQs93Gh-U Message-ID: Subject: Re: DELETE support in the VOP_STRATEGY(9)? From: Warner Losh To: =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= Cc: "freebsd-hackers@freebsd.org" , Steven Hartland Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 20:20:30 -0000 On Tue, Dec 8, 2015 at 1:13 PM, Dag-Erling Sm=C3=B8rgrav wrote= : > Warner Losh writes: > > And to be fair, having an additional property of =E2=80=98seeks are nea= rly > > free=E2=80=99 would also be a good way to tell. I=E2=80=99m not convinc= ed it is worth > > the effort to add it to all the storage devices in the tree when > > GEOM::candelete is a good proxy. > > I just provided you with an (admittedly fictional, but not unreasonable) > example of a layer which implements BIO_DELETE on top of storage that > may or may not have free seeks. The two are completely orthogonal; they > just happen to be strongly correlated on currently available hardware. > > Note that my fictional example would guarantee that BIO_DELETEd space > reads back as zeroes, even if the request doesn't align with physical > block boundaries. Sounds pretty useful to me, even if it doesn't > guarantee that the deleted data cannot be recovered from the physical > media by a sufficiently determined attacker with access to liquid > nitrogen and an electron microscope. And when there's an actual example, I'm happy to re-examine the use of GEOM::candelete. Warner From owner-freebsd-hackers@freebsd.org Wed Dec 9 06:25:07 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC7119D4C4A for ; Wed, 9 Dec 2015 06:25:07 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from shell1.rawbw.com (shell1.rawbw.com [198.144.192.42]) by mx1.freebsd.org (Postfix) with ESMTP id AB1111ED6 for ; Wed, 9 Dec 2015 06:25:07 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from yuri.doctorlan.com (c-50-184-63-128.hsd1.ca.comcast.net [50.184.63.128]) (authenticated bits=0) by shell1.rawbw.com (8.15.1/8.15.1) with ESMTPSA id tB96P0br035039 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Tue, 8 Dec 2015 22:25:01 -0800 (PST) (envelope-from yuri@rawbw.com) X-Authentication-Warning: shell1.rawbw.com: Host c-50-184-63-128.hsd1.ca.comcast.net [50.184.63.128] claimed to be yuri.doctorlan.com Subject: Re: How to get the deterministic result for FreeBSD tar(1)? To: Freebsd hackers list References: <5666B828.5000306@rawbw.com> From: Yuri Message-ID: <5667C93C.60307@rawbw.com> Date: Tue, 8 Dec 2015 22:25:00 -0800 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <5666B828.5000306@rawbw.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Dec 2015 06:25:07 -0000 On 12/08/2015 02:59, Yuri wrote: > So I have two questions: > 1. How do I actually achieve the output determinism for tar(1)? > 2. Is there an agreement that this is a bug that too long or non-ASCII > path name triggers the leakage of ctime into a tar file? Answering to myself: turns out this is a major bug in libarchive, making tar(1) in general unable to produce the deterministic output. https://github.com/libarchive/libarchive/pull/623 Yuri From owner-freebsd-hackers@freebsd.org Wed Dec 9 09:43:50 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0811C9D4CE1 for ; Wed, 9 Dec 2015 09:43:50 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from mo6-p00-ob.smtp.rzone.de (mo6-p00-ob.smtp.rzone.de [IPv6:2a01:238:20a:202:5300::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.smtp.rzone.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9F99A1DBD for ; Wed, 9 Dec 2015 09:43:49 +0000 (UTC) (envelope-from joerg@britannica.bec.de) X-RZG-AUTH: :JiIXek6mfvEEUpFQdo7Fj1/zg48CFjWjQuEfXeSt/nWoxdY2dvuAIbsw5PvjGQjhWhuW+2PsshoPXoZabJAznZsNufRmXw== X-RZG-CLASS-ID: mo00 Received: from britannica.bec.de (p20030057E21B0F007466C19E73AFB0E3.dip0.t-ipconnect.de [IPv6:2003:57:e21b:f00:7466:c19e:73af:b0e3]) by smtp.strato.de (RZmta 37.14 AUTH) with ESMTPSA id f02a06rB99hi6Io (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (curve secp521r1 with 521 ECDH bits, eq. 15360 bits RSA)) (Client did not present a certificate) for ; Wed, 9 Dec 2015 10:43:44 +0100 (CET) Date: Wed, 9 Dec 2015 10:43:42 +0100 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org Subject: Re: How to get the deterministic result for FreeBSD tar(1)? Message-ID: <20151209094342.GA31098@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org References: <5666B828.5000306@rawbw.com> <5667C93C.60307@rawbw.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5667C93C.60307@rawbw.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Dec 2015 09:43:50 -0000 On Tue, Dec 08, 2015 at 10:25:00PM -0800, Yuri wrote: > On 12/08/2015 02:59, Yuri wrote: > >So I have two questions: > >1. How do I actually achieve the output determinism for tar(1)? > >2. Is there an agreement that this is a bug that too long or non-ASCII > >path name triggers the leakage of ctime into a tar file? > > Answering to myself: turns out this is a major bug in libarchive, making > tar(1) in general unable to produce the deterministic output. As you have been told, there are mechanisms like mtree that can be used to explicitly control data. As such, please keep a sense of proportion -- it's hardly a major bug. Joerg From owner-freebsd-hackers@freebsd.org Thu Dec 10 03:24:36 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 87EFF9D528B; Thu, 10 Dec 2015 03:24:36 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-1.mit.edu (dmz-mailsec-scanner-1.mit.edu [18.9.25.12]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1F9601DC6; Thu, 10 Dec 2015 03:24:35 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 1209190c-f79c96d00000038e-f1-5668ef3ec876 Received: from mailhub-auth-4.mit.edu ( [18.7.62.39]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by dmz-mailsec-scanner-1.mit.edu (Symantec Messaging Gateway) with SMTP id 67.5C.00910.E3FE8665; Wed, 9 Dec 2015 22:19:26 -0500 (EST) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id tBA3JPAM026925; Wed, 9 Dec 2015 22:19:26 -0500 Received: from multics.mit.edu (system-low-sipb.mit.edu [18.187.2.37]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id tBA3JLUQ007465 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 9 Dec 2015 22:19:25 -0500 Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id tBA3JL0N021673; Wed, 9 Dec 2015 22:19:21 -0500 (EST) Date: Wed, 9 Dec 2015 22:19:20 -0500 (EST) From: Benjamin Kaduk X-X-Sender: kaduk@multics.mit.edu To: freebsd-hackers@FreeBSD.org cc: freebsd-current@FreeBSD.org Subject: call for 2015Q4 quarterly status reports Message-ID: User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrJIsWRmVeSWpSXmKPExsUixG6nrmv3PiPMYG6XssWcNx+YLLZv/sfo wOQx49N8lgDGKC6blNSczLLUIn27BK6Ms8ePsBdc4Ko42ODQwPiao4uRg0NCwETiZqdqFyMn kCkmceHeerYuRi4OIYHFTBLb/55khHA2MEq8n/gByjnIJLFx/zkWkBYhgXqJszdXMoLYLAJa ErOuL2EHsdkE1CTWr7jGDDFWUWLzqUlgtoiAvMS+pvdgNcxA9pbVk9lAbGEBQ4nuD/eYQS7i FXCUuL2iFCQsKqAjsXr/FLBVvAKCEidnPmGBaNWSWD59G8sERoFZSFKzkKQWMDKtYpRNya3S zU3MzClOTdYtTk7My0st0jXUy80s0UtNKd3ECApBTkmeHYxvDiodYhTgYFTi4b3gkh4mxJpY VlyZe4hRkoNJSZR309OMMCG+pPyUyozE4oz4otKc1OJDjBIczEoivMzPgXK8KYmVValF+TAp aQ4WJXHeuV98w4QE0hNLUrNTUwtSi2CyMhwcShK8f94CNQoWpaanVqRl5pQgpJk4OEGG8wAN 5wep4S0uSMwtzkyHyJ9iVJQS5/0HkhAASWSU5sH1glPEbibVV4ziQK8I8+q/A6riAaYXuO5X QIOZgAZ/uZIOMrgkESEl1cDoI1i9dMnST7EXF27J2LtplbJOAdfOjt2HVDonnPPZ+nfHned8 TDezs3p2XXmf/6ZhWuZxk4OXF7Kr7bq5cCf7raMpxUvlNv+44inK5SAVvjZDv1Bu4a/5dxau jX2gUnxnlkyK6Bc/hWVG76Z6WvaYX8qacG/14ROHbbr2ex+5Xhfv/H6+ziUOEyWW4oxEQy3m ouJEAFMNv0/sAgAA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2015 03:24:36 -0000 Dear FreeBSD Community, The deadline for the next FreeBSD Quarterly Status update is January 7, 2016, for work done in October through December. Status report submissions do not have to be very long. They may be about anything happening in the FreeBSD project and community, and provide a great way to inform FreeBSD users and developers about what you're working on. Submission of reports is not restricted to committers. Anyone doing anything interesting and FreeBSD-related can -- and should -- write one! The preferred and easiest submission method is to use the XML generator [1] with the results emailed to the status report team at monthly at freebsd.org . There is also an XML template [2] which can be filled out manually and attached if preferred. For the expected content and style, please study our guidelines on how to write a good status report [3]. You can also review previous issues [4][5] for ideas on the style and format. We are looking forward to all of your 2015Q4 reports! Thanks, Ben (on behalf of monthly@) [1] http://www.freebsd.org/cgi/monthly.cgi [2] http://www.freebsd.org/news/status/report-sample.xml [3] http://www.freebsd.org/news/status/howto.html [4] http://www.freebsd.org/news/status/report-2015-04-2015-06.html [5] http://www.freebsd.org/news/status/report-2015-07-2015-09.html From owner-freebsd-hackers@freebsd.org Thu Dec 10 18:26:09 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8D42E9D768A for ; Thu, 10 Dec 2015 18:26:09 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x22a.google.com (mail-wm0-x22a.google.com [IPv6:2a00:1450:400c:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 28AAC14AA for ; Thu, 10 Dec 2015 18:26:08 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x22a.google.com with SMTP id u63so35669801wmu.0 for ; Thu, 10 Dec 2015 10:26:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-type:content-transfer-encoding; bh=FBE+49qUV5bE8KYHYoAAd+QnnL+AKeNI5fF52oh7DVM=; b=oq6eaQjDhNAmC6NACH0et8704IDa6xaEHOrJ7Q8FWNGb+B9mXSnmjv7FWgkGcvekhF /yvbaZM0q2cZ3h+X8vbfyB5Z4ak8zu4mDvQ5NRKQ1vTleqviJidm5GUuUJyvjQxVZIlh MV1Dd3sGV4lhO7GRzUC/cma1FNXJ64h73xmAqdfa3VOTzcAyWxG2g2YKerF9yNKJUbM5 qW4V73u3UG/7ilaiI23EzTeSJHirKNm6kOnby9+f3k5wDRnH77WsAvmZ0R2LKdePj5Y+ 5/uakMmOzi2XkJRqe+Ci1EWU/r4LVKxe8L1JkTTqy1EWyi6iRZwOftV9lOfTgJ9lbRjv Lwzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=FBE+49qUV5bE8KYHYoAAd+QnnL+AKeNI5fF52oh7DVM=; b=nCr0kiybtTz6GFVX7Qdpl4SAc4AKfUSNK7s+4ddQQAvdKCAQmCdN4Q0Sdvrazpkrqq d2QmfCPQOYL6AuXx6T3iAraNgKoB0nyMnOZVGlfu66PZtYriHvvPeXsJfXcGhW7jkCjy PtWBBdeHYSVX5Sn2l+5mrXedkygdI5FgaBX2ZDINfaqrUtr6ul3pMsNbnaJckZQyKNj9 WJesVJ8IP++jVHDexinS3CXZQmddKIlPEWG7qkIzwl4EYdUaXDOjmVauSnG/7N8FQNWJ kjVh3vXtJmjnF2i2Or2YiT/ykW69NThuTu7SfcZ605oGPidP2OpceMd9SEZSMRppcLD5 9XwQ== X-Gm-Message-State: ALoCoQkhq46Z7IJSyt0MxTTaZVwTxVR5MxWzRMtmWngU3+ZUljRRiozynrUSYoH785lvEerF9K28Rzk2IVUVOm4qb3j2fBoz8A== X-Received: by 10.28.148.133 with SMTP id w127mr559044wmd.92.1449771967060; Thu, 10 Dec 2015 10:26:07 -0800 (PST) Received: from [10.10.1.58] (liv3d.labs.multiplay.co.uk. [82.69.141.171]) by smtp.gmail.com with ESMTPSA id u17sm14072918wmd.8.2015.12.10.10.26.05 for (version=TLSv1/SSLv3 cipher=OTHER); Thu, 10 Dec 2015 10:26:05 -0800 (PST) Subject: Re: EFI/ZFS Update: successful tests, need more complex vdevs To: freebsd-hackers@freebsd.org References: From: Steven Hartland Message-ID: <5669C3BD.4010402@multiplay.co.uk> Date: Thu, 10 Dec 2015 18:26:05 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2015 18:26:09 -0000 We've used Eric's hard work which is currently under review here: https://reviews.freebsd.org/D4104 I'm pleased to report we can now successfully EFI boot root ZFS from a raidz2 pool on Intel P3700 NVMe drives :) Here's a guide for those interested: http://blog.multiplay.co.uk/2015/12/freebsd-10-2-release-efi-zfs-root-boot/ On 04/11/2015 12:35, krad wrote: > is there not anyway freebsd could provide patched signed binaries outside > the main distros for testing purposes, as it should be fairly straight > forward to drop them in? I think you might be a much bigger audience for > testing then? > > On 2 November 2015 at 19:16, Gabor Radnai wrote: > >> Appreciate, thank you very much. I think my confusion is because the latest >> patch you provided on the list >> on *Fri Oct 23 11:19:07 UTC 2015* >> >> http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20151023/db1ac571/attachment.bin >> >> is not a full patch but a diff to your original patch. Anyhow, I'm ok now >> applying original patch and this latest diff everything seems fine >> (apart that for some reason my server dislikes booting automatically from >> the EFI partition, manually loading bootx64.efi works like charm. >> but that's definitely nothing to do with your great work). >> >> Thanks. >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >> > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Thu Dec 10 22:07:55 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1DE909D70F8 for ; Thu, 10 Dec 2015 22:07:55 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x233.google.com (mail-wm0-x233.google.com [IPv6:2a00:1450:400c:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B2F1E1FBE for ; Thu, 10 Dec 2015 22:07:54 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x233.google.com with SMTP id u63so42450289wmu.0 for ; Thu, 10 Dec 2015 14:07:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-type:content-transfer-encoding; bh=9958bTxCqPxNI3gPHirF4sYPuTCxHuHmbnxZH771pMM=; b=t8leky1d+Vj+VAy908cYhS8+i75uPt5qeT5N+i9N6K6YlvmW2yD/Y/S3/X4uuOmCYZ 3VzrL/KIsFwiFE9L56cCJfaReSaMzZVTWsuiXi+rw2l3/NMIvV0n3V7J4HO00ee53H0F 5vhuL0NcEvyU9uUfIuBVw7TxveOE5VbC1bSsCkstp84eIa5mncnY7vuiCugoJsWMKxOy l4IZ8xf+WXSo9oQZh9n4RGPQcU5dzLZgV93zHfcNQxt261yPMIYmYWoPNxi1bQIyIGhD 6WSRZc3P8GBgoPN7LZG25rcChV/7HFbYh/YADCw+VpZVBO3seektZR7yMU+xVIlnrBDn kk/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=9958bTxCqPxNI3gPHirF4sYPuTCxHuHmbnxZH771pMM=; b=QnEFr2rrhA5U3uNDaPRHiivLVT95k/fKCsKq/RIVTDVBRPULRtW53neeq/4WbeDOek GTKtA7ksPPmNNYso4FGMpi9sWkDXbP1H0ZKsbH2t05AUZW4LmtAoczwiMnDIVSdfhApB pZ4FatFfu9i6BKECCY6IZo0hbRGYagdVxUkpBCG9H6R1ubAYVA+XegqK+lzuuidlAwgU IQut4AaRmVyDEVe13f1cT9gnB/1glBCAKrCT7TDMSKbrrNIdZTlMfW7+U4IN9qqjOUTq X6ozFzx+ylldoFvXIL4ateQM1BqVRAU0oEx2FYYAePpAe0ikbKzWucPysu4adAUMts8a 7h5Q== X-Gm-Message-State: ALoCoQlXaliYDvgIBVmXS1wix2VSqeabICM/Rhk9f/hbfiNjbRN1CeNVi6HdjYr+92YkHLnRah8kO51q6yTj6WtLMJxGueD1NA== X-Received: by 10.28.147.203 with SMTP id v194mr1518314wmd.16.1449785272988; Thu, 10 Dec 2015 14:07:52 -0800 (PST) Received: from [10.10.1.58] (liv3d.labs.multiplay.co.uk. [82.69.141.171]) by smtp.gmail.com with ESMTPSA id 198sm661515wmr.18.2015.12.10.14.07.51 for (version=TLSv1/SSLv3 cipher=OTHER); Thu, 10 Dec 2015 14:07:51 -0800 (PST) Subject: Re: EFI/ZFS Update: successful tests, need more complex vdevs To: freebsd-hackers@freebsd.org References: <5669C3BD.4010402@multiplay.co.uk> From: Steven Hartland Message-ID: <5669F7B8.7030706@multiplay.co.uk> Date: Thu, 10 Dec 2015 22:07:52 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <5669C3BD.4010402@multiplay.co.uk> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2015 22:07:55 -0000 I have to take it back, while single disk worked fine as soon as we tried raidz2 we get: Initializing modules: ZFS UFS ZFS: i/o error - all block copies unavailable ZFS: i/o error - all block copies unavailable ZFS: can't find root dsl_dir ZFS: can't find root filesystem ... So some more work to be done yet it seems. On 10/12/2015 18:26, Steven Hartland wrote: > We've used Eric's hard work which is currently under review here: > https://reviews.freebsd.org/D4104 > > I'm pleased to report we can now successfully EFI boot root ZFS from a > raidz2 pool on Intel P3700 NVMe drives :) > > Here's a guide for those interested: > http://blog.multiplay.co.uk/2015/12/freebsd-10-2-release-efi-zfs-root-boot/ > > > On 04/11/2015 12:35, krad wrote: >> is there not anyway freebsd could provide patched signed binaries >> outside >> the main distros for testing purposes, as it should be fairly straight >> forward to drop them in? I think you might be a much bigger audience for >> testing then? >> >> On 2 November 2015 at 19:16, Gabor Radnai >> wrote: >> >>> Appreciate, thank you very much. I think my confusion is because the >>> latest >>> patch you provided on the list >>> on *Fri Oct 23 11:19:07 UTC 2015* >>> >>> http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20151023/db1ac571/attachment.bin >>> >>> >>> is not a full patch but a diff to your original patch. Anyhow, I'm >>> ok now >>> applying original patch and this latest diff everything seems fine >>> (apart that for some reason my server dislikes booting automatically >>> from >>> the EFI partition, manually loading bootx64.efi works like charm. >>> but that's definitely nothing to do with your great work). >>> >>> Thanks. >>> _______________________________________________ >>> freebsd-hackers@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>> To unsubscribe, send any mail to >>> "freebsd-hackers-unsubscribe@freebsd.org" >>> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to >> "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Fri Dec 11 04:52:17 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 373809D6DE6 for ; Fri, 11 Dec 2015 04:52:17 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x22f.google.com (mail-wm0-x22f.google.com [IPv6:2a00:1450:400c:c09::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CAAFF1C3B for ; Fri, 11 Dec 2015 04:52:16 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by wmnn186 with SMTP id n186so15040133wmn.0 for ; Thu, 10 Dec 2015 20:52:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-type:content-transfer-encoding; bh=cZ2mTPFp1wSouEybzYxIMSr7DWMEl23g5cZ6oYu5Wcg=; b=weTIvnQH55sEe3Y7vpC+VzSeUKTneGjDoEr2BiIsX3qrMUh+f4G9kD2RJAYh1Aq5SU AQI7DpYxw7FWLPHpC+E82SETdWzPkeobXLLTx/eezUaj1SGj+xSwbBc1Zxfn6wiSIa19 076Z3CJdlv7CquUl6fQ5PgkYUZ/N1uKW01n3EVIDa8ojT+HuVDTAzGCYijPcf9Lwbnpm SqeTFdM9UwjQZc7MICw6kn5pfyFDq/2DLBoJjWMz8rsJ9EjMBCacorWLm5u1+SdNaDLK XhQ0s9gcoQ6iPH9HiDZSxeNWkeeVSJGYZDtcWRtuj/vgDo/7ZfrGB76pLKfbllAdjlhl MKgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=cZ2mTPFp1wSouEybzYxIMSr7DWMEl23g5cZ6oYu5Wcg=; b=JHZDt2yfbpB+SWdfvnvT1D5rqFPuK0wfKxy2XN18bMTv+ZB2A+NK+NoSjnXL0oeoTm 3exnrJRH+5IEn46N2t62/ORyX90acBSDL3lR+evSu3pJLbkF9VDVF4X7vUCpVGqnQ07F 2I+AHUMb1N5g2vehAP3qcaTntMKfNXpo/hBwRY1wB9vQ41FlbYsntgTPgWkOQaEimjR1 JLlsSfBzuDbthx3H8AE1CM2ZIraDSM6O6dsHin9fpcHdQ2viaxaHUWmaO/GxAQt9k2qQ 23aD/i2JKGjhFZV8z4fSusNLaiSRteMOpS3llOrBd1lTAPq6LCHpDgRTzcZRzBhDiAFD CfKg== X-Gm-Message-State: ALoCoQkBefw7zWW4vv6jdMnVWNTPUoKZC0hqOWeqm4fxSGRm+UmwzEzJJMrPW1H2aDsrQ7au8deQXNwOM/Y96VkESEkJY11NLA== X-Received: by 10.194.222.135 with SMTP id qm7mr16924617wjc.106.1449809534047; Thu, 10 Dec 2015 20:52:14 -0800 (PST) Received: from [10.10.1.58] (liv3d.labs.multiplay.co.uk. [82.69.141.171]) by smtp.gmail.com with ESMTPSA id bg10sm15193041wjb.46.2015.12.10.20.52.12 for (version=TLSv1/SSLv3 cipher=OTHER); Thu, 10 Dec 2015 20:52:13 -0800 (PST) Subject: Re: EFI/ZFS Update: successful tests, need more complex vdevs To: freebsd-hackers@freebsd.org References: <5669C3BD.4010402@multiplay.co.uk> <5669F7B8.7030706@multiplay.co.uk> From: Steven Hartland Message-ID: <566A567D.2060101@multiplay.co.uk> Date: Fri, 11 Dec 2015 04:52:13 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <5669F7B8.7030706@multiplay.co.uk> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 04:52:17 -0000 Issue fixed, review updated with new patch in the comments so raidz is now bootable :) Regards Steve On 10/12/2015 22:07, Steven Hartland wrote: > I have to take it back, while single disk worked fine as soon as we > tried raidz2 we get: > > Initializing modules: ZFS UFS > ZFS: i/o error - all block copies unavailable > ZFS: i/o error - all block copies unavailable > ZFS: can't find root dsl_dir > ZFS: can't find root filesystem > ... > > So some more work to be done yet it seems. > > On 10/12/2015 18:26, Steven Hartland wrote: >> We've used Eric's hard work which is currently under review here: >> https://reviews.freebsd.org/D4104 >> >> I'm pleased to report we can now successfully EFI boot root ZFS from >> a raidz2 pool on Intel P3700 NVMe drives :) >> >> Here's a guide for those interested: >> http://blog.multiplay.co.uk/2015/12/freebsd-10-2-release-efi-zfs-root-boot/ >> >> >> On 04/11/2015 12:35, krad wrote: >>> is there not anyway freebsd could provide patched signed binaries >>> outside >>> the main distros for testing purposes, as it should be fairly straight >>> forward to drop them in? I think you might be a much bigger audience >>> for >>> testing then? >>> >>> On 2 November 2015 at 19:16, Gabor Radnai >>> wrote: >>> >>>> Appreciate, thank you very much. I think my confusion is because >>>> the latest >>>> patch you provided on the list >>>> on *Fri Oct 23 11:19:07 UTC 2015* >>>> >>>> http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20151023/db1ac571/attachment.bin >>>> >>>> >>>> is not a full patch but a diff to your original patch. Anyhow, I'm >>>> ok now >>>> applying original patch and this latest diff everything seems fine >>>> (apart that for some reason my server dislikes booting >>>> automatically from >>>> the EFI partition, manually loading bootx64.efi works like charm. >>>> but that's definitely nothing to do with your great work). >>>> >>>> Thanks. >>>> _______________________________________________ >>>> freebsd-hackers@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>> To unsubscribe, send any mail to >>>> "freebsd-hackers-unsubscribe@freebsd.org" >>>> >>> _______________________________________________ >>> freebsd-hackers@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>> To unsubscribe, send any mail to >>> "freebsd-hackers-unsubscribe@freebsd.org" >> > From owner-freebsd-hackers@freebsd.org Fri Dec 11 08:10:20 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36B4E9D512A for ; Fri, 11 Dec 2015 08:10:20 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "tensor.andric.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F3C571F21 for ; Fri, 11 Dec 2015 08:10:19 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from [IPv6:2001:7b8:3a7::6570:9019:4e1c:8d98] (unknown [IPv6:2001:7b8:3a7:0:6570:9019:4e1c:8d98]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id CB9026428; Fri, 11 Dec 2015 09:10:10 +0100 (CET) Subject: Re: EFI/ZFS Update: successful tests, need more complex vdevs Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Content-Type: multipart/signed; boundary="Apple-Mail=_D2EC556D-7FF0-4687-B1F9-A5441BB575F0"; protocol="application/pgp-signature"; micalg=pgp-sha1 X-Pgp-Agent: GPGMail 2.6b2 From: Dimitry Andric In-Reply-To: <5669C3BD.4010402@multiplay.co.uk> Date: Fri, 11 Dec 2015 09:10:02 +0100 Cc: freebsd-hackers@freebsd.org Message-Id: References: <5669C3BD.4010402@multiplay.co.uk> To: Steven Hartland X-Mailer: Apple Mail (2.3112) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 08:10:20 -0000 --Apple-Mail=_D2EC556D-7FF0-4687-B1F9-A5441BB575F0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 10 Dec 2015, at 19:26, Steven Hartland = wrote: >=20 > We've used Eric's hard work which is currently under review here: > https://reviews.freebsd.org/D4104 >=20 > I'm pleased to report we can now successfully EFI boot root ZFS from a = raidz2 pool on Intel P3700 NVMe drives :) >=20 > Here's a guide for those interested: > = http://blog.multiplay.co.uk/2015/12/freebsd-10-2-release-efi-zfs-root-boot= / The patch set linked there, at: http://blog.multiplay.co.uk/dropzone/freebsd-10-efi-zfs-boot.tgz results in a 404. Where is it? :-) -Dimitry --Apple-Mail=_D2EC556D-7FF0-4687-B1F9-A5441BB575F0 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.28 iEYEARECAAYFAlZqhOIACgkQsF6jCi4glqPL5gCg/qjVOgfj30WjCCCpuTHbLwgA /dEAn0SMfRNSTZZXdjWWnCuCU8WJedel =p7VL -----END PGP SIGNATURE----- --Apple-Mail=_D2EC556D-7FF0-4687-B1F9-A5441BB575F0-- From owner-freebsd-hackers@freebsd.org Fri Dec 11 10:47:34 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E21609D6E75 for ; Fri, 11 Dec 2015 10:47:33 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 80A401D67 for ; Fri, 11 Dec 2015 10:47:33 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x235.google.com with SMTP id c17so6659376wmd.1 for ; Fri, 11 Dec 2015 02:47:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-type:content-transfer-encoding; bh=YUr+iOADehf6YVvYuOZ5her3dwVgih20cjj3rbLVC7M=; b=ttYy/7g9idZUIpcGvONWh80tYS50Si1z1jCwcEdZN/CYYFPfXsg5Y4PkI3wuVhcGc9 YAYjmmxxpo2j+AnXMsYO/nhMSyDZL1EG+2eGZRrenTADu5Go0sDThL1ZCd1QsLncFs76 3U/xkrJ9gRSKIgneZFPvMZfdR+jhna9KPwDvQXWJJ7BYo2RzgN9WxLVoARkYxu5GoTaA eRdsuSYRV5XVX8zZExiK9bLYY0CUG8Xl8qPLrcg+c4A9pyyj8gx5TcoVnwYVW9RmzcIs X+99MHm6fIlZL3ynw5qy0PQsBOpBB46MvGOy4lVF080yV/ws00WNBnxYCrqIw4euUXZ7 alOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=YUr+iOADehf6YVvYuOZ5her3dwVgih20cjj3rbLVC7M=; b=GP5rCZdzTTshgMbKRJ0op1OlLOYCLlefkeEQxO0ghKn4BtmlXrcIcnLO/1oSP2EndG FD+ITX1lBKtyKJ0tPUIzjdi0SQ41k20HpeF+1v0bIvuKSAUi5BaY350ZKodzi55XcDqQ BMJsIi4aHFReGKChQsXoLZNQOMUtHTJDp8yBfjKKkVQT3diNYO++VnKu/vLZMtulObDl oeDC1MFjT5dKodgDqIrASwQTZSiNpoWbbwZS8te+3AsOZ//zumAWChUBdOzRoRbBjHTy 1AwcO5uUu5j34RbxpTdMKS2d2lp5+ZkcyGR4UPfVaNkhl53dAiER23ZAiDsheJUd/Q9F RwAA== X-Gm-Message-State: ALoCoQmp36kDl6CXmyjSt13/HxFR2tO3IaW69AohZiNW+qQ66oOS2i8EN28+hIVVhE3+/AlHtLRrIz/fnKB0SGZ+E1Znq3Tl5g== X-Received: by 10.194.250.39 with SMTP id yz7mr20642187wjc.92.1449830851563; Fri, 11 Dec 2015 02:47:31 -0800 (PST) Received: from [10.10.1.58] (liv3d.labs.multiplay.co.uk. [82.69.141.171]) by smtp.gmail.com with ESMTPSA id v196sm2825629wmv.10.2015.12.11.02.47.30 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 11 Dec 2015 02:47:30 -0800 (PST) Subject: Re: EFI/ZFS Update: successful tests, need more complex vdevs To: Dimitry Andric References: <5669C3BD.4010402@multiplay.co.uk> Cc: freebsd-hackers@freebsd.org From: Steven Hartland Message-ID: <566AA9C3.8040700@multiplay.co.uk> Date: Fri, 11 Dec 2015 10:47:31 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 10:47:34 -0000 On 11/12/2015 08:10, Dimitry Andric wrote: > On 10 Dec 2015, at 19:26, Steven Hartland wrote: >> We've used Eric's hard work which is currently under review here: >> https://reviews.freebsd.org/D4104 >> >> I'm pleased to report we can now successfully EFI boot root ZFS from a raidz2 pool on Intel P3700 NVMe drives :) >> >> Here's a guide for those interested: >> http://blog.multiplay.co.uk/2015/12/freebsd-10-2-release-efi-zfs-root-boot/ > The patch set linked there, at: > > http://blog.multiplay.co.uk/dropzone/freebsd-10-efi-zfs-boot.tgz > > results in a 404. Where is it? :-) > Ooops, link corrected its at: http://blog.multiplay.co.uk/dropzone/freebsd/freebsd-10-efi-zfs-boot.tgz Regards Steve From owner-freebsd-hackers@freebsd.org Fri Dec 11 11:04:17 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 847FD9D7AEA for ; Fri, 11 Dec 2015 11:04:17 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [IPv6:2001:7b8:3a7:1:2d0:b7ff:fea0:8c26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "tensor.andric.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2ECB3181F for ; Fri, 11 Dec 2015 11:04:17 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from [IPv6:2001:7b8:3a7::6570:9019:4e1c:8d98] (unknown [IPv6:2001:7b8:3a7:0:6570:9019:4e1c:8d98]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id CEE64665F; Fri, 11 Dec 2015 12:04:12 +0100 (CET) Subject: Re: EFI/ZFS Update: successful tests, need more complex vdevs Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Content-Type: multipart/signed; boundary="Apple-Mail=_E9525338-209A-48A7-A72A-F8F345DD7720"; protocol="application/pgp-signature"; micalg=pgp-sha1 X-Pgp-Agent: GPGMail 2.6b2 From: Dimitry Andric In-Reply-To: <566AA9C3.8040700@multiplay.co.uk> Date: Fri, 11 Dec 2015 12:04:12 +0100 Cc: freebsd-hackers@freebsd.org Message-Id: References: <5669C3BD.4010402@multiplay.co.uk> <566AA9C3.8040700@multiplay.co.uk> To: Steven Hartland X-Mailer: Apple Mail (2.3112) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 11:04:17 -0000 --Apple-Mail=_E9525338-209A-48A7-A72A-F8F345DD7720 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On 11 Dec 2015, at 11:47, Steven Hartland = wrote: >=20 > On 11/12/2015 08:10, Dimitry Andric wrote: >> On 10 Dec 2015, at 19:26, Steven Hartland = wrote: >>> We've used Eric's hard work which is currently under review here: >>> https://reviews.freebsd.org/D4104 >>>=20 >>> I'm pleased to report we can now successfully EFI boot root ZFS from = a raidz2 pool on Intel P3700 NVMe drives :) >>>=20 >>> Here's a guide for those interested: >>> = http://blog.multiplay.co.uk/2015/12/freebsd-10-2-release-efi-zfs-root-boot= / >> The patch set linked there, at: >>=20 >> http://blog.multiplay.co.uk/dropzone/freebsd-10-efi-zfs-boot.tgz >>=20 >> results in a 404. Where is it? :-) >>=20 > Ooops, link corrected its at: = http://blog.multiplay.co.uk/dropzone/freebsd/freebsd-10-efi-zfs-boot.tgz Thanks, that one works. Meanwhile, I setup a -CURRENT VM for testing https://reviews.freebsd.org/D4104 directly. I can build the boot1.efi loader just fine, and install it into an EFI partition, but the resulting VM does not boot. The console looks like this: >> FreeBSD EFI boot block Loader path: /boot/loader.efi Initializing modules: ZFS UFS Could not load file Could not load file panic: No bootable partitions found! Any clue? (It would be very nice if it printed *which* file it could not load, btw.) The VM was setup with two mirrored disks, using the Auto ZFS option during installation, like this: $ gpart show =3D> 40 209715120 da0 GPT (100G) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 8388608 2 freebsd-swap (4.0G) 8390656 201322496 3 freebsd-zfs (96G) 209713152 2008 - free - (1.0M) =3D> 40 209715120 da1 GPT (100G) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 8388608 2 freebsd-swap (4.0G) 8390656 201322496 3 freebsd-zfs (96G) 209713152 2008 - free - (1.0M) After rebuilding and reinstalling world with the D4104 patch, I deleted the freebsd-boot partitions, created EFI partitions in their place, created msdosfs filesystems on them, and copied the boot1.efi loader to those filesystems. -Dimitry --Apple-Mail=_E9525338-209A-48A7-A72A-F8F345DD7720 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.28 iEYEARECAAYFAlZqrawACgkQsF6jCi4glqOidQCgwfpv+xKKjnGRs1iWCv2drWv3 4xoAn3FH0VoiTgq/VcpiXP6E1YptRvC1 =PPUQ -----END PGP SIGNATURE----- --Apple-Mail=_E9525338-209A-48A7-A72A-F8F345DD7720-- From owner-freebsd-hackers@freebsd.org Fri Dec 11 16:07:14 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A8B9B9D73B9 for ; Fri, 11 Dec 2015 16:07:14 +0000 (UTC) (envelope-from Jerrold.Heyman@emc.com) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailuogwprd51.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 699B71FBF for ; Fri, 11 Dec 2015 16:07:14 +0000 (UTC) (envelope-from Jerrold.Heyman@emc.com) Received: from maildlpprd52.lss.emc.com (maildlpprd52.lss.emc.com [10.106.48.156]) by mailuogwprd51.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id tBBFMnVs026711 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Fri, 11 Dec 2015 10:22:50 -0500 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd51.lss.emc.com tBBFMnVs026711 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=emc.com; s=jan2013; t=1449847370; bh=cCNczinNzj1JAiNElGWOJw1l614=; h=From:To:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:Content-Transfer-Encoding:MIME-Version; b=B2E5p/r+BB/cFYElyevIZVje3ojZfVFBVekp4Dabn8TsqyBGIL6YTMi7gA1MwE9mP uGVY6+ulYgrSQg7zycDvYTZdJMJ2Nzu13Hrf/izSy8JQ/qW7kT8hGCLqt8hdbAkVP2 Bn3ha5Q2BM5SQqfFlpJW3kpFQ37Z5Jp5k+zlOjdY= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd51.lss.emc.com tBBFMnVs026711 Received: from mailusrhubprd02.lss.emc.com (mailusrhubprd02.lss.emc.com [10.253.24.20]) by maildlpprd52.lss.emc.com (RSA Interceptor) for ; Fri, 11 Dec 2015 10:22:41 -0500 Received: from mxhub09.corp.emc.com (mxhub09.corp.emc.com [10.254.92.104]) by mailusrhubprd02.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id tBBFMkjx032175 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Fri, 11 Dec 2015 10:22:47 -0500 Received: from MXHUB222.corp.emc.com (10.253.68.92) by mxhub09.corp.emc.com (10.254.92.104) with Microsoft SMTP Server (TLS) id 8.3.327.1; Fri, 11 Dec 2015 10:22:46 -0500 Received: from MX204CL01.corp.emc.com ([fe80::b84b:d314:29a2:cd6e]) by MXHUB222.corp.emc.com ([10.253.68.92]) with mapi id 14.03.0266.001; Fri, 11 Dec 2015 10:22:46 -0500 From: "Heyman, Jerrold" To: "freebsd-hackers@freebsd.org" Subject: and rpc_createerr Thread-Topic: and rpc_createerr Thread-Index: AdEzg0yBm3kk7sToTF2+wjgksEN8IAAelUAAAApQJ5A= Date: Fri, 11 Dec 2015 15:22:42 +0000 Message-ID: <9CDA60925D09954CA4BAD0284E2DFC43025552@MX204CL01.corp.emc.com> References: <9CDA60925D09954CA4BAD0284E2DFC43024EDE@MX204CL01.corp.emc.com> <1449811260.30424.50.camel@michaeleichorn.com> In-Reply-To: <1449811260.30424.50.camel@michaeleichorn.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.226.32] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd02.lss.emc.com X-RSA-Classifications: Source Code, public X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 16:07:14 -0000 T3JpZ2luYWxseSBwb3N0ZWQgdG8gZnJlZWJzZC1xdWVzdGlvbnMsIHdoaWNoIHJlY29tbWVuZGVk IEkgY29tZSBoZXJlDQoNCkkndmUganVzdCBpbnN0YWxsZWQgRnJlZUJTRCAxMC4yIGluIG9yZGVy IHRvIGRldGVybWluZSB0aGUgcG9ydGFiaWxpdHkNCm9mIG15IGNvbXBhbmllcyBjb2RlLsKgwqBC dWlsdCBnY2M0LjYgb3V0IG9mIHRoZSBwb3J0cy9sYW5nIGFyZWEsIGJ1dA0Kc2VlIHRoZSBzYW1l IGlzc3VlIHVzaW5nIC91c3IvYmluL2NjIChjbGFuZyAzLjQuMSkuDQoNCmluIC91c3IvaW5jbHVk ZS9ycGMvY2xudC5oIHRoZSBmb2xsb3dpbmcgc25pcHBldDoNCg0KLyoNCiogSWYgYSBjcmVhdGlv biBmYWlscywgdGhlIGZvbGxvd2luZyBhbGxvd3MgdGhlIHVzZXIgdG8gZmlndXJlIG91dCB3aHku DQoqLw0Kc3RydWN0IHJwY19jcmVhdGVlcnIgew0KwqDCoMKgwqDCoMKgwqDCoGVudW0gY2xudF9z dGF0IGNmX3N0YXQ7DQrCoMKgwqDCoMKgwqDCoMKgc3RydWN0IHJwY19lcnIgY2ZfZXJyb3I7wqDC oC8qIHVzZXJmdWwgd2hlbiBjZl9zdGF0ID09IFJQQ19QTUFQRkFJTFVSRSAqLw0KfTsNCg0KX19C RUdJTl9ERUNMUw0KZXh0ZXJuIHN0cnVjdCBycGNfY3JlYXRlZXJywqDCoMKgwqAqX19ycGNfY3Jl YXRlZWVyKHZvaWQpOw0KX19FTkRfREVDTFMNCiNkZWZpbmUgcnBjX2NyZWF0ZWVycsKgwqDCoMKg wqDCoMKgwqDCoMKgKCooX19ycGNfY3JlYXRlZWVycigpKSkNCg0KTm90ZSB0aGF0IHRoZSAjZGVm aW5lIGJlY29tZXMgYWN0aXZlIG9uY2UgdGhlIGZpbGUgaXMgaW5jbHVkZWQsIGFuZCBpbg0KbXkg c291cmNlIGNvZGUgSSBoYXZlIG11bHRpcGxlDQoNCsKgwqDCoHN0cnVjdCBycGNfY3JlYXRlZXJy ICpjZTsNCg0KZGVjbGFyYXRpb25zLsKgwqBCb3RoIGNjIGFuZCBnY2MgY2l0ZSB0aGlzIGFzIGFu IGVycm9yLCB0aG91Z2ggZm9yIGRpZmZlcmVudCByZWFzb25zLg0KDQpnY2MgY29tcGxhaW5zIHRo YXQgYSAnKCcgaXMgZm91bmQgd2hlcmUgYSAneycgaXMgZXhwZWN0ZWQuDQpUaGUgY2MgZXJyb3Ig bWVzc2FnZSBpcyAnZXJyb3I6IGRlY2xhcmF0aW9uIG9mIGFueW9ueW1vdXMgc3RydWN0IG11c3Qg YmUgYSBkZWZpbml0aW9uJy4NCg0KTXkgb3RoZXIgcG9ydHMgLSBMaW51eCwgQUlYLCBTb2xhcmlz LCBNYWMgT1NYLCBkbyBub3QgaGF2ZSB0aGUgI2RlZmluZSBpbiAvdXNyL2luY2x1ZGUvcnBjL2Ns bnQuaC4NClRoZSBIUC1VWCBkb2VzLCBidXQgaXQgaXMgZW5jYXBzdWxhdGVkIHdpdGhpbiBhICNp ZmRlZiBfUkVFTlRSQU5UIC8gI2VuZGlmIGJsb2NrLg0KDQpJcyB0aGlzIGFuIGFjdHVhbCBlcnJv ciwgb3IgaXMgdGhlcmUgc29tZXRoaW5nIG9uIEZyZWVCU0QgdGhhdCBJIG5lZWQNCnRvIGRvIHRo YXQgaXMgZGlmZmVyZW50IHRoYW4gdGhlIG90aGVyIHBsYXRmb3Jtcz8NCg0KVGhhbmtzIGluIGFk dmFuY2UsDQoNCkplcnJ5DQpKZXJyeSBIZXltYW4gICAgICAgICAgICAgICAgICAgICAgICAgICAg fCANClByaW5jaXBhbCBTb2Z0d2FyZSBFbmdpbmVlciAgICAgICAgICAgICB8ICAgU29mdHdhcmUg aXMgdGhlIGRpZmZlcmVuY2UgYmV0d2Vlbg0KRU1DIERhdGEgRG9tYWluICAgICAgICAgICAgICAg ICAgICAgICAgIHwgICAgaGFyZHdhcmUgYW5kIHJlYWxpdHkNCkplcnJvbGQuSGV5bWFuQGVtYy5j b20gLyA5MTkuNTk3Ljc4MTIgICB8DQoNCg0K From owner-freebsd-hackers@freebsd.org Fri Dec 11 16:17:53 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE33B9D7C2F for ; Fri, 11 Dec 2015 16:17:53 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 976811642 for ; Fri, 11 Dec 2015 16:17:53 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.15.2/8.15.2) with ESMTPS id tBBG3s7E001125 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Fri, 11 Dec 2015 08:03:54 -0800 (PST) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.15.2/8.15.2/Submit) id tBBG3sVg001107 for freebsd-hackers@freebsd.org; Fri, 11 Dec 2015 08:03:54 -0800 (PST) (envelope-from sgk) Date: Fri, 11 Dec 2015 08:03:54 -0800 From: Steve Kargl To: freebsd-hackers@freebsd.org Subject: sonewconn issue? Message-ID: <20151211160354.GA1005@troutmask.apl.washington.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.24 (2015-08-30) X-Mailman-Approved-At: Fri, 11 Dec 2015 17:12:10 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 16:17:53 -0000 Last night my system became wedged. Inspection of/var/log/messages revealed 290 messages of the form Dec 10 19:27:49 troutmask kernel: sonewconn: pcb 0xfffff8000a76c4b0: Listen queue overflow: 16 already in queue awaiting acceptance (1 occurrences) Dec 10 19:28:55 troutmask kernel: sonewconn: pcb 0xfffff8000a76c4b0: Listen queue overflow: 16 already in queue awaiting acceptance (9 occurrences) Dec 10 19:37:40 troutmask kernel: sonewconn: pcb 0xfffff8000a76c4b0: Listen queue overflow: 16 already in queue awaiting acceptance (27 occurrences) ... Dec 11 07:40:46 troutmask kernel: sonewconn: pcb 0xfffff8000a76c4b0: Listen queue overflow: 16 already in queue awaiting acceptance (1 occurrences) During this time, I could connect to the webserver on the system. ssh into the box would connect, but I never got an actaul session. This morning I found the console unresponse. Any suggestions for identifying the rogue process? -- Steve From owner-freebsd-hackers@freebsd.org Fri Dec 11 17:24:04 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6869B9D8FF7 for ; Fri, 11 Dec 2015 17:24:04 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-pa0-x233.google.com (mail-pa0-x233.google.com [IPv6:2607:f8b0:400e:c03::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2FBE413DD for ; Fri, 11 Dec 2015 17:24:04 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: by pacwq6 with SMTP id wq6so68224315pac.1 for ; Fri, 11 Dec 2015 09:24:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-type:content-transfer-encoding; bh=tiNwjyNwXGvCJBkafqYYy2yKtYDCF1QZylZKvAedyuo=; b=nklTg3bSFAcNlbgVmvDWtTg9WHDEHAVInR8Wufdk9Z2K2Je9enuIQj75gD5qtg5x3Z XB3r1Oqf9/vNF8jDUqTFIfogrqG492sl8UT1jvvljeLC51ppawUQAk2yrhpw8MgihhSx onz6kILs64utuAqQV6ptxvpvyeG7lja4rjvXsBarXzZ8lSfxVlsnL8csgjfg3uEYsWwJ eDDbW+io20LHZEDRVTUQOTar63V1ryp7kDoSjwn2tIqD+s+EGwors8VeIxUICJh8gzzW cVWrCZBZYtOzlk4K+gvmQqD4CWJBKt4OlFQAyT8PSuHvlqDYyo5XGD/NCGqvfUlYoE5S C22w== X-Received: by 10.67.2.73 with SMTP id bm9mr26679186pad.94.1449854643384; Fri, 11 Dec 2015 09:24:03 -0800 (PST) Received: from [10.192.166.0] (stargate.chelsio.com. [12.32.117.8]) by smtp.googlemail.com with ESMTPSA id i70sm2338198pfj.32.2015.12.11.09.24.01 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 11 Dec 2015 09:24:02 -0800 (PST) Sender: Navdeep Parhar Subject: Re: sonewconn issue? To: Steve Kargl , freebsd-hackers@freebsd.org References: <20151211160354.GA1005@troutmask.apl.washington.edu> From: Navdeep Parhar Message-ID: <566B06B0.5060809@FreeBSD.org> Date: Fri, 11 Dec 2015 09:24:00 -0800 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <20151211160354.GA1005@troutmask.apl.washington.edu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 17:24:04 -0000 On 12/11/2015 08:03, Steve Kargl wrote: > Last night my system became wedged. Inspection of/var/log/messages > revealed 290 messages of the form > > Dec 10 19:27:49 troutmask kernel: sonewconn: pcb 0xfffff8000a76c4b0: > Listen queue overflow: 16 already in queue awaiting > acceptance (1 occurrences) > Dec 10 19:28:55 troutmask kernel: sonewconn: pcb 0xfffff8000a76c4b0: > Listen queue overflow: 16 already in queue awaiting > acceptance (9 occurrences) > Dec 10 19:37:40 troutmask kernel: sonewconn: pcb 0xfffff8000a76c4b0: > Listen queue overflow: 16 already in queue awaiting > acceptance (27 occurrences) > > ... > > Dec 11 07:40:46 troutmask kernel: sonewconn: pcb 0xfffff8000a76c4b0: > Listen queue overflow: 16 already in queue awaiting > acceptance (1 occurrences) > > During this time, I could connect to the webserver on the system. > ssh into the box would connect, but I never got an actaul session. > This morning I found the console unresponse. > > Any suggestions for identifying the rogue process? > Try "netstat -aLnp tcp" and look at the "Listen" column to see which sockets are backlogged. Then "sockstat -4l" to identify the process that owns the socket. Regards, Navdeep From owner-freebsd-hackers@freebsd.org Fri Dec 11 17:55:38 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AAB529D7B0A for ; Fri, 11 Dec 2015 17:55:38 +0000 (UTC) (envelope-from prt@prt.org) Received: from smtp3.mail.clearhost.co.uk (smtp3.mail.clearhost.co.uk [IPv6:2001:1420::25:103]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.mail.clearhost.co.uk", Issuer "RapidSSL CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7D22713B0 for ; Fri, 11 Dec 2015 17:55:38 +0000 (UTC) (envelope-from prt@prt.org) Received: from [2001:1420:a:105:c62c:3ff:fe2f:bf] (port=62218 helo=parsnip.heronsbrook.org.uk) by smtp3.mail.clearhost.co.uk with esmtpa (Exim 4.76 (FreeBSD)) (envelope-from ) id 1a7Ruq-00004g-18 for freebsd-hackers@freebsd.org; Fri, 11 Dec 2015 17:55:36 +0000 To: freebsd-hackers@freebsd.org From: Paul Thornton Subject: Troubleshooting EFI boot problem Message-ID: <566B0E47.4010407@prt.org> Date: Fri, 11 Dec 2015 17:56:23 +0000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 17:55:38 -0000 Hi Folks, I've been trying to install the stock UEFI memstick of 10.2 RELEASE on an Acer V5-122P laptop. Its UEFI BIOS seems to work ok (it installs and boots Windows 10 fine). When I choose the USB drive from the boot menu to start the install, I get no further than the loader. All I see on screen is: >> FreeBSD EFI boot block Loader path: /boot/loader.efi panic: Load failed Putting the memstick into another machine, everything looks like it is in the correct place: # gpart show /dev/da8 => 3 15663099 da8 GPT (7.5G) 3 1600 1 efi (800K) 1603 32 2 freebsd-boot (16K) 1635 1511040 3 freebsd-ufs (738M) 1512675 2048 4 freebsd-swap (1.0M) 1514723 14148379 - free - (6.7G) # mount /dev/da8p3 /mnt # ls -al /mnt/boot/loader.efi -r-xr-xr-x 1 root wheel 337363 Aug 12 15:44 /mnt/boot/loader.efi # umount /mnt # mount -t msdosfs /dev/da8p1 /mnt # ls -al /mnt/efi/boot/ total 65 drwxr-xr-x 1 root wheel 512 Apr 26 2014 . drwxr-xr-x 1 root wheel 512 Apr 26 2014 .. -rwxr-xr-x 1 root wheel 65536 Apr 26 2014 BOOTx64.efi # umount /mnt How can I get some more useful information out to start debugging this? Paul. From owner-freebsd-hackers@freebsd.org Fri Dec 11 19:30:36 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5EDF0A041C3 for ; Fri, 11 Dec 2015 19:30:36 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 43CFD13C2; Fri, 11 Dec 2015 19:30:36 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.15.2/8.15.2) with ESMTPS id tBBJUZPg003156 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 11 Dec 2015 11:30:35 -0800 (PST) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.15.2/8.15.2/Submit) id tBBJUYBu003155; Fri, 11 Dec 2015 11:30:34 -0800 (PST) (envelope-from sgk) Date: Fri, 11 Dec 2015 11:30:34 -0800 From: Steve Kargl To: Navdeep Parhar Cc: freebsd-hackers@freebsd.org Subject: Re: sonewconn issue? Message-ID: <20151211193034.GA3100@troutmask.apl.washington.edu> References: <20151211160354.GA1005@troutmask.apl.washington.edu> <566B06B0.5060809@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <566B06B0.5060809@FreeBSD.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-Mailman-Approved-At: Fri, 11 Dec 2015 19:58:59 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 19:30:36 -0000 On Fri, Dec 11, 2015 at 09:24:00AM -0800, Navdeep Parhar wrote: > On 12/11/2015 08:03, Steve Kargl wrote: > > Last night my system became wedged. Inspection of/var/log/messages > > revealed 290 messages of the form > > > > Dec 10 19:27:49 troutmask kernel: sonewconn: pcb 0xfffff8000a76c4b0: > > Listen queue overflow: 16 already in queue awaiting > > acceptance (1 occurrences) > > > > During this time, I could connect to the webserver on the system. > > ssh into the box would connect, but I never got an actaul session. > > This morning I found the console unresponse. > > > > Any suggestions for identifying the rogue process? > > > > Try "netstat -aLnp tcp" and look at the "Listen" column to see which > sockets are backlogged. Then "sockstat -4l" to identify the process > that owns the socket. > Thanks for the suggestions. I'll keep this in mind if the problem re-appears. Unfortunately, I had no access to the box. ssh didn't work. console was unresponse. No serial console. I forgot to mention that this is FreeBSD-current circa Nov 9th, 2015. The box gets updated about once a month or so. -- Steve From owner-freebsd-hackers@freebsd.org Fri Dec 11 20:59:09 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8834AA04F6F for ; Fri, 11 Dec 2015 20:59:09 +0000 (UTC) (envelope-from lidl@pix.net) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.pix.net", Issuer "Pix.Com Technologies, LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4107C1E43 for ; Fri, 11 Dec 2015 20:59:09 +0000 (UTC) (envelope-from lidl@pix.net) Received: from torb.pix.net (torb.pix.net [192.168.16.32]) (authenticated bits=0) by hydra.pix.net (8.15.2/8.15.2) with ESMTPA id tBBKx76Q046540; Fri, 11 Dec 2015 15:59:07 -0500 (EST) (envelope-from lidl@pix.net) Subject: Re: and rpc_createerr To: freebsd-hackers@freebsd.org References: <9CDA60925D09954CA4BAD0284E2DFC43024EDE@MX204CL01.corp.emc.com> <1449811260.30424.50.camel@michaeleichorn.com> <9CDA60925D09954CA4BAD0284E2DFC43025552@MX204CL01.corp.emc.com> From: Kurt Lidl Message-ID: <566B391B.7090100@pix.net> Date: Fri, 11 Dec 2015 15:59:07 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <9CDA60925D09954CA4BAD0284E2DFC43025552@MX204CL01.corp.emc.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 20:59:09 -0000 On 12/11/15 10:22 AM, Heyman, Jerrold wrote: > Originally posted to freebsd-questions, which recommended I come here > > I've just installed FreeBSD 10.2 in order to determine the portability > of my companies code. Built gcc4.6 out of the ports/lang area, but > see the same issue using /usr/bin/cc (clang 3.4.1). > > in /usr/include/rpc/clnt.h the following snippet: > > /* > * If a creation fails, the following allows the user to figure out why. > */ > struct rpc_createerr { > enum clnt_stat cf_stat; > struct rpc_err cf_error; /* userful when cf_stat == RPC_PMAPFAILURE */ > }; > > __BEGIN_DECLS > extern struct rpc_createerr *__rpc_createeer(void); > __END_DECLS > #define rpc_createerr (*(__rpc_createeerr())) > > Note that the #define becomes active once the file is included, and in > my source code I have multiple > > struct rpc_createerr *ce; > > declarations. Both cc and gcc cite this as an error, though for different reasons. > > gcc complains that a '(' is found where a '{' is expected. > The cc error message is 'error: declaration of anyonymous struct must be a definition'. > > My other ports - Linux, AIX, Solaris, Mac OSX, do not have the #define in /usr/include/rpc/clnt.h. > The HP-UX does, but it is encapsulated within a #ifdef _REENTRANT / #endif block. > > Is this an actual error, or is there something on FreeBSD that I need > to do that is different than the other platforms? Well, the rpc_clnt_client(3) manpage says: > struct rpc_createerr rpc_createerr; > A global variable whose value is set by any RPC client handle > creation routine that fails. It is used by the routine > clnt_pcreateerror() to print the reason for the failure. As a global variable, I'm not sure how you're going to have multiple different ones... The following code compiles with no warnings -- note that the rpc_createerr structure isn't allocated (explictly) in this code. #include int main(int argc, char *argv[]) { CLIENT *client = NULL; client = clnt_create("localhost", 1, 1, "udp"); if (client == NULL) return ((int)rpc_createerr.cf_stat); return 0; } -Kurt From owner-freebsd-hackers@freebsd.org Fri Dec 11 21:00:30 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CB4FC9D7136 for ; Fri, 11 Dec 2015 21:00:30 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-pa0-x235.google.com (mail-pa0-x235.google.com [IPv6:2607:f8b0:400e:c03::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A36771F9E for ; Fri, 11 Dec 2015 21:00:30 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: by pabur14 with SMTP id ur14so71034921pab.0 for ; Fri, 11 Dec 2015 13:00:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-type:content-disposition :in-reply-to:user-agent; bh=1TBH1T3s1OcK9v4a8KnCEIdeY3YEfFP8sa/10feNzXQ=; b=XoQsoxoTRYJeJ4zKkYlSrcYZmXGMIiBf9fgirRgOEvsYp8KiAR8C+ObiTamtyBfJko NukjGmQZulgAMIpOzau5f02ayyjAxlt+afuMFC1c3o8DED3dYNFeroR848r9yrVqbsHX 1v8ywTJ5hFiiJCtZCcKAA4z1u6mnr1VSiW8DMaNdHmkaRkqUaHtCMd+HzwCAGupSItEE XuLEzeLmXWLJP7jHDXE+Sa0kd5omr03fCsBabViD8MQOm4Z3jT9D2GEw9/PTkRwuZ3NZ jg7tVaD72fi/LE3NsCuOQY9ZyuDBRfbi4KZ2s+ifkTsjnFpwAcP5LAfFVviw/OJxS6nS Gtuw== X-Received: by 10.66.155.8 with SMTP id vs8mr16935318pab.18.1449867630271; Fri, 11 Dec 2015 13:00:30 -0800 (PST) Received: from nparhar-pc (nat-198-95-226-228.netapp.com. [198.95.226.228]) by smtp.gmail.com with ESMTPSA id c20sm26994455pfd.17.2015.12.11.13.00.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 11 Dec 2015 13:00:28 -0800 (PST) Sender: Navdeep Parhar Date: Fri, 11 Dec 2015 13:00:24 -0800 From: Navdeep Parhar To: Steve Kargl Cc: freebsd-hackers@freebsd.org Subject: Re: sonewconn issue? Message-ID: <20151211210024.GA2613@nparhar-pc> Mail-Followup-To: Steve Kargl , freebsd-hackers@freebsd.org References: <20151211160354.GA1005@troutmask.apl.washington.edu> <566B06B0.5060809@FreeBSD.org> <20151211193034.GA3100@troutmask.apl.washington.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151211193034.GA3100@troutmask.apl.washington.edu> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 21:00:30 -0000 On Fri, Dec 11, 2015 at 11:30:34AM -0800, Steve Kargl wrote: > On Fri, Dec 11, 2015 at 09:24:00AM -0800, Navdeep Parhar wrote: > > On 12/11/2015 08:03, Steve Kargl wrote: > > > Last night my system became wedged. Inspection of/var/log/messages > > > revealed 290 messages of the form > > > > > > Dec 10 19:27:49 troutmask kernel: sonewconn: pcb 0xfffff8000a76c4b0: > > > Listen queue overflow: 16 already in queue awaiting > > > acceptance (1 occurrences) > > > > > > During this time, I could connect to the webserver on the system. > > > ssh into the box would connect, but I never got an actaul session. > > > This morning I found the console unresponse. > > > > > > Any suggestions for identifying the rogue process? > > > > > > > Try "netstat -aLnp tcp" and look at the "Listen" column to see which > > sockets are backlogged. Then "sockstat -4l" to identify the process > > that owns the socket. > > > > Thanks for the suggestions. I'll keep this in mind if the > problem re-appears. Unfortunately, I had no access to the > box. ssh didn't work. console was unresponse. No serial > console. If you can get into ddb after this happens then you can look up the 4 tuple from the inpcb (its address is in the message displayed by sonewconn). The local port is probably the interesting part if it's a server. db> show inpcb db> show tcpcb Regards, Navdeep > > I forgot to mention that this is FreeBSD-current circa > Nov 9th, 2015. The box gets updated about once a month > or so. > > -- > Steve From owner-freebsd-hackers@freebsd.org Fri Dec 11 21:30:49 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 61B67A04F32 for ; Fri, 11 Dec 2015 21:30:49 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D512114F0 for ; Fri, 11 Dec 2015 21:30:48 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id tBBLUhQd093156 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 11 Dec 2015 23:30:43 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua tBBLUhQd093156 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id tBBLUgMW093143; Fri, 11 Dec 2015 23:30:42 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 11 Dec 2015 23:30:42 +0200 From: Konstantin Belousov To: Steve Kargl , freebsd-hackers@freebsd.org Subject: Re: sonewconn issue? Message-ID: <20151211213042.GK82577@kib.kiev.ua> References: <20151211160354.GA1005@troutmask.apl.washington.edu> <566B06B0.5060809@FreeBSD.org> <20151211193034.GA3100@troutmask.apl.washington.edu> <20151211210024.GA2613@nparhar-pc> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151211210024.GA2613@nparhar-pc> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 21:30:49 -0000 On Fri, Dec 11, 2015 at 01:00:24PM -0800, Navdeep Parhar wrote: > On Fri, Dec 11, 2015 at 11:30:34AM -0800, Steve Kargl wrote: > > On Fri, Dec 11, 2015 at 09:24:00AM -0800, Navdeep Parhar wrote: > > > On 12/11/2015 08:03, Steve Kargl wrote: > > > > Last night my system became wedged. Inspection of/var/log/messages > > > > revealed 290 messages of the form > > > > > > > > Dec 10 19:27:49 troutmask kernel: sonewconn: pcb 0xfffff8000a76c4b0: > > > > Listen queue overflow: 16 already in queue awaiting > > > > acceptance (1 occurrences) > > > > > > > > During this time, I could connect to the webserver on the system. > > > > ssh into the box would connect, but I never got an actaul session. > > > > This morning I found the console unresponse. > > > > > > > > Any suggestions for identifying the rogue process? > > > > > > > > > > Try "netstat -aLnp tcp" and look at the "Listen" column to see which > > > sockets are backlogged. Then "sockstat -4l" to identify the process > > > that owns the socket. > > > > > > > Thanks for the suggestions. I'll keep this in mind if the > > problem re-appears. Unfortunately, I had no access to the > > box. ssh didn't work. console was unresponse. No serial > > console. > > If you can get into ddb after this happens then you can look up the 4 > tuple from the inpcb (its address is in the message displayed by > sonewconn). The local port is probably the interesting part if it's > a server. > > db> show inpcb > db> show tcpcb > > Regards, > Navdeep > > > > > I forgot to mention that this is FreeBSD-current circa > > Nov 9th, 2015. The box gets updated about once a month > > or so. I believe that the network messages are the consequences of some deadlock which stopped the usermode. If you do get at the ddb prompt, start with the ps command. From owner-freebsd-hackers@freebsd.org Fri Dec 11 21:42:16 2015 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5BE5E9D7A53 for ; Fri, 11 Dec 2015 21:42:16 +0000 (UTC) (envelope-from Jerrold.Heyman@emc.com) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailuogwprd51.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D12B1E6B for ; Fri, 11 Dec 2015 21:42:15 +0000 (UTC) (envelope-from Jerrold.Heyman@emc.com) Received: from maildlpprd56.lss.emc.com (maildlpprd56.lss.emc.com [10.106.48.160]) by mailuogwprd51.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id tBBLgC3x000786 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Fri, 11 Dec 2015 16:42:13 -0500 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd51.lss.emc.com tBBLgC3x000786 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=emc.com; s=jan2013; t=1449870133; bh=GEw4EEMmTFJEZY7xPjzlbr4lqMU=; h=From:To:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:Content-Transfer-Encoding:MIME-Version; b=vCn9LtAZvjNlydjKjRrDwpTM95JlrER4xhq0kRcsTdXxu3D0Nh1BuhcN7yjEbGj6s bSQly19k/BsmMk8+nRvdQ8SwVqDaX0G1GgtvaiLGEz11tYHAssd7wvDTGOZqyGJywv Uy9gdcKZRpObqbuSpcjIs2qbn2ZUTLUiu4C03yUE= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd51.lss.emc.com tBBLgC3x000786 Received: from mailusrhubprd51.lss.emc.com (mailusrhubprd51.lss.emc.com [10.106.48.24]) by maildlpprd56.lss.emc.com (RSA Interceptor) for ; Fri, 11 Dec 2015 16:41:04 -0500 Received: from mxhub19.corp.emc.com (mxhub19.corp.emc.com [10.254.93.48]) by mailusrhubprd51.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id tBBLftrv030777 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Fri, 11 Dec 2015 16:41:55 -0500 Received: from MXHUB223.corp.emc.com (10.253.68.93) by mxhub19.corp.emc.com (10.254.93.48) with Microsoft SMTP Server (TLS) id 8.3.327.1; Fri, 11 Dec 2015 16:41:55 -0500 Received: from MX204CL01.corp.emc.com ([fe80::b84b:d314:29a2:cd6e]) by MXHUB223.corp.emc.com ([10.253.68.93]) with mapi id 14.03.0266.001; Fri, 11 Dec 2015 16:41:54 -0500 From: "Heyman, Jerrold" To: "freebsd-hackers@freebsd.org" Subject: RE: and rpc_createerr Thread-Topic: and rpc_createerr Thread-Index: AdEzg0yBm3kk7sToTF2+wjgksEN8IAAelUAAAApQJ5AAFnNEgAAJWFIw Date: Fri, 11 Dec 2015 21:41:51 +0000 Message-ID: <9CDA60925D09954CA4BAD0284E2DFC4302585E@MX204CL01.corp.emc.com> References: <9CDA60925D09954CA4BAD0284E2DFC43024EDE@MX204CL01.corp.emc.com> <1449811260.30424.50.camel@michaeleichorn.com> <9CDA60925D09954CA4BAD0284E2DFC43025552@MX204CL01.corp.emc.com> <566B391B.7090100@pix.net> In-Reply-To: <566B391B.7090100@pix.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.226.32] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd51.lss.emc.com X-RSA-Classifications: Source Code, public X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2015 21:42:16 -0000 > -----Original Message----- > From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd-hackers@fre= ebsd.org] On Behalf Of Kurt > Lidl > Sent: Friday, December 11, 2015 3:59 PM > To: freebsd-hackers@freebsd.org > Subject: Re: and rpc_createerr >=20 > On 12/11/15 10:22 AM, Heyman, Jerrold wrote: > > Originally posted to freebsd-questions, which recommended I come here > > > > I've just installed FreeBSD 10.2 in order to determine the portability > > of my companies code. Built gcc4.6 out of the ports/lang area, but > > see the same issue using /usr/bin/cc (clang 3.4.1). > > > > in /usr/include/rpc/clnt.h the following snippet: > > > > /* > > * If a creation fails, the following allows the user to figure out why. > > */ > > struct rpc_createerr { > > enum clnt_stat cf_stat; > > struct rpc_err cf_error; /* userful when cf_stat =3D=3D > > RPC_PMAPFAILURE */ }; > > > > __BEGIN_DECLS > > extern struct rpc_createerr *__rpc_createeer(void); > > __END_DECLS > > #define rpc_createerr (*(__rpc_createeerr())) > > > > Note that the #define becomes active once the file is included, and in > > my source code I have multiple > > > > struct rpc_createerr *ce; > > > > declarations. Both cc and gcc cite this as an error, though for differ= ent reasons. > > > > gcc complains that a '(' is found where a '{' is expected. > > The cc error message is 'error: declaration of anyonymous struct must b= e a definition'. > > > > My other ports - Linux, AIX, Solaris, Mac OSX, do not have the #define = in /usr/include/rpc/clnt.h. > > The HP-UX does, but it is encapsulated within a #ifdef _REENTRANT / #en= dif block. > > > > Is this an actual error, or is there something on FreeBSD that I need > > to do that is different than the other platforms? >=20 > Well, the rpc_clnt_client(3) manpage says: >=20 > > struct rpc_createerr rpc_createerr; > > A global variable whose value is set by any RPC client ha= ndle > > creation routine that fails. It is used by the routine > > clnt_pcreateerror() to print the reason for the failure. >=20 > As a global variable, I'm not sure how you're going to have multiple diff= erent ones... >=20 > The following code compiles with no warnings -- note that the rpc_createe= rr structure isn't allocated (explictly) in > this code. >=20 > #include > int main(int argc, char *argv[]) > { > CLIENT *client =3D NULL; >=20 > client =3D clnt_create("localhost", 1, 1, "udp"); >=20 > if (client =3D=3D NULL) > return ((int)rpc_createerr.cf_stat); >=20 > return 0; > } Kurt, Compiling the above example code with -E gives the following for the return= ((int)rpc_createerr.cf_stat); line return ((int)(*(__rpc_createerr())).cf_stat); which is what is expected based on the #define. Adding a declaration, 'struct rpc_createerr *ce;' results in the generation= of struct (*(__rpc_createerr())) *ce; Using (-E) compile option. Again, correct textual substitution for the #de= fine, but when compiling with -c I get the following=20 tester.c:6:4 error: declaration of anonymous struct must be a definition struct rpc_createerr *ce; ^ tester.c:6:4: warning: declaration does not declare anything [-Wmissing-dec= larations] struct rpc_createerr *ce; ^^^^^ Jerry Jerry Heyman |=20 Principal Software Engineer | Software is the difference betw= een EMC Data Domain | hardware and reality Jerrold.Heyman@emc.com / 919.597.7812 |