From owner-freebsd-fs@freebsd.org Sun Sep 24 22:20:07 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5000CE02A26 for ; Sun, 24 Sep 2017 22:20:07 +0000 (UTC) (envelope-from bsd@vink.pl) Received: from mail-qk0-x236.google.com (mail-qk0-x236.google.com [IPv6:2607:f8b0:400d:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 09528644E7 for ; Sun, 24 Sep 2017 22:20:06 +0000 (UTC) (envelope-from bsd@vink.pl) Received: by mail-qk0-x236.google.com with SMTP id d70so5110156qkc.11 for ; Sun, 24 Sep 2017 15:20:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vink-pl.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=5SsrcxqeYYh2OgRQxcZVAKQVnAEb6YNjssU2xDneNxk=; b=OgZ8hTRhVci6zvX4c8goNzm9JwCopLHfcdDL7gU3ZvZkqNiG9/yUgmdYvgxcYB4/9i R7kPs0LoxXtQ59w9lM7QYTYg3yCCHgrHKJmKEpEozsv8fF7E4/JQTJfxg0HyizwwCwoS FP3Ym9H7uyHtf/MAorLqXJp0p1dyWsYFvYODSf/T3gfZ2uwI/aQ1VkHqx7bD31AWzKBs FdmOdO67g4Lohly0E8EVlwuupGzG09r+dbYESbwaA1xmeRIMzE7QKvQwsV+H57En4PxH q7JUty+M5IoMhNT3TriYMZCOsktFRreRHJtFwXOgCR1VX4/JxTp1g8YCJtIkEvPfWX03 ZjNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=5SsrcxqeYYh2OgRQxcZVAKQVnAEb6YNjssU2xDneNxk=; b=B2HNnzQ0CxA4TSq0Kh4SHL+OhLCa1MmhlfgU09dilPk9bsmL0iD8h787pfk5Z9RHs0 +vFbjWk+QgHfAHQDt3ObmDdvbOycibQCPudTUSmFuvN4vOgl7jlB1Erv3170DhxETDaV RAjbqs7i3jMhVcSpqUSGgci2yJAGAM9OI0Ia6id0Y/fj7IOYrLBOlRFlMP25aAK+GP1G necl/zZWdpkV0mKUQczPilso/RlclMphlrzC+WYEwGelECniUIb6zD2Died4lZnvasY0 gODTUOdouw3D2qKsqpC8rStVtufUjfGbErkWGxJ7UKdNFEV/w4VEV302N+abQ+zi20jP yyIQ== X-Gm-Message-State: AHPjjUjc5yoOA/dhGITyBXr6XtarYTgF+6tPgJ+Ef6NVTEj83c/cWZD9 JkyOCFEvsZxBXxPI0OZPDGk7BlDh X-Received: by 10.55.162.7 with SMTP id l7mr7478529qke.154.1506291605669; Sun, 24 Sep 2017 15:20:05 -0700 (PDT) Received: from mail-qk0-f176.google.com (mail-qk0-f176.google.com. [209.85.220.176]) by smtp.gmail.com with ESMTPSA id 4sm3909794qke.50.2017.09.24.15.20.05 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 24 Sep 2017 15:20:05 -0700 (PDT) Received: by mail-qk0-f176.google.com with SMTP id q8so5061443qkl.12 for ; Sun, 24 Sep 2017 15:20:05 -0700 (PDT) X-Google-Smtp-Source: AOwi7QAJ8RbI3V8AUNk4MWF+lcOCiWY5hhx23whTJydoKJSq3nAm8/KNgvLnxinLjfpKpVZz3s2tRuG0VYhUwT0epuA= X-Received: by 10.55.132.130 with SMTP id g124mr8384564qkd.268.1506291604942; Sun, 24 Sep 2017 15:20:04 -0700 (PDT) MIME-Version: 1.0 Received: by 10.12.139.1 with HTTP; Sun, 24 Sep 2017 15:20:04 -0700 (PDT) In-Reply-To: <062dfcf9-b56e-c6c4-4039-c48bf7bdd610@FreeBSD.org> References: <062dfcf9-b56e-c6c4-4039-c48bf7bdd610@FreeBSD.org> From: Wiktor Niesiobedzki Date: Mon, 25 Sep 2017 00:20:04 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: ZVOL with volblocksize=64k+ results in data corruption [was Re: Resolving errors with ZVOL-s] To: Andriy Gapon Cc: freebsd-fs Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Sep 2017 22:20:07 -0000 2017-09-22 10:46 GMT+02:00 Andriy Gapon : > On 21/09/2017 23:00, Wiktor Niesiobedzki wrote: > > And gave them more than 24h of work with no apparent errors (I also moved > > other volumes to 4k and they did not show any checksum errors for more > than > > 2 weeks). > > > > I was running with volblocksize=128k from January this year. The problem > > started to appear only after I updated from 11.0 to 11.1. > > > > Should I file bug report for this? What additional information should I > > gather? > > Are you able to patch your version of FreeBSD? > Could you please try https://svnweb.freebsd.org/changeset/base/323918 > and see if it makes things better? > > My tests are running for more than 48h and no errors till now. Usually after 6 hours I already saw failures so I it looks like you've nailed it. Thanks, Wiktor From owner-freebsd-fs@freebsd.org Mon Sep 25 14:08:30 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AF899E16C20 for ; Mon, 25 Sep 2017 14:08:30 +0000 (UTC) (envelope-from mlobo@digiart.art.br) Received: from burlywood.maple.relay.mailchannels.net (burlywood.maple.relay.mailchannels.net [23.83.214.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6636C7FA5A for ; Mon, 25 Sep 2017 14:08:28 +0000 (UTC) (envelope-from mlobo@digiart.art.br) X-Sender-Id: hostmach|x-authuser|mlobo@digiart.art.br Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 6658E10919B for ; Mon, 25 Sep 2017 12:52:38 +0000 (UTC) Received: from logan.hmnoc.net (unknown [100.96.127.236]) (Authenticated sender: hostmach) by relay.mailchannels.net (Postfix) with ESMTPA id 0E6211092E9 for ; Mon, 25 Sep 2017 12:52:37 +0000 (UTC) X-Sender-Id: hostmach|x-authuser|mlobo@digiart.art.br Received: from logan.hmnoc.net (logan.hmnoc.net [172.20.82.168]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.9.14); Mon, 25 Sep 2017 12:52:38 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: hostmach|x-authuser|mlobo@digiart.art.br X-MailChannels-Auth-Id: hostmach X-Spicy-Name: 3d456d802753165a_1506343958268_3698795689 X-MC-Loop-Signature: 1506343958268:2961897587 X-MC-Ingress-Time: 1506343958268 Received: from [187.114.217.233] (port=10057 helo=Papi.lobos) by logan.hmnoc.net with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1dwSsF-000A0o-Jp for freebsd-fs@freebsd.org; Mon, 25 Sep 2017 09:52:36 -0300 Date: Mon, 25 Sep 2017 09:54:24 -0300 From: Mario Lobo To: freebsd-fs@freebsd.org Subject: mount_smbfs question Message-ID: <20170925095424.42227ea0@Papi.lobos> Organization: DigiArt X-Mailer: Claws Mail 3.15.0 (GTK+ 2.24.29; amd64-portbld-freebsd11.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-AuthUser: mlobo@digiart.art.br X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Sep 2017 14:08:30 -0000 Hi; Since the wannacry scare, all sharing servers (be them BSD, Linux or Windoes) migrated to SMB v2 or above. mount_smbfs only works on SMB v1, which means that now I can no longer use it. Is there any alternative to mount SMB v2+ shares that can be used on fstab? Thanks, -- Mario Lobo http://www.mallavoodoo.com.br FreeBSD since 2.2.8 [not Pro-Audio.... YET!!] "UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things." From owner-freebsd-fs@freebsd.org Mon Sep 25 14:31:13 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 150B2E1749F for ; Mon, 25 Sep 2017 14:31:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 03CB78035A for ; Mon, 25 Sep 2017 14:31:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v8PEVBOx064015 for ; Mon, 25 Sep 2017 14:31:12 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 222288] g_bio leak after zfs ABD commit Date: Mon, 25 Sep 2017 14:31:11 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: dnelson_1901@yahoo.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: avg@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Sep 2017 14:31:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222288 Dan Nelson changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dnelson_1901@yahoo.com --- Comment #9 from Dan Nelson --- I also applied the patch to 11-STABLE and have been running for a few days = with it. No leaks here either; everything looks back to normal. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon Sep 25 19:11:36 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60BDDE21F29 for ; Mon, 25 Sep 2017 19:11:36 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-wr0-x22e.google.com (mail-wr0-x22e.google.com [IPv6:2a00:1450:400c:c0c::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EB0B3653A5 for ; Mon, 25 Sep 2017 19:11:35 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-wr0-x22e.google.com with SMTP id w12so9575623wrc.7 for ; Mon, 25 Sep 2017 12:11:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=hLcchXE7TwX5OgAKBgn2/+9mH8hpbnDIzQtL3yf1PO4=; b=G09HFXgWqjQqLLaNYUTExjCB17JdMte9vkLpMrN2qloSnjHcFbFCk7HkFiUdn4H/sM te9UAfqS6vVF1Q0FUXT3+DAj2eVx18OOP3kPnnFQogN5QdVx5rn9Jjt/x1N1N+2TWOp2 G3ihmfSolSzcsstSAo1gY3pbnbBWz06WbnGc4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=hLcchXE7TwX5OgAKBgn2/+9mH8hpbnDIzQtL3yf1PO4=; b=ty1QQlgH7GRJRfFYA7MemtgGXXleyRoAJX3Gqnu7K/pQAjnqBzBU3E2QjhEp1s0UOu lHnCJ77asa9SW6TLgfQ0akBtkw76s8X4uUdglmcgl/K54wFzlvqhpCmHZqPIWMnKbJu2 Cf9BzT1SZTPhFNMsmOXw8sErEBxfdRTTI4V+uIknpWOO4KnGj91l01FtujhhbLboPgaI Dg6CiqPm/azsv16Mr4PFKqLH71tVR/UNyL6OXHhLg/oUTp4ZbeVptc4KMfPLYNvOHnrZ L8+QNRJfNVx5Oh+ap71WIpDXVgS0dLr5Ux90heongHjyJEhHhiegk2ixk/CrMy98K/X5 uBLw== X-Gm-Message-State: AHPjjUgxqrArhN332VFdaJhGHGZmd1HHki4LfRO4DArH2dEqno25UvuE 8GZZQuVovDxCh3z50c5+K8rboUGP0KJRy/XeQpi3nQ== X-Google-Smtp-Source: AOwi7QBQoJ6HyHW5WCCf/gh1c2Z86TlAwCWyouIo95/foK2fUrwiZqlAl4XIo8KIq2WtPcqhFzdBA321IUKZezKb9Ds= X-Received: by 10.25.155.71 with SMTP id d68mr2139478lfe.181.1506366693544; Mon, 25 Sep 2017 12:11:33 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.83.213 with HTTP; Mon, 25 Sep 2017 12:11:32 -0700 (PDT) From: Matthew Ahrens Date: Mon, 25 Sep 2017 12:11:32 -0700 Message-ID: Subject: OpenZFS Developer Summit 2017 - talk announcement To: "admin@open-zfs.org" , developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Sep 2017 19:11:36 -0000 The fifth annual OpenZFS Developer Summit will be held in 4 weeks, in San Francisco, *October 24-25, 2017* (Tuesday-Wednesday). See http://www.open-zfs.org/wiki/OpenZFS_Developer_Summit for all of the details, including slides and videos from previous years' conferences. Register to attend now! We expect the conference to sell out; we wanted to keep the chance of collaboration so we haven't increased the size from last year: https://www.eventbrite.com/e/openzfs-developer-summit-2017-tickets-34912232427 Today we're announcing the speakers and talks for the conference: *Talks: Day 1* TitleSpeakerCompany State of the Union Matt Ahrens Delphix Keynote: ZFS Past & Future Mark Maybee Oracle MMP: Safe "zpool import" for Clusters Olaf Faaland LLNL ZIL Performance: How I Doubled Sync Write Speed Prakash Sruya Delphix iFlash: Dynamic Adaptive L2ARC Caching Shailendra Tripathi Tegile Faster Allocation with the Log Spacemap Serapheim Dimitropoulos Delphix DRAID Isaac Huang Intel Porting With OSX Jorgen Lundman independent Fast Clone Deletion Sara Hartse Delphix ZSTD Compression Allan Jude ScaleEngine *Talks: Day 2* TitleSpeakerCompany A proposal for 1,000x better dedup performance Matt Ahrens Delphix Improving resilver: results & operational impacts Saso Kiselkov Nexenta Storage Pool Checkpoint Serapheim Dimitropoulos Delphix New prefetcher for sequential scrub Tom Caputi Datto RAID-Z Expansion Matt Ahrens Delphix + FreeBSD Foundation From owner-freebsd-fs@freebsd.org Wed Sep 27 11:25:39 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B480FE317E3 for ; Wed, 27 Sep 2017 11:25:39 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu1176c.smtpx.saremail.com (cu1176c.smtpx.saremail.com [195.16.148.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7C0876E37E for ; Wed, 27 Sep 2017 11:25:38 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.41] (unknown [192.148.167.11]) by proxypop02.sare.net (Postfix) with ESMTPA id 3C1229DD0DD for ; Wed, 27 Sep 2017 13:17:25 +0200 (CEST) From: Borja Marcos Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 11.0 \(3445.1.6\)) Subject: Problem with the periodic scrub of ZFS pools Message-Id: Date: Wed, 27 Sep 2017 13:17:24 +0200 To: FreeBSD FS X-Mailer: Apple Mail (2.3445.1.6) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 11:25:39 -0000 Hi, There=E2=80=99s a problem with the periodic script for ZFS pool = scrubbing (/etc/periodic/daily/800.scrub-zfs). It can be configured with an interval between scrubs = (daily_scrub_zfs_default_threshold=3D=E2=80=9C17=E2=80=9D) but the feature depends on making searches in the pool history. In a very = active ZFS pool (Mine has a lot of zfs dataset replications and there is a lot of zfs snapshot = creation/deletion/replication the scrub entries will disappear (the history has a finite length). In my case, for example, I configured 17 days between scrubs, I actually = get a weekly one. A better mechanism should be used to store the timestamp of the last = automated scrub. Custom attributes are a good candidate, but zpool doesn=E2=80=99t have that option. What = about adding it as a custom attribute to the root dataset of the pool? Ie, for a pool called =E2=80=9Ctank=E2=80=9D we could use =E2=80=9Czfs = set freebsd_period:last_scrub=3Dtimestamp=E2=80=9D or something slmilar. Ideas? Thanks! Borja. From owner-freebsd-fs@freebsd.org Wed Sep 27 13:11:33 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8E9C1E037C5; Wed, 27 Sep 2017 13:11:33 +0000 (UTC) (envelope-from mlobo@digiart.art.br) Received: from burlywood.maple.relay.mailchannels.net (burlywood.maple.relay.mailchannels.net [23.83.214.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3764671F6E; Wed, 27 Sep 2017 13:11:31 +0000 (UTC) (envelope-from mlobo@digiart.art.br) X-Sender-Id: hostmach|x-authuser|mlobo@digiart.art.br Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 62050127EA9; Wed, 27 Sep 2017 13:04:49 +0000 (UTC) Received: from logan.hmnoc.net (unknown [100.96.147.85]) (Authenticated sender: hostmach) by relay.mailchannels.net (Postfix) with ESMTPA id C130E127D15; Wed, 27 Sep 2017 13:04:48 +0000 (UTC) X-Sender-Id: hostmach|x-authuser|mlobo@digiart.art.br Received: from logan.hmnoc.net (logan.hmnoc.net [100.96.147.1]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.9.14); Wed, 27 Sep 2017 13:04:49 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: hostmach|x-authuser|mlobo@digiart.art.br X-MailChannels-Auth-Id: hostmach X-Army-Obese: 4579a14b32d0c674_1506517489231_3162370249 X-MC-Loop-Signature: 1506517489231:1720677985 X-MC-Ingress-Time: 1506517489230 Received: from [187.113.116.71] (port=65170 helo=Papi.lobos) by logan.hmnoc.net with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1dxC18-002CZP-My; Wed, 27 Sep 2017 10:04:47 -0300 Date: Wed, 27 Sep 2017 10:06:35 -0300 From: Mario Lobo To: freebsd-fs@freebsd.org Cc: FreeBSD questions Subject: mount_smbfs question (re-post) Message-ID: <20170927100635.7b56f8fd@Papi.lobos> Organization: DigiArt X-Mailer: Claws Mail 3.15.0 (GTK+ 2.24.29; amd64-portbld-freebsd11.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-AuthUser: mlobo@digiart.art.br X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 13:11:33 -0000 Please forgive my repost. I've searching for an answer to this for quite a while and the only 2 most recent links I found on the subject were: https://forums.freebsd.org/threads/61374/ https://forums.freebsd.org/threads/61422/ and they both say basically this: "mount_smbfs only supports SMBv1 which may be disabled on your sharing machines due to the WannaCry outbreak. There are no definite plans to add support for SMBv2 or SMBv3 to mount_smbfs." and "I cannot turn on SMBv1 for the handful of systems because the OS hasn't updated to work with something that has been out for over 10 years. While NFS might be an easier route in the "now", it is the principle of the matter. Who knows what other common features in the future will just get left behind because they aren't updated for integration. I do get that BSD is community ran and funded. SMBv1 has been around since 1990 and SMBv2 came out 11 years ago in 2006. Linux has moved ahead and kept up with their security. It makes me sad that BSD hasn't made any movement to keep up with SMB security versions" Since my environment is tottaly surrounded with shares that no longer accept SMBv1 (Windows, Linux AND FreeBSD servers), so basically in the end, what I'm really looking for is a confirmation that I'll just have to dump all Freebsd samba clients because the OS can't deal with SMBv2 or above. Thanks, -- Mario Lobo http://www.mallavoodoo.com.br FreeBSD since 2.2.8 [not Pro-Audio.... YET!!] "UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things." From owner-freebsd-fs@freebsd.org Wed Sep 27 14:20:56 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 08FEEE0542E; Wed, 27 Sep 2017 14:20:56 +0000 (UTC) (envelope-from freebsd@erwanlegrand.com) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [IPv6:2001:4b98:c:538::196]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C87F774E61; Wed, 27 Sep 2017 14:20:55 +0000 (UTC) (envelope-from freebsd@erwanlegrand.com) X-Originating-IP: 74.125.83.51 Received: from mail-pg0-f51.google.com (mail-pg0-f51.google.com [74.125.83.51]) (Authenticated sender: moi@erwan.legrand.name) by relay4-d.mail.gandi.net (Postfix) with ESMTPSA id C912A17209C; Wed, 27 Sep 2017 16:20:52 +0200 (CEST) Received: by mail-pg0-f51.google.com with SMTP id d8so7853791pgt.4; Wed, 27 Sep 2017 07:20:52 -0700 (PDT) X-Gm-Message-State: AHPjjUgexL0a8oVRRNuHXmb4BfTwpE33hN5YT7DRfxdzn71p3poiOlsb rBSKLQBjGqgmkMT0qN+W5uqTov0bhWJ9VjtsGtk= X-Google-Smtp-Source: AOwi7QAGu/FsTlMliHL5lDaI58u3FycSwKXHld6lwVhUM3Mkc610Yd4VY8hq4vOX3qJdQe1zqbFMSq6CBZ0A6RdVdls= X-Received: by 10.99.109.134 with SMTP id i128mr1514997pgc.345.1506522051076; Wed, 27 Sep 2017 07:20:51 -0700 (PDT) MIME-Version: 1.0 Received: by 10.100.163.3 with HTTP; Wed, 27 Sep 2017 07:20:50 -0700 (PDT) In-Reply-To: <20170927100635.7b56f8fd@Papi.lobos> References: <20170927100635.7b56f8fd@Papi.lobos> From: Erwan Legrand Date: Wed, 27 Sep 2017 16:20:50 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: mount_smbfs question (re-post) To: Mario Lobo Cc: freebsd-fs@freebsd.org, FreeBSD questions Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 14:20:56 -0000 On Wed, Sep 27, 2017 at 3:06 PM, Mario Lobo wrote: > Since my environment is tottaly surrounded with shares that no longer > accept SMBv1 (Windows, Linux AND FreeBSD servers), so basically in the > end, what I'm really looking for is a confirmation that I'll just have > to dump all Freebsd samba clients because the OS can't deal with SMBv2 > or above. Perhaps have a look at implementations of SMB on top of FUSE? http://portsmon.freebsd.org/portoverview.py?category=sysutils&portname=fusefs-smbnetfs From owner-freebsd-fs@freebsd.org Thu Sep 28 14:16:30 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A7FCE32C70 for ; Thu, 28 Sep 2017 14:16:30 +0000 (UTC) (envelope-from lobo@bsd.com.br) Received: from mail-yw0-x22f.google.com (mail-yw0-x22f.google.com [IPv6:2607:f8b0:4002:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D4FEBC8D for ; Thu, 28 Sep 2017 14:16:29 +0000 (UTC) (envelope-from lobo@bsd.com.br) Received: by mail-yw0-x22f.google.com with SMTP id q80so1092277ywg.2 for ; Thu, 28 Sep 2017 07:16:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsd.com.br; s=capeta; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=c8fN+0Vb3h2j5vJQ2foRjJes8Lxhn5I+6390nlBeM8M=; b=QagexafBT/N3roBBFuU3qJGhodg3SOLnuKFaGBDC6FeQpcFxjWGWB6B08wLuHi3qks ZjSrt/oNWpi+cyhaCBaFdqKSvWpJe9eBhxYEHEviXc8F46Nmrn8cTwr7qa8z59xIw3Zo vPsm3UMifNHvMeSbuUQ3bMpV7pXbNycSQdG1A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=c8fN+0Vb3h2j5vJQ2foRjJes8Lxhn5I+6390nlBeM8M=; b=FiZ7tp/3j0j5/yhg8rmbugI7YzoyqUQHUvQlF7fOVHMcRYqTm/y9TiQFFxz/+pgNbx t2gTy2i7eXigfcfpz7bYx5Pn5C3DNjaMzbHrOWV+T3OBJ94dwtX2LHuziddCWL/FCMCA F+AdLmziwJQeJp0yJr2PPV5VOxq3GlT8r6V82rlUVuEkkkDY3I2JmS6IJS6YzbeWfdBF dQ1JrvM2WmxosKn5olEdaU/NHmBVPJ3qCXmFANGAGRM5dpBpyyrqeNFG2tut/XgfvkEZ nzkH9fRFGZ8rv2lp06m8W8XQRwk7U2Zlfpw6KDLVI0WgpE5ouY+qSjjjrh4cZOJfwkOj KgeQ== X-Gm-Message-State: AHPjjUhf/Y/1VCuxkElXoCknZtgobgb/jOoLrJaIzr7b81gKZ2Vzi24A jl61SjPHvq5k2BqVwGTsezsq0SlL8hohKwdPjG6v0Q== X-Google-Smtp-Source: AOwi7QCE6ka+PHNjlJYeydQ+AMGyw1+twMgw5o3P1zER1SmcoeNBS2xMl8ciYw/jjtP+UfkgOIjutf5kEmDVFcZnwIQ= X-Received: by 10.37.174.165 with SMTP id b37mr3528238ybj.375.1506608188245; Thu, 28 Sep 2017 07:16:28 -0700 (PDT) MIME-Version: 1.0 Received: by 10.37.170.193 with HTTP; Thu, 28 Sep 2017 07:16:27 -0700 (PDT) In-Reply-To: References: <20170927100635.7b56f8fd@Papi.lobos> From: Mario Lobo Date: Thu, 28 Sep 2017 11:16:27 -0300 Message-ID: Subject: Re: mount_smbfs question (re-post) To: Erwan Legrand Cc: freebsd-fs@freebsd.org, FreeBSD questions Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 14:16:30 -0000 2017-09-27 11:20 GMT-03:00 Erwan Legrand : > On Wed, Sep 27, 2017 at 3:06 PM, Mario Lobo wrote: > > Since my environment is tottaly surrounded with shares that no longer > > accept SMBv1 (Windows, Linux AND FreeBSD servers), so basically in the > > end, what I'm really looking for is a confirmation that I'll just have > > to dump all Freebsd samba clients because the OS can't deal with SMBv2 > > or above. > > Perhaps have a look at implementations of SMB on top of FUSE? > http://portsmon.freebsd.org/portoverview.py?category= > sysutils&portname=fusefs-smbnetfs > _______________________________________________ > freebsd-questions@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions- > unsubscribe@freebsd.org" > I did. Same problem. smbnetfs only works with SMBv1 -- Mario Lobo http://www.mallavoodoo.com.br FreeBSD since version 2.2.8 [not Pro-Audio.... YET!!] (99,7% winfoes FREE) From owner-freebsd-fs@freebsd.org Thu Sep 28 14:39:33 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A86DBE335EC for ; Thu, 28 Sep 2017 14:39:33 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x230.google.com (mail-wm0-x230.google.com [IPv6:2a00:1450:400c:c09::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3D2DE1AC3 for ; Thu, 28 Sep 2017 14:39:33 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x230.google.com with SMTP id m127so2748348wmm.1 for ; Thu, 28 Sep 2017 07:39:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:content-transfer-encoding:subject:message-id:date:to :mime-version; bh=YoGRGYUGsSnRz9A+M4wMjy4dFO9kXNrOi4U/uHb2Et0=; b=QyPKotWf8sAATjdxnqUuPVptBRlsB3lPn9OJg3tZincyZwyL6D8nbDqldjGfG3LG8G Hy8swasGQ74cYkZSgKRpDgJoLP0r07kC8O2JFq20zyd09RF/jJS5owX7etDfGJECi6Hd KEEE68VXlmrJjNtZ1lzzq1qOYg536ZytRH8vVHiJTQH/0rM+Zgszlo6rpYd5lYZTTWNs fAvif+tR0RAHjtqzOsVcNLptgBohp9hHda+qLan0bbg9H6rMRqd3vfdENIgZxenobqgx j6r9W+YP4Evu8d8kbWHi/M8e6EuVMN47FVNZOjoLCto4aHBUexI9s2q5bwHtXD1YCr2U DUog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:subject :message-id:date:to:mime-version; bh=YoGRGYUGsSnRz9A+M4wMjy4dFO9kXNrOi4U/uHb2Et0=; b=e5l4CaMTGfcEY+HhuutndQ7ObkZHNmlsRzUFeLbyxGpqkHSrQDeeNdUhxhDokI39ws 1wDx1k14ALSjl1gcu0cROocq7gK8bIXSQz331RCAZbBgTFYBxnYuJe94WlwrzmLe8MOu DoyRfrotVqy1KWEYp2eQFSS2DR7j5mr8j8S7izLx8AU4gHDawu08CT1j/ZS/rcNBelbP VNU5qU5LUfkIkMdjMbv7SwGwqAEXolk8ecxJz6wdtYSncumbxAN9joc/PrfFuJw2whwn yNIaebmX3zXGe4EKS8KBvv3K5dCdi3hOuSpQ1E6Ccd4jOWqqkf+zA+dFDoPi07GHsIQo 4YLw== X-Gm-Message-State: AHPjjUjlImkE++Q0dT3hKdIjN32RlFHIAHTcAF3jwHGVFjCiTgnRM8nb 0qJJv+G08HTfCT6I8A/hH9koHhXC X-Google-Smtp-Source: AOwi7QCj286H0Vsigk/5q6jZpXBc75OLr49bpwksMIR5RQ6tDH/IzA0x/Ar0OhEWbkml9Rah0gzFeg== X-Received: by 10.28.21.205 with SMTP id 196mr1386335wmv.77.1506609571338; Thu, 28 Sep 2017 07:39:31 -0700 (PDT) Received: from bens-mac.home (LFbn-MAR-1-445-220.w2-15.abo.wanadoo.fr. [2.15.38.220]) by smtp.gmail.com with ESMTPSA id f140sm1184220wmd.12.2017.09.28.07.39.30 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 28 Sep 2017 07:39:30 -0700 (PDT) From: Ben RUBSON Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: How to create holes in files ? Message-Id: <44E32501-4213-4A09-992C-92DB4EF33C0C@gmail.com> Date: Thu, 28 Sep 2017 16:39:28 +0200 To: Freebsd fs Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 14:39:33 -0000 Hi, I'm trying to make holes in files in C. Goal is to deallocate huge files on ZFS while (randomly) reading them. The only thing I found is on Linux, fallocate(2) with FALLOC_FL_PUNCH_HOLE. What about FreeBSD ? Thank you very much ! Ben From owner-freebsd-fs@freebsd.org Thu Sep 28 14:53:27 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 02620E3397B for ; Thu, 28 Sep 2017 14:53:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B7D9021B6 for ; Thu, 28 Sep 2017 14:53:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v8SErQ6d072411 for ; Thu, 28 Sep 2017 14:53:26 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 222227] ZFS: Confusing error message when attempting to create zpool on too small media Date: Thu, 28 Sep 2017 14:53:26 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: asomers@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: asomers@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to bug_status cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 14:53:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222227 Alan Somers changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-fs@FreeBSD.org |asomers@FreeBSD.org Status|New |In Progress CC| |asomers@FreeBSD.org --- Comment #2 from Alan Somers --- I can reproduce this bug when using GEOM devices but not file-backed vdevs.= =20 I've got a pretty good idea what the problem is. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Sep 28 15:12:13 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 34771E33D6E for ; Thu, 28 Sep 2017 15:12:13 +0000 (UTC) (envelope-from se@freebsd.org) Received: from mailout08.t-online.de (mailout08.t-online.de [194.25.134.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailout00.t-online.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ED5CF2BF0 for ; Thu, 28 Sep 2017 15:12:12 +0000 (UTC) (envelope-from se@freebsd.org) Received: from fwd16.aul.t-online.de (fwd16.aul.t-online.de [172.20.26.243]) by mailout08.t-online.de (Postfix) with SMTP id 576AA41CB3CF; Thu, 28 Sep 2017 17:12:10 +0200 (CEST) Received: from Stefans-MBP-2.fritz.box (rSDbNOZDYh4q+A66OTJsZpiuEsdgzx6W3UaXjo5PDmbjG0HXdc9FaslVsxcyeS3wMK@[84.154.105.197]) by fwd16.t-online.de with (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384 encrypted) esmtp id 1dxaTl-4Dm0bw0; Thu, 28 Sep 2017 17:11:57 +0200 Subject: Re: mount_smbfs question (re-post) To: Mario Lobo References: <20170927100635.7b56f8fd@Papi.lobos> From: Stefan Esser Cc: FreeBSD File-Systems Message-ID: <6a2b3bf8-b70b-a39e-be33-fa28967ebb88@freebsd.org> Date: Thu, 28 Sep 2017 17:11:56 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Language: de-DE Content-Transfer-Encoding: 7bit X-ID: rSDbNOZDYh4q+A66OTJsZpiuEsdgzx6W3UaXjo5PDmbjG0HXdc9FaslVsxcyeS3wMK X-TOI-MSGID: b728a6b3-ec0d-4bfe-bc62-095458ed3bde X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 15:12:13 -0000 Am 28.09.17 um 16:16 schrieb Mario Lobo: > 2017-09-27 11:20 GMT-03:00 Erwan Legrand : > >> On Wed, Sep 27, 2017 at 3:06 PM, Mario Lobo wrote: >>> Since my environment is tottaly surrounded with shares that no longer >>> accept SMBv1 (Windows, Linux AND FreeBSD servers), so basically in the >>> end, what I'm really looking for is a confirmation that I'll just have >>> to dump all Freebsd samba clients because the OS can't deal with SMBv2 >>> or above. >> >> Perhaps have a look at implementations of SMB on top of FUSE? >> http://portsmon.freebsd.org/portoverview.py?category= >> sysutils&portname=fusefs-smbnetfs >> _______________________________________________ >> freebsd-questions@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-questions >> To unsubscribe, send any mail to "freebsd-questions- >> unsubscribe@freebsd.org" >> > > I did. Same problem. > smbnetfs only works with SMBv1 No, I've just tested it, it can see my SMB3-only samba server. I do not have any windows server to test against, but you can rise the debug level in smbnetfs.conf and start smbnetfs with the -d option to see what's going wrong on your system. From owner-freebsd-fs@freebsd.org Thu Sep 28 15:36:34 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E807AE019B7 for ; Thu, 28 Sep 2017 15:36:34 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay02.ispgateway.de (smtprelay02.ispgateway.de [80.67.31.36]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8B0543D17 for ; Thu, 28 Sep 2017 15:36:34 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from [78.35.164.62] (helo=fabiankeil.de) by smtprelay02.ispgateway.de with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1dxajB-0002RF-NM for freebsd-fs@freebsd.org; Thu, 28 Sep 2017 17:27:53 +0200 Date: Thu, 28 Sep 2017 17:26:09 +0200 From: Fabian Keil To: Freebsd fs Subject: Re: How to create holes in files ? Message-ID: <20170928172609.0e6d5c77@fabiankeil.de> In-Reply-To: <44E32501-4213-4A09-992C-92DB4EF33C0C@gmail.com> References: <44E32501-4213-4A09-992C-92DB4EF33C0C@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/kFRu/vYwbbUeM7q2Hajwz1x"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 15:36:35 -0000 --Sig_/kFRu/vYwbbUeM7q2Hajwz1x Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Ben RUBSON wrote: > I'm trying to make holes in files in C. > Goal is to deallocate huge files on ZFS while (randomly) reading them. My interpretation of the above is that you want to create holes without changing the file size and without affecting data that is located before or after the holes that you want to create. Otherwise you could simply "deallocate" the content with truncate(1). > The only thing I found is on Linux, fallocate(2) with > FALLOC_FL_PUNCH_HOLE. >=20 > What about FreeBSD ? I remember some discussions about adding something similar for FreeBSD but I don't remember ever seeing any patches (or commits) for it. As a work around you can enable ZFS compression (if it isn't enabled already) and "overwrite" data you don't need anymore with zeros. As an example see the delete_range() function in: https://www.fabiankeil.de/sourcecode/electrobsd/ElectroBSD-r312620-6cfa243f= 1516/0033-ggate-cd-Add-BIO_DELETE-support.diff Obviously this only works with ZFS, but you didn't mention other file systems anyway. Fabian --Sig_/kFRu/vYwbbUeM7q2Hajwz1x Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iF0EARECAB0WIQTKUNd6H/m3+ByGULIFiohV/3dUnQUCWc0UkQAKCRAFiohV/3dU ncxfAKCScSzAx+nmuz4qndr/n0Cgvr5w5ACgjcdzkmgCW95l/67zI1nk1Mb2I4U= =ppiM -----END PGP SIGNATURE----- --Sig_/kFRu/vYwbbUeM7q2Hajwz1x-- From owner-freebsd-fs@freebsd.org Thu Sep 28 16:11:06 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC80AE02999 for ; Thu, 28 Sep 2017 16:11:06 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wr0-x229.google.com (mail-wr0-x229.google.com [IPv6:2a00:1450:400c:c0c::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4573B644DB for ; Thu, 28 Sep 2017 16:11:06 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wr0-x229.google.com with SMTP id l39so3581508wrl.12 for ; Thu, 28 Sep 2017 09:11:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=F/waumL4fK+5yJAu34SbdbMtWFQTRAXq9g1Zi4JuLcQ=; b=i9DSfb9+eO180dvU0ZF5YdzdfaMG7zRiaPgoHSN6L+iZIFcbqqiXqb5/tlVGAQRbt8 nlY5btnyLipiDcMDdF1o4RudVOXi8dTXkPhkO73TnYxLXtaanXKAPan4VLJjEyG0nguy 4CK0fb2ilDD9xISLuWUoqdasD6R2T82c1chBy6fabvA+/WhnKKv18kC7i2fDQlgQKZWf ETZ/SWa+TfztmNnZc9kppokPDYoI7JHtaXIKgQ1TziTv54u7XiRwv6ZkAiEkqLuFHMeJ 2xEZorh11sHCLZMr2KmkkCLeIhNN6xblofw/xCjZbfXYf1DCe2rnOE1aEzFdrUhhKlma ZJhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=F/waumL4fK+5yJAu34SbdbMtWFQTRAXq9g1Zi4JuLcQ=; b=f2wjwT/RH3Tclek99pn9NsGDN+GAEzMmCX5i7yIcMFBXGGdH7ltyTJpdcjZc48dpxq VdvJklj1PKCNC+sm/j5ujSK8PGGdT40qDgcR1BSrBcQDqmzsIRe3Lb2144ho/ERpxLGu jSFgh6uaimTT496puXpXtH+D5wd4inlji2cipdpSjS6OamkHqziM0+F/ZEk/aEFE5uKl WUfNHBItX1H/UgvBedgHshMFbaAYaeoRTyWb4/YdaszYpc9MQMOqHnLukGM8cwHtiCO3 ha4RIHGoKCI5L5AuJGTf1VhxQJFEMO2zsznhzD8m/Klht71NCIghj8qeTZBaHFtCaBdr LW+Q== X-Gm-Message-State: AHPjjUiyjOD+B5s8oTwJaQPyJO9617CAsvUOWtoQU4mV9JTkWX2Z4QgY qt3EcnT0bR+B2rCP8xrYUQr7Z0Dh X-Google-Smtp-Source: AOwi7QDpL4uOwO5ahJx3ZYv6bTlr3pamtQQNLio3ztj2Y5bhnKFvNmP3QSY81bPfU0OSDorUopy3sg== X-Received: by 10.223.174.150 with SMTP id y22mr5313812wrc.29.1506615064377; Thu, 28 Sep 2017 09:11:04 -0700 (PDT) Received: from bens-mac.home (LFbn-MAR-1-445-220.w2-15.abo.wanadoo.fr. [2.15.38.220]) by smtp.gmail.com with ESMTPSA id w4sm3063506wrc.78.2017.09.28.09.11.03 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 28 Sep 2017 09:11:03 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: How to create holes in files ? From: Ben RUBSON In-Reply-To: <20170928172609.0e6d5c77@fabiankeil.de> Date: Thu, 28 Sep 2017 18:11:02 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <15FEA4C2-7434-4CED-8369-AB4695B47654@gmail.com> References: <44E32501-4213-4A09-992C-92DB4EF33C0C@gmail.com> <20170928172609.0e6d5c77@fabiankeil.de> To: Freebsd fs X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 16:11:06 -0000 Hi Fabian, And thank you very much for your answer ! > On 28 Sep 2017, at 17:26, Fabian Keil = wrote: >=20 > Ben RUBSON wrote: >=20 >> I'm trying to make holes in files in C. >> Goal is to deallocate huge files on ZFS while (randomly) reading = them. >=20 > My interpretation of the above is that you want to create holes > without changing the file size and without affecting data that > is located before or after the holes that you want to create. Exactly. > Otherwise you could simply "deallocate" the content with > truncate(1). >=20 >> The only thing I found is on Linux, fallocate(2) with >> FALLOC_FL_PUNCH_HOLE. >>=20 >> What about FreeBSD ? >=20 > I remember some discussions about adding something similar > for FreeBSD but I don't remember ever seeing any patches > (or commits) for it. Would really be nice ! > As a work around you can enable ZFS compression (if it isn't > enabled already) and "overwrite" data you don't need anymore > with zeros. >=20 > As an example see the delete_range() function in: > = https://www.fabiankeil.de/sourcecode/electrobsd/ElectroBSD-r312620-6cfa243= f1516/0033-ggate-cd-Add-BIO_DELETE-support.diff >=20 > Obviously this only works with ZFS, but you didn't mention > other file systems anyway. You're right I only use ZFS. This is clearly a nice workaround ! Unfortunately, for some reasons, I can't enable ZFS compression. (I absolutely need userquota on logical size so can't enable = compression) Thank you again ! Ben From owner-freebsd-fs@freebsd.org Thu Sep 28 16:40:05 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1032E033AE for ; Thu, 28 Sep 2017 16:40:05 +0000 (UTC) (envelope-from rde@tavi.co.uk) Received: from kipling.tavi.co.uk (kipling.tavi.co.uk [81.187.145.130]) by mx1.freebsd.org (Postfix) with ESMTP id 74A2D653DE for ; Thu, 28 Sep 2017 16:40:04 +0000 (UTC) (envelope-from rde@tavi.co.uk) Received: from kipling.tavi.co.uk (localhost [127.0.0.1]) by kipling.tavi.co.uk (Postfix) with ESMTP id 5C82B892C4 for ; Thu, 28 Sep 2017 17:34:41 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=tavi.co.uk; h=date:from:to :subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=selector1; bh=2VEAG3I 6kb7Sloz1K0x8NAP65ZA=; b=U24fDyOzOPH3YoSKjcXrZybjru2vL64YhzTfNX2 f20g5MkVgHMSOIu3M7au614tDXeNNMjx51wiZy50enqnlgdBGUgQdShh3ABHYpXU n20cpS56fPgNYTY2I0kyc/blStHAZ9evzzOdlUtqAgh/417SlCWNkwqBJ26TuuMo yPds= Received: from raksha.tavi.co.uk (raksha.tavi.co.uk [81.187.145.139]) (Authenticated sender: rde@tavi.co.uk) by kipling.tavi.co.uk (Postfix) with ESMTPA id 28289892C3 for ; Thu, 28 Sep 2017 17:34:41 +0100 (BST) Date: Thu, 28 Sep 2017 17:34:28 +0100 From: Bob Eager To: freebsd-fs@freebsd.org Subject: Re: How to create holes in files ? Message-ID: <20170928173428.42d1619c@raksha.tavi.co.uk> In-Reply-To: <20170928172609.0e6d5c77@fabiankeil.de> References: <44E32501-4213-4A09-992C-92DB4EF33C0C@gmail.com> <20170928172609.0e6d5c77@fabiankeil.de> X-Mailer: Claws Mail 3.15.0 (GTK+ 2.24.31; i386-portbld-freebsd11.0) Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAMFBMVEUwXjFLc0vD0cS7y7zw9PDZ4tkWSRaVrZZ+m39qi2tXfVj////7+/utwK4IPggAOAAJUUA7AAABKklEQVQ4jWPYjQMwDFYJp0NKEKCNJmEf9h8CsimXiL2e33s3/e7F7K2Cs3f3dCMkQkMKj4YuCY3K3iR+e7fMaiSjvkX0/5cFGrWpe2uLzOpaExUVqMS/8PX/Re5ey960OLBTZpFA8+IlSBKPQ92zNyUUBsosN58uIY0k8f+/ONCoYytkVuhWzVwNkYiYbqk5M3NmOVBi41YZ8RsGF7shEtFb5KJ3r969CyixM7OTPeFUxG2IxLO8/9/SvqXlc+/x3h295YzLlj2nIRJQj//nRvc5TEIal8RsXBLVuCQwIgoq/u80DomP6HEOk/iOS+IJLonZOCT+ReOQ+Lkbh0QKLonbOCR+7MYhsRqHBJrVcIl/1TgklqKLQyQ+tGKIgyQOqXpjig94diZRAgAXmDX6jyWafAAAAABJRU5ErkJggg====== MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: base64 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 16:40:05 -0000 LS0tLS1CRUdJTiBQR1AgU0lHTkVEIE1FU1NBR0UtLS0tLQ0KSGFzaDogU0hBMjU2DQoNCk9uIFRo dSwgMjggU2VwIDIwMTcgMTc6MjY6MDkgKzAyMDANCkZhYmlhbiBLZWlsIDxmcmVlYnNkLWxpc3Rl bkBmYWJpYW5rZWlsLmRlPiB3cm90ZToNCg0KPiBCZW4gUlVCU09OIDxiZW4ucnVic29uQGdtYWls LmNvbT4gd3JvdGU6DQo+IA0KPiA+IEknbSB0cnlpbmcgdG8gbWFrZSBob2xlcyBpbiBmaWxlcyBp biBDLg0KPiA+IEdvYWwgaXMgdG8gZGVhbGxvY2F0ZSBodWdlIGZpbGVzIG9uIFpGUyB3aGlsZSAo cmFuZG9tbHkpIHJlYWRpbmcNCj4gPiB0aGVtLiAgDQo+IA0KPiBNeSBpbnRlcnByZXRhdGlvbiBv ZiB0aGUgYWJvdmUgaXMgdGhhdCB5b3Ugd2FudCB0byBjcmVhdGUgaG9sZXMNCj4gd2l0aG91dCBj aGFuZ2luZyB0aGUgZmlsZSBzaXplIGFuZCB3aXRob3V0IGFmZmVjdGluZyBkYXRhIHRoYXQNCj4g aXMgbG9jYXRlZCBiZWZvcmUgb3IgYWZ0ZXIgdGhlIGhvbGVzIHRoYXQgeW91IHdhbnQgdG8gY3Jl YXRlLg0KPiANCj4gT3RoZXJ3aXNlIHlvdSBjb3VsZCBzaW1wbHkgImRlYWxsb2NhdGUiIHRoZSBj b250ZW50IHdpdGgNCj4gdHJ1bmNhdGUoMSkuDQoNCklmIGhlIGRvZXNuJ3QgbWluZCBjb3B5aW5n IHRoZSBmaWxlcywgZGQoMSkgd2lsbCBkbyB0aGUgam9iLiBIb3dldmVyLCBJDQpleHBlY3QgdGhh dCBkb2Vzbid0IG1lZXQgaGlzIGNyaXRlcmlhLg0KLS0tLS1CRUdJTiBQR1AgU0lHTkFUVVJFLS0t LS0NCg0KaVFFekJBRUJDQUFkRmlFRVZnZEkyS2VWbGRQQWhVWWFLQmRmMmF6OGU2Z0ZBbG5OSkpR QUNna1FLQmRmMmF6OA0KZTZqd3RRZi9XZXh0OW1wZGlnaEtZZnNaVlo0bUhVZkR1OGxQcmIrQ1Rj MDcxa2JBU3pTMGJ2MzZuVnM2cnZ4Wg0KN0o1akZQK0VNQTh4S0E3V04ycTF0OElaYWlWVkVaSG9x cCtOUW9qcDEzYzBnN0R0YUtNSGNDclpkZXNuWXBRVg0Kejl1MFd4cFpjQ20vZjZIK1R2Tzh3bG5Z QVh2QW50QjBuSUE1UjZVUFR5TVhjR1NuamZEa2cwMlR6cStBS0owcw0KbmNYazR3WFNST05NQW5i Q3hIdVkwOXY5MnJmY2lKdjhWbHZha0d4RmMyMFJ6WHFEdXZLWXNhRm1rK2dPdURCTA0KUE1kZE95 V0tVVVdPY1RBcHVEc2lqN1lxN3dBN0UyNTZ1VitpSStuZFUrRjh0WHNMdTB0T21NU0J3aDRFRm9w YQ0KVzlQYVF2ZHpyNm45dkh1Mm14aFpuVmtOY1VWeVl3PT0NCj1kTnEwDQotLS0tLUVORCBQR1Ag U0lHTkFUVVJFLS0tLS0NCg== From owner-freebsd-fs@freebsd.org Thu Sep 28 16:47:08 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 78217E037F7 for ; Thu, 28 Sep 2017 16:47:08 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wr0-x234.google.com (mail-wr0-x234.google.com [IPv6:2a00:1450:400c:c0c::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DB3C965952 for ; Thu, 28 Sep 2017 16:47:07 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wr0-x234.google.com with SMTP id g29so3767255wrg.11 for ; Thu, 28 Sep 2017 09:47:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=wifcp7ujqB4dFGLxahJg6T2nZzpnUq9gPa0FdwLjp7E=; b=bP+vuOsGPceYGZSMwSl7zXycbX9qd07oxgZEGDofBAOXtiTWNdaHSBzuxtSslhZdFB M+B1cAO8nZ70AVgDanRrEOfbgHM00O55P7gL1PoSs5UZzmgUqUpvnclJEfwA8eyYnzKZ si3jrxl6knfDlePxWNmHykrauga2rM011tCdudbohKk0Iy/SByCXpJqBkg9P0sSSEfKh oggjs7R1H94bJFkRSxj4/8Go1+ByX0dHCFtK0DntwGA81MZwqNJKt7LYY9ags72rKIw2 CGotKB0kUP2jER5MomnfT+SqDIuBWvVxkQ5cDEkc7nvwBNWP08tgYHZFiYlxXT620OQ5 6Dkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=wifcp7ujqB4dFGLxahJg6T2nZzpnUq9gPa0FdwLjp7E=; b=k8pP9krpaxq71ALJtSp/whuo45pv7yp/powL0qoa7ZRevYcqeUrR+sOOz+mUjaooYi u5V7jEKkhBmc1AdVCTJVTNMi7EHMfEPfasJwccruqh/Od7aTPgyGKT4ox9N3eu6LxMuC +XlfL3rSkUE+8I8pqtVIMtH/mZuWnCwL/y46tC+bZcS3IPu25I9feWxC2vLfXT+qvyHx YUxzRtNgHawJaENYsVUojFtJN1U8SIHOiYvlZ24UELFNWBJ6TdKPDPY7p+/UpDmM50Em Z4d//ATo0ohVkI6ZYUU1AxQtzbZyfiKy9cuJrxfXbgcrHSatE+reiYcX6SIwO8BjMaMb IwRw== X-Gm-Message-State: AHPjjUjttcDjIIMS5/YQMchav1e9Omk+OSssx7mgyEYCdba9J/RlY4b8 75i6Acw3NXqiELiP9o+uk5asuBp/ X-Google-Smtp-Source: AOwi7QD0LA0SrqKjFZd3j4nIB9FCzvUhLuZZidkXpcSQS8pKkYGqKV3w5Z1TkmlXaYzWi4pLDdvaOA== X-Received: by 10.223.144.71 with SMTP id h65mr5361994wrh.41.1506617224715; Thu, 28 Sep 2017 09:47:04 -0700 (PDT) Received: from bens-mac.home (LFbn-MAR-1-445-220.w2-15.abo.wanadoo.fr. [2.15.38.220]) by smtp.gmail.com with ESMTPSA id y39sm3242638wry.70.2017.09.28.09.47.03 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 28 Sep 2017 09:47:03 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: How to create holes in files ? From: Ben RUBSON In-Reply-To: <20170928173428.42d1619c@raksha.tavi.co.uk> Date: Thu, 28 Sep 2017 18:47:03 +0200 Content-Transfer-Encoding: 7bit Message-Id: <6B96C3C6-44D0-4BE3-B5A1-3533BBAB6B92@gmail.com> References: <44E32501-4213-4A09-992C-92DB4EF33C0C@gmail.com> <20170928172609.0e6d5c77@fabiankeil.de> <20170928173428.42d1619c@raksha.tavi.co.uk> To: Freebsd fs X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 16:47:08 -0000 > On 28 Sep 2017, at 18:34, Bob Eager wrote: > > On Thu, 28 Sep 2017 17:26:09 +0200 > Fabian Keil wrote: > >> Ben RUBSON wrote: >> >>> I'm trying to make holes in files in C. >>> Goal is to deallocate huge files on ZFS while (randomly) reading >>> them. >> >> My interpretation of the above is that you want to create holes >> without changing the file size and without affecting data that >> is located before or after the holes that you want to create. >> >> Otherwise you could simply "deallocate" the content with >> truncate(1). > > If he doesn't mind copying the files, dd(1) will do the job. However, I > expect that doesn't meet his criteria. Thank you Bob for your suggestion. You're right goal is to avoid copying data : free space would not necessarily allow this, and as the files I'm working on are some hundreds of GB in size, it would really be a counterproductive long stressing storage operation. Ben From owner-freebsd-fs@freebsd.org Thu Sep 28 17:08:15 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0FF65E0402E; Thu, 28 Sep 2017 17:08:15 +0000 (UTC) (envelope-from freebsd@erwanlegrand.com) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [IPv6:2001:4b98:c:538::195]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CDD316651B; Thu, 28 Sep 2017 17:08:14 +0000 (UTC) (envelope-from freebsd@erwanlegrand.com) X-Originating-IP: 74.125.83.41 Received: from mail-pg0-f41.google.com (mail-pg0-f41.google.com [74.125.83.41]) (Authenticated sender: moi@erwan.legrand.name) by relay3-d.mail.gandi.net (Postfix) with ESMTPSA id A60D7A80DB; Thu, 28 Sep 2017 19:08:11 +0200 (CEST) Received: by mail-pg0-f41.google.com with SMTP id b11so1268630pgn.12; Thu, 28 Sep 2017 10:08:11 -0700 (PDT) X-Gm-Message-State: AHPjjUjvVnO0s+OJYw0A6kUMbTuvp/cjtdPU11PWNsT6AyVew+4bIW+b pTcmiYiZkU9h9jXE3H/5iHuKUBDshyhqZC1Vkvg= X-Google-Smtp-Source: AOwi7QCnYJDA9kDGiyHZ+Q9iwWzuHDtyehwaQxbHPGueureLjh8Qq4xJwTpA5SsqQ1rR/n05J4bhAFw8ukd25EyfebI= X-Received: by 10.159.244.9 with SMTP id x9mr4499848plr.34.1506618485129; Thu, 28 Sep 2017 10:08:05 -0700 (PDT) MIME-Version: 1.0 Received: by 10.100.163.3 with HTTP; Thu, 28 Sep 2017 10:08:04 -0700 (PDT) In-Reply-To: References: <20170927100635.7b56f8fd@Papi.lobos> From: Erwan Legrand Date: Thu, 28 Sep 2017 19:08:04 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: mount_smbfs question (re-post) To: Mario Lobo Cc: freebsd-fs@freebsd.org, FreeBSD questions Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 17:08:15 -0000 On Thu, Sep 28, 2017 at 4:16 PM, Mario Lobo wrote: > 2017-09-27 11:20 GMT-03:00 Erwan Legrand : >> On Wed, Sep 27, 2017 at 3:06 PM, Mario Lobo wrote: >> > Since my environment is tottaly surrounded with shares that no longer >> > accept SMBv1 (Windows, Linux AND FreeBSD servers), so basically in the >> > end, what I'm really looking for is a confirmation that I'll just have >> > to dump all Freebsd samba clients because the OS can't deal with SMBv2 >> > or above. >> >> Perhaps have a look at implementations of SMB on top of FUSE? >> >> http://portsmon.freebsd.org/portoverview.py?category=sysutils&portname=fusefs-smbnetfs > > I did. Same problem. > smbnetfs only works with SMBv1 > It is based on libsmbclient, thus it should support SMB2 if smb.conf allows it. According to the following thread, the client protocol is resticted to SMB1 by default: https://lists.samba.org/archive/samba-technical/2016-November/thread.html#116999 This might be fixed by setting "client max protocol = SMB2" in smb.conf. ($HOME/.smb/smb.conf in this case?) From owner-freebsd-fs@freebsd.org Thu Sep 28 18:48:23 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 03FB7E0606A for ; Thu, 28 Sep 2017 18:48:23 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from mail.inparadise.se (h-246-50.A444.priv.bahnhof.se [155.4.246.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9C197699A0 for ; Thu, 28 Sep 2017 18:48:21 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 790655061A; Thu, 28 Sep 2017 20:48:12 +0200 (CEST) Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id mqrp_F9wcfvL; Thu, 28 Sep 2017 20:48:12 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 29D4C50620; Thu, 28 Sep 2017 20:48:12 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.inparadise.se 29D4C50620 X-Virus-Scanned: amavisd-new at inparadise.se Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Bl62UgcIEA1X; Thu, 28 Sep 2017 20:48:12 +0200 (CEST) Received: from mail.inparadise.se (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 097F95061A; Thu, 28 Sep 2017 20:48:12 +0200 (CEST) Date: Thu, 28 Sep 2017 20:48:11 +0200 (CEST) Subject: Re: How to create holes in files ? Message-ID: <12ed413d-00d1-4182-bdd5-0e5230d46f01@email.android.com> X-Android-Message-ID: <12ed413d-00d1-4182-bdd5-0e5230d46f01@email.android.com> To: Ben RUBSON Cc: Freebsd fs Importance: Normal X-Priority: 3 X-MSMail-Priority: Normal From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= X-Originating-IP: [172.16.1.122, 127.0.0.1] X-Mailer: Zimbra 8.7.1_GA_1670 (Android-Mail/7.8.27.168289052.release(...883836) devip=172.16.1.122 ZPZB/66) Thread-Index: kGwIExfiPIrbpPRsKcK1Nxc9kjmWOA== Thread-Topic: How to create holes in files ? MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 18:48:23 -0000 From owner-freebsd-fs@freebsd.org Thu Sep 28 20:16:45 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B87F1E08321 for ; Thu, 28 Sep 2017 20:16:45 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wr0-x22c.google.com (mail-wr0-x22c.google.com [IPv6:2a00:1450:400c:c0c::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1714C6CAEC for ; Thu, 28 Sep 2017 20:16:45 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wr0-x22c.google.com with SMTP id z39so4729744wrb.8 for ; Thu, 28 Sep 2017 13:16:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:message-id:references:to; bh=6Xs2JrRrbmUPcFf0nX75jSNj8LCrIDcBL5EE2WVMUrw=; b=uKoQun8U/BgAnkENL51lL0dgqxT58K3590GyEADLU3ykybF3D4nIRc/6O9EIJVwbTh lyDfx3jBxKbmSPKrIQViuvmOJ91JkWpH/mpgIYY8XEMfK+g+5jDd2yKsYkxaUL4WHJBN 3SohW9FgaSJE/d7DzVk6h5lHxg+VRRj5Rx6LgkeapgSWKLPoJz9OM4esX3bilmkVvZtp PQJH4VQ+avi9i8VoXt4Cu+HLUCBehkGbJbhM9ybHtJkymOip07OGSPDawlwx1P2OSv+F OULPSySSxnRgZxm1MiZAM1oFIbd+wwyHQ7fVlEgS0wkiqwuAXeAWU2Da89j0M6OSleJG hmzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :message-id:references:to; bh=6Xs2JrRrbmUPcFf0nX75jSNj8LCrIDcBL5EE2WVMUrw=; b=hjLc5/rYhhevp1dCC4A+GMoWlhRrAmG3oGGeOU9Zs2ca50VmwM5KzrNjGrlLRRBYw8 q9ea932bM61icy+1sA/d3/D3hgb+6rM9UD3eF9xV/yL7HOaU1aWNM0fHicReRtsgUHLC qdzbqZzjRLPBao+/DO1CLXC58l2MQ/wpFrtAb7EnO37/7+BdOYRJoMtGwrAaZ5HXZAc8 A6Kn0FKjDFbpSHrSUEHwoPpO+znK53NU4tx8Puqj+PaGKGt1BfG3SXZzsTcAzyWopOHA Gr5QyOIlvej3DdAIul/qUEntWawYqR8bkdrAqRbw9ftiKpNIvzSY0kuNjj8xmPfYi1sr TNgQ== X-Gm-Message-State: AHPjjUjG6NNW6OBP1pBWT1MggoVB/Ub49YHv97JFNCPEgRWjU5cv4u/O hWkgANifmhtkoR+tq1c/ZwCSWLMf X-Google-Smtp-Source: AOwi7QC/b/Hhq3foFdJVk9vkMSnF4w32kNXQq7TgST6w3ewQ5dzpJfdXn9dE8VH9XPEzvYO6Ho9sIg== X-Received: by 10.223.157.203 with SMTP id q11mr5307684wre.96.1506629802902; Thu, 28 Sep 2017 13:16:42 -0700 (PDT) Received: from bens-mac.home (LFbn-MAR-1-445-220.w2-15.abo.wanadoo.fr. [2.15.38.220]) by smtp.gmail.com with ESMTPSA id b190sm1688292wma.41.2017.09.28.13.16.41 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 28 Sep 2017 13:16:42 -0700 (PDT) Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: How to create holes in files ? From: Ben RUBSON X-Priority: 3 In-Reply-To: <12ed413d-00d1-4182-bdd5-0e5230d46f01@email.android.com> Date: Thu, 28 Sep 2017 22:16:40 +0200 Message-Id: <4D304428-E4AF-45D4-AB2E-D4B73A7FEE87@gmail.com> References: <12ed413d-00d1-4182-bdd5-0e5230d46f01@email.android.com> To: Freebsd fs X-Mailer: Apple Mail (2.3124) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 20:16:45 -0000 > On 28 Sep 2017, at 20:48, Karli Sj=C3=B6berg = wrote: >=20 > Den 28 sep. 2017 6:47 em skrev Ben RUBSON : > > On 28 Sep 2017, at 18:34, Bob Eager wrote: > >=20 > > On Thu, 28 Sep 2017 17:26:09 +0200 > > Fabian Keil wrote: > >=20 > >> Ben RUBSON wrote: > >>=20 > >>> I'm trying to make holes in files in C. > >>> Goal is to deallocate huge files on ZFS while (randomly) reading > >>> them.=20 > >>=20 > >> My interpretation of the above is that you want to create holes > >> without changing the file size and without affecting data that > >> is located before or after the holes that you want to create. > >>=20 > >> Otherwise you could simply "deallocate" the content with > >> truncate(1). > >=20 > > If he doesn't mind copying the files, dd(1) will do the job. = However, I > > expect that doesn't meet his criteria. >=20 > Thank you Bob for your suggestion. > You're right goal is to avoid copying data : free space would not = necessarily > allow this, and as the files I'm working on are some hundreds of GB in = size, > it would really be a counterproductive long stressing storage = operation. >=20 > Well, correct me if I'm wrong, but wouldn't dd with "seek" mitigate = the issue of writing out all of the data from beginning to end. If you = seek from the beginning of the file to the point you want to start = writing from and use bs to specify how large of a "hole" you want, the = operation wouldn't take long at all. You would, in my opinion, achieve = exactly what you want, to "create holes in files". Am I wrong? I would have liked to do this in C. And the holes I need to create are not necessarily at the beginning of a = file. I randomly read parts of huge input files that I need to "free" on the = fly to recover storage space for the output files. Ben From owner-freebsd-fs@freebsd.org Thu Sep 28 20:36:14 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 896C8E08B3D; Thu, 28 Sep 2017 20:36:14 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.92]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 501256D5BD; Thu, 28 Sep 2017 20:36:13 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp.greenhost.nl ([213.108.104.138]) by smarthost1.greenhost.nl with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1dxfIc-0001I0-OH; Thu, 28 Sep 2017 22:20:47 +0200 Content-Type: text/plain; charset=iso-8859-15; format=flowed; delsp=yes To: "Mario Lobo" , "Erwan Legrand" Cc: freebsd-fs@freebsd.org, "FreeBSD questions" Subject: Re: mount_smbfs question (re-post) References: <20170927100635.7b56f8fd@Papi.lobos> Date: Thu, 28 Sep 2017 22:20:41 +0200 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Ronald Klop" Message-ID: In-Reply-To: User-Agent: Opera Mail/1.0 (Win32) X-Authenticated-As-Hash: bdb49c4ff80bd276e321aade33e76e02752072e2 X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: -0.2 X-Spam-Status: No, score=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 autolearn=disabled version=3.4.0 X-Scan-Signature: 1fac81774939798d1cdb19633eb460de X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 20:36:14 -0000 On Thu, 28 Sep 2017 19:08:04 +0200, Erwan Legrand wrote: > On Thu, Sep 28, 2017 at 4:16 PM, Mario Lobo wrote: >> 2017-09-27 11:20 GMT-03:00 Erwan Legrand : >>> On Wed, Sep 27, 2017 at 3:06 PM, Mario Lobo >>> wrote: >>> > Since my environment is tottaly surrounded with shares that no longer >>> > accept SMBv1 (Windows, Linux AND FreeBSD servers), so basically in >>> the >>> > end, what I'm really looking for is a confirmation that I'll just >>> have >>> > to dump all Freebsd samba clients because the OS can't deal with >>> SMBv2 >>> > or above. >>> >>> Perhaps have a look at implementations of SMB on top of FUSE? >>> >>> http://portsmon.freebsd.org/portoverview.py?category=sysutils&portname=fusefs-smbnetfs >> >> I did. Same problem. >> smbnetfs only works with SMBv1 >> > > It is based on libsmbclient, thus it should support SMB2 if smb.conf > allows it. According to the following thread, the client protocol is > resticted to SMB1 by default: > > https://lists.samba.org/archive/samba-technical/2016-November/thread.html#116999 > > This might be fixed by setting "client max protocol = SMB2" in > smb.conf. ($HOME/.smb/smb.conf in this case?) I'd suggest setting "client min protocol". ^^^ Regards, Ronald. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Thu Sep 28 20:48:25 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68573E08FF4 for ; Thu, 28 Sep 2017 20:48:25 +0000 (UTC) (envelope-from lobo@bsd.com.br) Received: from mail-yw0-x22f.google.com (mail-yw0-x22f.google.com [IPv6:2607:f8b0:4002:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1FB296DC43 for ; Thu, 28 Sep 2017 20:48:25 +0000 (UTC) (envelope-from lobo@bsd.com.br) Received: by mail-yw0-x22f.google.com with SMTP id u205so1904713ywa.5 for ; Thu, 28 Sep 2017 13:48:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsd.com.br; s=capeta; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=+EDRrX1mF+pi8f5p0o6Xc33geh5M5oc5RA3njIuJhkU=; b=E/1C5PTqe8s3TmgVATaa2IJasOLKXTUT4WrTgC6X0GseLemRhaFSJRpHmteD8AdI4w nviAUivJTgiFohWWzYAnI4m113tB9ysIJ7luTO3SG55lTIRTycWZxnSxrSBtXva/yPQI Sp7ixEMsqlAWvJ87Uhi3rBLDFU/YuR6irseGg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=+EDRrX1mF+pi8f5p0o6Xc33geh5M5oc5RA3njIuJhkU=; b=Bwd/j6gAReW2C+BkoNDhoE6wfePinK2/F0wHiiG79ElBDO9wZrHSJGASL2CHLuSREV fqGSIIw89NF4bQhPGCo59791o5RZ0zha1oAGIW9NHIyMs04cVtRqTcmpyrPmNV1OSyun U9KE1ov33jH2jIRhUSivieOv2KZ8qc8wCDXgvWtuQZlp1eGCpe4g1hd+XY/ALj5uyGx4 5KLjCtJakBzBoyqH8aHq/7EEYmKd3jycERkLm9s73qZzdcqJQ3U1iW1Ok5DoZEIzkRTc HkzMUEAqe7wHOr0n9RqCXDb8fwFhbM+iqq/tTfIV+gwUTa1xlb8xXHNOzgNS9knNp0qv iO5A== X-Gm-Message-State: AHPjjUhwzeKbmdJrTuydcxuDQMXhIoJr+VYi1EVkwgsk3zRX6eGOANMy zet/2fCcfaEgPe5hC3c+vsQ3VN2sCbmiEKNl/MxhaQ== X-Google-Smtp-Source: AOwi7QCcCe02qg3auxz/M9AbeMwqj4e3Y8WRICWY7r//G04cmoHwmP4R3KSCKLn3cZVdqPskpq332xAn2pVRhaCaQFU= X-Received: by 10.129.109.151 with SMTP id i145mr4605218ywc.82.1506631704128; Thu, 28 Sep 2017 13:48:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.37.170.193 with HTTP; Thu, 28 Sep 2017 13:48:23 -0700 (PDT) In-Reply-To: References: <20170927100635.7b56f8fd@Papi.lobos> From: Mario Lobo Date: Thu, 28 Sep 2017 17:48:23 -0300 Message-ID: Subject: Re: mount_smbfs question (re-post) To: Ronald Klop Cc: Erwan Legrand , freebsd-fs@freebsd.org, FreeBSD questions Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 20:48:25 -0000 2017-09-28 17:20 GMT-03:00 Ronald Klop : > On Thu, 28 Sep 2017 19:08:04 +0200, Erwan Legrand < > freebsd@erwanlegrand.com> wrote: > > On Thu, Sep 28, 2017 at 4:16 PM, Mario Lobo wrote: >> >>> 2017-09-27 11:20 GMT-03:00 Erwan Legrand : >>> >>>> On Wed, Sep 27, 2017 at 3:06 PM, Mario Lobo >>>> wrote: >>>> > Since my environment is tottaly surrounded with shares that no longer >>>> > accept SMBv1 (Windows, Linux AND FreeBSD servers), so basically in the >>>> > end, what I'm really looking for is a confirmation that I'll just have >>>> > to dump all Freebsd samba clients because the OS can't deal with SMBv2 >>>> > or above. >>>> >>>> Perhaps have a look at implementations of SMB on top of FUSE? >>>> >>>> http://portsmon.freebsd.org/portoverview.py?category=sysutil >>>> s&portname=fusefs-smbnetfs >>>> >>> >>> I did. Same problem. >>> smbnetfs only works with SMBv1 >>> >>> >> It is based on libsmbclient, thus it should support SMB2 if smb.conf >> allows it. According to the following thread, the client protocol is >> resticted to SMB1 by default: >> >> https://lists.samba.org/archive/samba-technical/2016-Novembe >> r/thread.html#116999 >> >> This might be fixed by setting "client max protocol = SMB2" in >> smb.conf. ($HOME/.smb/smb.conf in this case?) >> > > I'd suggest setting "client min protocol". > ^^^ > > Regards, > Ronald. > > > _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> > I just tested it! 2 shares. 1 with SMBv1 and 1 with client min protocol=SMBv2 They are both SAMBA with FREEBSD. The first I can read/write anywhere in it The second I can't even see whats in it. These are the msgs from running smbnetfs -d /shares when accessing the SMBv1 share (OK): unique: 1178, opcode: READLINK (5), nodeid: 11, insize: 40, pid: 36429 readlink /SEAWAY/NEWSITE 1025 unique: 1178, success, outsize: 26 unique: 1179, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36429 getattr / unique: 1179, success, outsize: 112 unique: 1180, opcode: LOOKUP (1), nodeid: 12, insize: 51, pid: 36429 LOOKUP /NEWSITE/.directory getattr /NEWSITE/.directory unique: 1180, error: -2 (No such file or directory), outsize: 16 unique: 1181, opcode: READLINK (5), nodeid: 11, insize: 40, pid: 36429 readlink /SEAWAY/NEWSITE 1025 unique: 1181, success, outsize: 26 unique: 1182, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36429 getattr / unique: 1182, success, outsize: 112 unique: 1183, opcode: READLINK (5), nodeid: 11, insize: 40, pid: 36429 readlink /SEAWAY/NEWSITE 1025 unique: 1183, success, outsize: 26 unique: 1184, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36429 getattr / unique: 1184, success, outsize: 112 unique: 1185, opcode: LOOKUP (1), nodeid: 12, insize: 51, pid: 36429 LOOKUP /NEWSITE/.directory getattr /NEWSITE/.directory unique: 1185, error: -2 (No such file or directory), outsize: 16 unique: 1186, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36429 getattr / unique: 1186, success, outsize: 112 unique: 1187, opcode: GETATTR (3), nodeid: 3, insize: 40, pid: 36429 getattr /SEAWAY unique: 1187, success, outsize: 112 unique: 1188, opcode: GETATTR (3), nodeid: 11, insize: 40, pid: 36429 getattr /SEAWAY/NEWSITE unique: 1188, success, outsize: 112 unique: 1189, opcode: READLINK (5), nodeid: 11, insize: 40, pid: 36429 readlink /SEAWAY/NEWSITE 1025 unique: 1189, success, outsize: 26 unique: 1190, opcode: READLINK (5), nodeid: 11, insize: 40, pid: 36429 readlink /SEAWAY/NEWSITE 1025 unique: 1190, success, outsize: 26 unique: 1191, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36429 getattr / unique: 1191, success, outsize: 112 unique: 1192, opcode: GETATTR (3), nodeid: 12, insize: 40, pid: 36429 getattr /NEWSITE unique: 1192, success, outsize: 112 unique: 1193, opcode: GETATTR (3), nodeid: 3, insize: 40, pid: 36429 getattr /SEAWAY unique: 1193, success, outsize: 112 unique: 1194, opcode: GETATTR (3), nodeid: 12, insize: 40, pid: 36429 getattr /NEWSITE unique: 1194, success, outsize: 112 unique: 1195, opcode: GETATTR (3), nodeid: 11, insize: 40, pid: 36429 getattr /SEAWAY/NEWSITE unique: 1195, success, outsize: 112 unique: 1196, opcode: READLINK (5), nodeid: 11, insize: 40, pid: 36429 readlink /SEAWAY/NEWSITE 1025 when accessing the SMBv2 share (BAD): readlink /SEAWAY/SITE-DESIGN 1025 unique: 1157, success, outsize: 30 unique: 1158, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36446 getattr / unique: 1158, success, outsize: 112 unique: 1159, opcode: OPENDIR (27), nodeid: 18, insize: 48, pid: 36446 opendir flags: 0x0 /SITE-DESIGN unique: 1159, error: -5 (Input/output error), outsize: 16 2017-09-28 17:40:21.948 srv(39637)->smb_conn_srv_opendir: errno=22, Invalid argument unique: 1160, opcode: GETATTR (3), nodeid: 17, insize: 40, pid: 36429 getattr /SEAWAY/SITE-DESIGN unique: 1160, success, outsize: 112 unique: 1161, opcode: READLINK (5), nodeid: 17, insize: 40, pid: 36429 readlink /SEAWAY/SITE-DESIGN 1025 unique: 1161, success, outsize: 30 unique: 1162, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36429 getattr / unique: 1162, success, outsize: 112 unique: 1163, opcode: GETATTR (3), nodeid: 18, insize: 40, pid: 36429 getattr /SITE-DESIGN unique: 1163, success, outsize: 112 unique: 1164, opcode: READLINK (5), nodeid: 17, insize: 40, pid: 36429 readlink /SEAWAY/SITE-DESIGN 1025 unique: 1164, success, outsize: 30 unique: 1165, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36429 getattr / unique: 1165, success, outsize: 112 unique: 1166, opcode: GETATTR (3), nodeid: 17, insize: 40, pid: 36429 getattr /SEAWAY/SITE-DESIGN unique: 1166, success, outsize: 112 Unless I'm missing some tuning option in smbnetfs for SMBv2 and above, It doesen't work! Thanks, -- Mario Lobo http://www.mallavoodoo.com.br FreeBSD since version 2.2.8 [not Pro-Audio.... YET!!] (99,7% winfoes FREE) From owner-freebsd-fs@freebsd.org Thu Sep 28 21:39:19 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B9B72E0CF9A for ; Thu, 28 Sep 2017 21:39:19 +0000 (UTC) (envelope-from lobo@bsd.com.br) Received: from mail-qk0-x244.google.com (mail-qk0-x244.google.com [IPv6:2607:f8b0:400d:c09::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 76CB56F20B for ; Thu, 28 Sep 2017 21:39:18 +0000 (UTC) (envelope-from lobo@bsd.com.br) Received: by mail-qk0-x244.google.com with SMTP id g128so1904408qke.0 for ; Thu, 28 Sep 2017 14:39:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsd.com.br; s=capeta; h=date:from:to:cc:subject:message-id:in-reply-to:references :organization:mime-version:content-transfer-encoding; bh=YYTRXIuDeGWEXpdp2T83ELhnAPVEcrgYZbOQyWlCC/k=; b=EMsu0AMFZsD4m+uyHZZDAXzwz8J8EFgz4sjUv4XztGe8PanoQFVKlt2OoUEJF3mUOa XldD+wZmMc260kW6Pbax5I0zinC7NGzDooXEBEKhFpI8GICRk4pHgG8H2eWif1xJABN6 Ud6xHpmCP+zQPlEYxtBOfSwtpza92k1NXl49I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:organization:mime-version:content-transfer-encoding; bh=YYTRXIuDeGWEXpdp2T83ELhnAPVEcrgYZbOQyWlCC/k=; b=hQcNKyObFZyQjs/I7BKgCDX8ao3VL/4UAuJwa2+Iid4VETHGZaCi6YiUNIlPour5uA 3e2r025E5/y8Rn+WuIU6GEqLJFgbnJvOby2sDnDX2NqmHBRe2CE7sYNPgGYa4sMyJNmk mPEFLUkJeJchUTTfmJN6auZWRTE4FXjp2Zh+2YsPEUAlsB801ykYTbXwi0+tjSYj4TAz uldIn2gVO3h8duL2rbi/TWLbelKwCe2Bbp7qdwpOK4/7SS4EDMBgLOQY/wiEwUz+xsBO RbulMsvOYmr6bls7cHogenxK2l+Bq7pews27gtsOCYh6dptwMVWIG9Ko5COq3TX1RyE9 HuUg== X-Gm-Message-State: AMCzsaWVNMp4AD0OcBTVSUOGkvkQ8mui8TQzxL5k/vUa64F8+hWU5X8f 3k5G6upUceHrXNSHQagvOWJpsg== X-Google-Smtp-Source: AOwi7QBA52j7W29GpHUJPfvjSgxHbYbcg6p+ieyavurl/usnKzhNneKLSVruqj/Hf8yP8kywjVYOQQ== X-Received: by 10.55.104.132 with SMTP id d126mr523126qkc.142.1506634757667; Thu, 28 Sep 2017 14:39:17 -0700 (PDT) Received: from Papi.lobos ([187.59.113.250]) by smtp.gmail.com with ESMTPSA id v32sm1700728qtc.66.2017.09.28.14.39.15 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 28 Sep 2017 14:39:17 -0700 (PDT) Date: Thu, 28 Sep 2017 18:41:01 -0300 From: Mario Lobo To: Ronald Klop Cc: Erwan Legrand , freebsd-fs@freebsd.org, FreeBSD questions Subject: Re: mount_smbfs question (re-post) Message-ID: <20170928184101.55c8a0ec@Papi.lobos> In-Reply-To: References: <20170927100635.7b56f8fd@Papi.lobos> Organization: BSD X-Mailer: Claws Mail 3.15.0 (GTK+ 2.24.29; amd64-portbld-freebsd11.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 21:39:19 -0000 On Thu, 28 Sep 2017 17:48:23 -0300 Mario Lobo wrote: > 2017-09-28 17:20 GMT-03:00 Ronald Klop : > > > On Thu, 28 Sep 2017 19:08:04 +0200, Erwan Legrand < > > freebsd@erwanlegrand.com> wrote: > > > > On Thu, Sep 28, 2017 at 4:16 PM, Mario Lobo > > wrote: > >> > >>> 2017-09-27 11:20 GMT-03:00 Erwan Legrand > >>> : > >>>> On Wed, Sep 27, 2017 at 3:06 PM, Mario Lobo > >>>> wrote: > >>>> > Since my environment is tottaly surrounded with shares that no > >>>> > longer accept SMBv1 (Windows, Linux AND FreeBSD servers), so > >>>> > basically in the end, what I'm really looking for is a > >>>> > confirmation that I'll just have to dump all Freebsd samba > >>>> > clients because the OS can't deal with SMBv2 or above. > >>>> > >>>> Perhaps have a look at implementations of SMB on top of FUSE? > >>>> > >>>> http://portsmon.freebsd.org/portoverview.py?category=sysutil > >>>> s&portname=fusefs-smbnetfs > >>>> > >>> > >>> I did. Same problem. > >>> smbnetfs only works with SMBv1 > >>> > >>> > >> It is based on libsmbclient, thus it should support SMB2 if > >> smb.conf allows it. According to the following thread, the client > >> protocol is resticted to SMB1 by default: > >> > >> https://lists.samba.org/archive/samba-technical/2016-Novembe > >> r/thread.html#116999 > >> > >> This might be fixed by setting "client max protocol = SMB2" in > >> smb.conf. ($HOME/.smb/smb.conf in this case?) > >> > > > > I'd suggest setting "client min protocol". > > ^^^ > > > > Regards, > > Ronald. > > > > > > _______________________________________________ > >> freebsd-fs@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > >> To unsubscribe, send any mail to > >> "freebsd-fs-unsubscribe@freebsd.org" > > > I just tested it! > > 2 shares. 1 with SMBv1 and 1 with client min protocol=SMBv2 > They are both SAMBA with FREEBSD. > > The first I can read/write anywhere in it > The second I can't even see whats in it. > > These are the msgs from running smbnetfs -d /shares > > when accessing the SMBv1 share (OK): > > unique: 1178, opcode: READLINK (5), nodeid: 11, insize: 40, pid: 36429 > readlink /SEAWAY/NEWSITE 1025 > unique: 1178, success, outsize: 26 > unique: 1179, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36429 > getattr / > unique: 1179, success, outsize: 112 > unique: 1180, opcode: LOOKUP (1), nodeid: 12, insize: 51, pid: 36429 > LOOKUP /NEWSITE/.directory > getattr /NEWSITE/.directory > unique: 1180, error: -2 (No such file or directory), outsize: 16 > unique: 1181, opcode: READLINK (5), nodeid: 11, insize: 40, pid: 36429 > readlink /SEAWAY/NEWSITE 1025 > unique: 1181, success, outsize: 26 > unique: 1182, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36429 > getattr / > unique: 1182, success, outsize: 112 > unique: 1183, opcode: READLINK (5), nodeid: 11, insize: 40, pid: 36429 > readlink /SEAWAY/NEWSITE 1025 > unique: 1183, success, outsize: 26 > unique: 1184, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36429 > getattr / > unique: 1184, success, outsize: 112 > unique: 1185, opcode: LOOKUP (1), nodeid: 12, insize: 51, pid: 36429 > LOOKUP /NEWSITE/.directory > getattr /NEWSITE/.directory > unique: 1185, error: -2 (No such file or directory), outsize: 16 > unique: 1186, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36429 > getattr / > unique: 1186, success, outsize: 112 > unique: 1187, opcode: GETATTR (3), nodeid: 3, insize: 40, pid: 36429 > getattr /SEAWAY > unique: 1187, success, outsize: 112 > unique: 1188, opcode: GETATTR (3), nodeid: 11, insize: 40, pid: 36429 > getattr /SEAWAY/NEWSITE > unique: 1188, success, outsize: 112 > unique: 1189, opcode: READLINK (5), nodeid: 11, insize: 40, pid: 36429 > readlink /SEAWAY/NEWSITE 1025 > unique: 1189, success, outsize: 26 > unique: 1190, opcode: READLINK (5), nodeid: 11, insize: 40, pid: 36429 > readlink /SEAWAY/NEWSITE 1025 > unique: 1190, success, outsize: 26 > unique: 1191, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36429 > getattr / > unique: 1191, success, outsize: 112 > unique: 1192, opcode: GETATTR (3), nodeid: 12, insize: 40, pid: 36429 > getattr /NEWSITE > unique: 1192, success, outsize: 112 > unique: 1193, opcode: GETATTR (3), nodeid: 3, insize: 40, pid: 36429 > getattr /SEAWAY > unique: 1193, success, outsize: 112 > unique: 1194, opcode: GETATTR (3), nodeid: 12, insize: 40, pid: 36429 > getattr /NEWSITE > unique: 1194, success, outsize: 112 > unique: 1195, opcode: GETATTR (3), nodeid: 11, insize: 40, pid: 36429 > getattr /SEAWAY/NEWSITE > unique: 1195, success, outsize: 112 > unique: 1196, opcode: READLINK (5), nodeid: 11, insize: 40, pid: 36429 > readlink /SEAWAY/NEWSITE 1025 > > > > when accessing the SMBv2 share (BAD): > > readlink /SEAWAY/SITE-DESIGN 1025 > unique: 1157, success, outsize: 30 > unique: 1158, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36446 > getattr / > unique: 1158, success, outsize: 112 > unique: 1159, opcode: OPENDIR (27), nodeid: 18, insize: 48, pid: 36446 > opendir flags: 0x0 /SITE-DESIGN > unique: 1159, error: -5 (Input/output error), outsize: 16 > 2017-09-28 17:40:21.948 srv(39637)->smb_conn_srv_opendir: errno=22, > Invalid argument > unique: 1160, opcode: GETATTR (3), nodeid: 17, insize: 40, pid: 36429 > getattr /SEAWAY/SITE-DESIGN > unique: 1160, success, outsize: 112 > unique: 1161, opcode: READLINK (5), nodeid: 17, insize: 40, pid: 36429 > readlink /SEAWAY/SITE-DESIGN 1025 > unique: 1161, success, outsize: 30 > unique: 1162, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36429 > getattr / > unique: 1162, success, outsize: 112 > unique: 1163, opcode: GETATTR (3), nodeid: 18, insize: 40, pid: 36429 > getattr /SITE-DESIGN > unique: 1163, success, outsize: 112 > unique: 1164, opcode: READLINK (5), nodeid: 17, insize: 40, pid: 36429 > readlink /SEAWAY/SITE-DESIGN 1025 > unique: 1164, success, outsize: 30 > unique: 1165, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 36429 > getattr / > unique: 1165, success, outsize: 112 > unique: 1166, opcode: GETATTR (3), nodeid: 17, insize: 40, pid: 36429 > getattr /SEAWAY/SITE-DESIGN > unique: 1166, success, outsize: 112 > > > Unless I'm missing some tuning option in smbnetfs for SMBv2 and above, > > It doesen't work! > > Thanks, > One more thing. If I gear down the second server to SMBv1, I can access it just fine. -- Mario Lobo http://www.mallavoodoo.com.br FreeBSD since 2.2.8 [not Pro-Audio.... YET!!] "UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things." From owner-freebsd-fs@freebsd.org Fri Sep 29 05:54:45 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8E0B2E24183 for ; Fri, 29 Sep 2017 05:54:45 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from mail.inparadise.se (h-246-50.A444.priv.bahnhof.se [155.4.246.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 281637F0AD for ; Fri, 29 Sep 2017 05:54:44 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 5116F48954; Fri, 29 Sep 2017 07:54:40 +0200 (CEST) Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id uJ5oLM6lZtOQ; Fri, 29 Sep 2017 07:54:39 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id AB7334A4A3; Fri, 29 Sep 2017 07:54:39 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.inparadise.se AB7334A4A3 X-Virus-Scanned: amavisd-new at inparadise.se Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id WodTe1mO7sQV; Fri, 29 Sep 2017 07:54:39 +0200 (CEST) Received: from vpnpool190-35.anst.uu.se (vpnpool190-35.anst.uu.se [130.238.190.35]) by mail.inparadise.se (Postfix) with ESMTPSA id 85DF048954; Fri, 29 Sep 2017 07:54:39 +0200 (CEST) Message-ID: <1506664468.7204.37.camel@inparadise.se> Subject: Re: How to create holes in files ? From: Karli =?ISO-8859-1?Q?Sj=F6berg?= Reply-To: karli@inparadise.se To: Ben RUBSON , Freebsd fs Date: Fri, 29 Sep 2017 07:54:28 +0200 In-Reply-To: <4D304428-E4AF-45D4-AB2E-D4B73A7FEE87@gmail.com> References: <12ed413d-00d1-4182-bdd5-0e5230d46f01@email.android.com> <4D304428-E4AF-45D4-AB2E-D4B73A7FEE87@gmail.com> Organization: InParadise Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.18.5.2-0ubuntu3.2 Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 05:54:45 -0000 On tor, 2017-09-28 at 22:16 +0200, Ben RUBSON wrote: > >=20 > > On 28 Sep 2017, at 20:48, Karli Sj=C3=B6berg > > wrote: > >=20 > > Den 28 sep. 2017 6:47 em skrev Ben RUBSON : > > >=20 > > > On 28 Sep 2017, at 18:34, Bob Eager wrote: > > >=20 > > > On Thu, 28 Sep 2017 17:26:09 +0200 > > > Fabian Keil wrote: > > >=20 > > > >=20 > > > > Ben RUBSON wrote: > > > >=20 > > > > >=20 > > > > > I'm trying to make holes in files in C. > > > > > Goal is to deallocate huge files on ZFS while (randomly) > > > > > reading > > > > > them.=C2=A0 > > > > My interpretation of the above is that you want to create holes > > > > without changing the file size and without affecting data that > > > > is located before or after the holes that you want to create. > > > >=20 > > > > Otherwise you could simply "deallocate" the content with > > > > truncate(1). > > > If he doesn't mind copying the files, dd(1) will do the job. > > > However, I > > > expect that doesn't meet his criteria. > > Thank you Bob for your suggestion. > > You're right goal is to avoid copying data : free space would not > > necessarily > > allow this, and as the files I'm working on are some hundreds of GB > > in size, > > it would really be a counterproductive long stressing storage > > operation. > >=20 > > Well, correct me if I'm wrong, but wouldn't dd with "seek" mitigate > > the issue of writing out all of the data from beginning to end. If > > you seek from the beginning of the file to the point you want to > > start writing from and use bs to specify how large of a "hole" you > > want, the operation wouldn't take long at all. You would, in my > > opinion, achieve exactly what you want, to "create holes in files". > > Am I wrong? > I would have liked to do this in C. Yepp, I understand that, I was just bringing up the idea. But hey, if dd can do it, just look in the source of how it does it? > And the holes I need to create are not necessarily at the beginning > of a file. No, you _do not_ have to create holes in the beginning of a file, you can do it anywhere in the file you like. E.g. it=C2=B4s the trick I use t= o "clean" hardrives to make them look empty, by only erasing the first and last 10 MB of the drives, a whole JBOD only takes seconds to clean. Say you have a 10 MB large file and you want to make a 1 MB large hole somewhere in the middle of it, you do it like this: # dd if=3D/dev/zero of=3D/foo/bar.bin bs=3D1M seek=3D7 It=C2=B4ll make the file look like this inside (hope the ASCII gods are w= ith me): =C2=A0____________ |_______|_|__| Important to know is that "bs" determines the unit by wich you "seek"! I don=C2=B4t know if this might be a solution to your problem, just throw= ing it out there. /K > I randomly read parts of huge input files that I need to "free" on > the fly > to recover storage space for the output files. >=20 > Ben >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Fri Sep 29 06:26:18 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50159E24A23 for ; Fri, 29 Sep 2017 06:26:18 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wr0-x236.google.com (mail-wr0-x236.google.com [IPv6:2a00:1450:400c:c0c::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DFB0A7FCB9 for ; Fri, 29 Sep 2017 06:26:17 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wr0-x236.google.com with SMTP id a43so615900wrc.0 for ; Thu, 28 Sep 2017 23:26:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=qcq5VfOd0CMFOGS8oqTaDS7Zd78+22TseRCDFp9SL5o=; b=jbZo3J0/65t2EtcXTC/Z06Nh9UuJfgG0AmCp+R5WWOtsXjkTF4IofZyLjOa5zRk9F8 wiNK2V6KVMnLwYvOwuik2DVMMnHFrwQs6GsAefdIwxspAceIWQSDcL0PJnMIjwOQw4Em uV4yKt6iiHTaFhiHUo7OE7OFqSVD/ooISn5ctMPoKF4XaC9VVwAyIbYyU78nhTYYwqaA jmhzHi6p6f2prfBuLDyqOWeJU+xCy6fjUkQeyelr463R0HK3+Z76nY4dIoLzS4IWuWNK 08MtpWrDyBKsPDYF90ZWJ9XR/wtil2nKoRViXw6CNKOuWy5JTBP2ZIWbzGtdIMKyPym9 rjtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=qcq5VfOd0CMFOGS8oqTaDS7Zd78+22TseRCDFp9SL5o=; b=k8LOzXp2Wg5YyVjHoC5ogOJWb4HAuKgA6dJLTU3KNx6YxvCvG6qXczuGpUGSJe74kz 9WSV92CDy41NBPbJiINP97va58DqQQ56hRy0bSjb2itDlRmQwC0bL97HKm+9HSmFyOJO dt5fBrCwVU+VJt3BhJ9mmFM3JJhDjQDsyVVSG74/hVtMF0qZu4mC6zvdZm7f/PnmCTOn Il8orwxJFbbE5FtMNbvefuRskqTIpgkWefjwHpio3aRM4TC7uHfQZH+qDIQISoThbdyn wXcOxerIvpL6Lupy6ikPzz6zg09ALukmdCPJM4Xp0z8tLBv/SrlOJ0fmcIIXe1pZeKcP hfqw== X-Gm-Message-State: AHPjjUgkXomTGA7FnRVqms7xdzk/WvvvvA6OUPT1hwqM8bHv8Y/qgTn4 gtu9mcV3fJstDEJ5UZ2ztdj/p3bv X-Google-Smtp-Source: AOwi7QAqxQLn4NDRsPepUlbXHIjg6r/gTf1XHdroQsv//V3z9EEPuYUdyWtjzGMsivKrljSImYIaRw== X-Received: by 10.223.196.73 with SMTP id a9mr6260851wrg.144.1506666375924; Thu, 28 Sep 2017 23:26:15 -0700 (PDT) Received: from bens-mac.home (LFbn-MAR-1-445-220.w2-15.abo.wanadoo.fr. [2.15.38.220]) by smtp.gmail.com with ESMTPSA id 204sm2178326wml.10.2017.09.28.23.26.14 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 28 Sep 2017 23:26:15 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: How to create holes in files ? From: Ben RUBSON In-Reply-To: <1506664468.7204.37.camel@inparadise.se> Date: Fri, 29 Sep 2017 08:26:13 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <454261D6-0061-4DB9-8184-BDD7061870FB@gmail.com> References: <12ed413d-00d1-4182-bdd5-0e5230d46f01@email.android.com> <4D304428-E4AF-45D4-AB2E-D4B73A7FEE87@gmail.com> <1506664468.7204.37.camel@inparadise.se> To: Freebsd fs X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 06:26:18 -0000 > On 29 Sep 2017, at 07:54, Karli Sj=C3=B6berg = wrote: >=20 > On tor, 2017-09-28 at 22:16 +0200, Ben RUBSON wrote: >>>=20 >>> On 28 Sep 2017, at 20:48, Karli Sj=C3=B6berg >>> wrote: >>>=20 >>> Den 28 sep. 2017 6:47 em skrev Ben RUBSON : >>>>=20 >>>> On 28 Sep 2017, at 18:34, Bob Eager wrote: >>>>=20 >>>> On Thu, 28 Sep 2017 17:26:09 +0200 >>>> Fabian Keil wrote: >>>>=20 >>>>>=20 >>>>> Ben RUBSON wrote: >>>>>=20 >>>>>>=20 >>>>>> I'm trying to make holes in files in C. >>>>>> Goal is to deallocate huge files on ZFS while (randomly) >>>>>> reading >>>>>> them.=20 >>>>> My interpretation of the above is that you want to create holes >>>>> without changing the file size and without affecting data that >>>>> is located before or after the holes that you want to create. >>>>>=20 >>>>> Otherwise you could simply "deallocate" the content with >>>>> truncate(1). >>>> If he doesn't mind copying the files, dd(1) will do the job. >>>> However, I >>>> expect that doesn't meet his criteria. >>> Thank you Bob for your suggestion. >>> You're right goal is to avoid copying data : free space would not >>> necessarily >>> allow this, and as the files I'm working on are some hundreds of GB >>> in size, >>> it would really be a counterproductive long stressing storage >>> operation. >>>=20 >>> Well, correct me if I'm wrong, but wouldn't dd with "seek" mitigate >>> the issue of writing out all of the data from beginning to end. If >>> you seek from the beginning of the file to the point you want to >>> start writing from and use bs to specify how large of a "hole" you >>> want, the operation wouldn't take long at all. You would, in my >>> opinion, achieve exactly what you want, to "create holes in files". >>> Am I wrong? >> I would have liked to do this in C. >=20 > Yepp, I understand that, I was just bringing up the idea. But hey, if > dd can do it, just look in the source of how it does it? Of course yes, you're right :) >> And the holes I need to create are not necessarily at the beginning >> of a file. >=20 > No, you _do not_ have to create holes in the beginning of a file, you > can do it anywhere in the file you like. E.g. it=C2=B4s the trick I = use to > "clean" hardrives to make them look empty, by only erasing the first > and last 10 MB of the drives, a whole JBOD only takes seconds to = clean. > Say you have a 10 MB large file and you want to make a 1 MB large hole > somewhere in the middle of it, you do it like this: >=20 > # dd if=3D/dev/zero of=3D/foo/bar.bin bs=3D1M seek=3D7 >=20 > It=C2=B4ll make the file look like this inside (hope the ASCII gods = are with > me): > ____________ > |_______|_|__| Unfortunately here storage blocks will be set/written to 0, they will not be freed/recovered, a file hole will not be created. Below are some commands which demonstrate this. The last 3/4 commands show a _newly_ created file with a hole. # zfs userspace home POSIX User root 36.5K none # dd if=3D/dev/random of=3Df bs=3D1M count=3D4 4+0 records out # zfs userspace home POSIX User root 4.04M none # dd if=3D/dev/zero of=3Df bs=3D1M count=3D2 seek=3D2 2+0 records out # zfs userspace home POSIX User root 4.04M none # rm f # dd if=3D/dev/zero of=3Df bs=3D1M count=3D2 seek=3D2 2+0 records out # zfs userspace home POSIX User root 2.04M none # ls -lh f -rw------- 1 root wheel 4.0M 29 Sep 08:19 f From owner-freebsd-fs@freebsd.org Fri Sep 29 06:34:01 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C8CC5E24CEF for ; Fri, 29 Sep 2017 06:34:01 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from mail.inparadise.se (h-246-50.A444.priv.bahnhof.se [155.4.246.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6CDED80016 for ; Fri, 29 Sep 2017 06:34:01 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 45D264A9E0; Fri, 29 Sep 2017 08:33:59 +0200 (CEST) Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id ujHXEG04xetE; Fri, 29 Sep 2017 08:33:58 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id A39314B740; Fri, 29 Sep 2017 08:33:58 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.inparadise.se A39314B740 X-Virus-Scanned: amavisd-new at inparadise.se Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 02UtlZVP-ynq; Fri, 29 Sep 2017 08:33:58 +0200 (CEST) Received: from vpnpool190-35.anst.uu.se (vpnpool190-35.anst.uu.se [130.238.190.35]) by mail.inparadise.se (Postfix) with ESMTPSA id 85B2E4A633; Fri, 29 Sep 2017 08:33:58 +0200 (CEST) Message-ID: <1506666838.7204.39.camel@inparadise.se> Subject: Re: How to create holes in files ? From: Karli =?ISO-8859-1?Q?Sj=F6berg?= Reply-To: karli@inparadise.se To: Ben RUBSON , Freebsd fs Date: Fri, 29 Sep 2017 08:33:58 +0200 In-Reply-To: <454261D6-0061-4DB9-8184-BDD7061870FB@gmail.com> References: <12ed413d-00d1-4182-bdd5-0e5230d46f01@email.android.com> <4D304428-E4AF-45D4-AB2E-D4B73A7FEE87@gmail.com> <1506664468.7204.37.camel@inparadise.se> <454261D6-0061-4DB9-8184-BDD7061870FB@gmail.com> Organization: InParadise Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.18.5.2-0ubuntu3.2 Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 06:34:01 -0000 On fre, 2017-09-29 at 08:26 +0200, Ben RUBSON wrote: > >=20 > > On 29 Sep 2017, at 07:54, Karli Sj=C3=B6berg > > wrote: > >=20 > > On tor, 2017-09-28 at 22:16 +0200, Ben RUBSON wrote: > > >=20 > > > >=20 > > > >=20 > > > > On 28 Sep 2017, at 20:48, Karli Sj=C3=B6berg > > > > wrote: > > > >=20 > > > > Den 28 sep. 2017 6:47 em skrev Ben RUBSON > > > >: > > > > >=20 > > > > >=20 > > > > > On 28 Sep 2017, at 18:34, Bob Eager wrote: > > > > >=20 > > > > > On Thu, 28 Sep 2017 17:26:09 +0200 > > > > > Fabian Keil wrote: > > > > >=20 > > > > > >=20 > > > > > >=20 > > > > > > Ben RUBSON wrote: > > > > > >=20 > > > > > > >=20 > > > > > > >=20 > > > > > > > I'm trying to make holes in files in C. > > > > > > > Goal is to deallocate huge files on ZFS while (randomly) > > > > > > > reading > > > > > > > them.=C2=A0 > > > > > > My interpretation of the above is that you want to create > > > > > > holes > > > > > > without changing the file size and without affecting data > > > > > > that > > > > > > is located before or after the holes that you want to > > > > > > create. > > > > > >=20 > > > > > > Otherwise you could simply "deallocate" the content with > > > > > > truncate(1). > > > > > If he doesn't mind copying the files, dd(1) will do the job. > > > > > However, I > > > > > expect that doesn't meet his criteria. > > > > Thank you Bob for your suggestion. > > > > You're right goal is to avoid copying data : free space would > > > > not > > > > necessarily > > > > allow this, and as the files I'm working on are some hundreds > > > > of GB > > > > in size, > > > > it would really be a counterproductive long stressing storage > > > > operation. > > > >=20 > > > > Well, correct me if I'm wrong, but wouldn't dd with "seek" > > > > mitigate > > > > the issue of writing out all of the data from beginning to end. > > > > If > > > > you seek from the beginning of the file to the point you want > > > > to > > > > start writing from and use bs to specify how large of a "hole" > > > > you > > > > want, the operation wouldn't take long at all. You would, in my > > > > opinion, achieve exactly what you want, to "create holes in > > > > files". > > > > Am I wrong? > > > I would have liked to do this in C. > > Yepp, I understand that, I was just bringing up the idea. But hey, > > if > > dd can do it, just look in the source of how it does it? > Of course yes, you're right :) >=20 > >=20 > > >=20 > > > And the holes I need to create are not necessarily at the > > > beginning > > > of a file. > > No, you _do not_ have to create holes in the beginning of a file, > > you > > can do it anywhere in the file you like. E.g. it=C2=B4s the trick I u= se > > to > > "clean" hardrives to make them look empty, by only erasing the > > first > > and last 10 MB of the drives, a whole JBOD only takes seconds to > > clean. > > Say you have a 10 MB large file and you want to make a 1 MB large > > hole > > somewhere in the middle of it, you do it like this: > >=20 > > # dd if=3D/dev/zero of=3D/foo/bar.bin bs=3D1M seek=3D7 > >=20 > > It=C2=B4ll make the file look like this inside (hope the ASCII gods a= re > > with > > me): > > =C2=A0____________ > > >=20 > > > _______|_|__| > Unfortunately here storage blocks will be set/written to 0, > they will not be freed/recovered, a file hole will not be created. > Below are some commands which demonstrate this. > The last 3/4 commands show a _newly_ created file with a hole. >=20 > # zfs userspace home > POSIX User=C2=A0=C2=A0root=C2=A0=C2=A036.5K=C2=A0=C2=A0=C2=A0none >=20 > # dd if=3D/dev/random of=3Df bs=3D1M count=3D4 > 4+0 records out >=20 > # zfs userspace home > POSIX User=C2=A0=C2=A0root=C2=A0=C2=A04.04M=C2=A0=C2=A0=C2=A0none >=20 > # dd if=3D/dev/zero of=3Df bs=3D1M count=3D2 seek=3D2 > 2+0 records out >=20 > # zfs userspace home > POSIX User=C2=A0=C2=A0root=C2=A0=C2=A04.04M=C2=A0=C2=A0=C2=A0none >=20 > # rm f >=20 > # dd if=3D/dev/zero of=3Df bs=3D1M count=3D2 seek=3D2 > 2+0 records out >=20 > # zfs userspace home > POSIX User=C2=A0=C2=A0root=C2=A0=C2=A02.04M=C2=A0=C2=A0=C2=A0none >=20 > # ls -lh f > -rw-------=C2=A0=C2=A01 root=C2=A0=C2=A0wheel=C2=A0=C2=A0=C2=A04.0M 29 = Sep 08:19 f Ah, I see, nice demonstration! But wouldn=C2=B4t TRIM take of that though (if that=C2=B4s available)? /K >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Fri Sep 29 06:54:32 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A7D71E252D2 for ; Fri, 29 Sep 2017 06:54:32 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wr0-x22a.google.com (mail-wr0-x22a.google.com [IPv6:2a00:1450:400c:c0c::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 383C980914 for ; Fri, 29 Sep 2017 06:54:32 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wr0-x22a.google.com with SMTP id y95so706587wrb.4 for ; Thu, 28 Sep 2017 23:54:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=+oc7Kwi793UQ5uGnppwnd1inFOUXkKhTpJXopADRNvs=; b=tJYElwGbCH6+qLXEa0/jkv3OkDGeC8YM2/UW+k7aCU/hrGlcoa5KR07nNOfXw2DSAV k1DAzUWSqjlF1KU8rJt8rfbxtW0ldCGMAsj+dYYpw0Gc92xyhpq3BXVE7plZnMdyCNqq znqknlogVGQFZ61QZ4yhxwhlDCMuHsI5MLyU/uCV4oHyULy3LnDbCDa+6ju37eczq9Bc POO72FyZtjn6dOanotbjIMXuZsBP1+b94Xtw5qYeQ98f44hr/5+UeY+blbEERNsRGk88 5Ko1xdJUd1Y5Zm22KtqELI0UDSaASGc48OVBhYcNYK8d1+iCUHcwylGSTpIYqXDOvt17 h04A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=+oc7Kwi793UQ5uGnppwnd1inFOUXkKhTpJXopADRNvs=; b=aldf5lKeQWl3EFAS5r4WlDdwT5J6efqG22oEeWVLZafGJIFGyc/VBs4wIZgU3dqYWA I8/MsQdrYg8bjt3OiHkSvHzfFjU/qbyDPtIOeo6/40eFRuWXsm1lqnXZtwZhdENLvqrL eI8f4+nr6C44N5NZhItv6K5+K6HzKmFqjq7dJnX0QV4yZcgdDlTX8QhK8YJfE2wC+QPk Bm+evyH+Ur6in07cvSVqJSqMV9Z9zjVvq/5ml8RHa3Rzw8OyuiR9FFCLFZSAUrhCMVn2 8CU839329B1fyAJqiUzTAdjKeay1VDy8hr64TWNuYs1JUlZDkk/qnstLuw2ed1K3YKml mYsg== X-Gm-Message-State: AHPjjUjrH9MkjFwVEglmFL7I+QhK4BMa3+qlAo4TglsOd6hsczoU56HJ 7xKL3+ZPXfBwOCdZqkyY43o6npQ4 X-Google-Smtp-Source: AOwi7QDz9zRRPenfyiecCtGiq1XMLf3XUn4P3fdz+UoN+HIIkPY5eiq20pXWYJ8aXxhe4hoibzo1Yg== X-Received: by 10.223.155.194 with SMTP id e2mr7504534wrc.49.1506668069566; Thu, 28 Sep 2017 23:54:29 -0700 (PDT) Received: from bens-mac.home (LFbn-MAR-1-445-220.w2-15.abo.wanadoo.fr. [2.15.38.220]) by smtp.gmail.com with ESMTPSA id i136sm3186044wmd.23.2017.09.28.23.54.28 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 28 Sep 2017 23:54:28 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: How to create holes in files ? From: Ben RUBSON In-Reply-To: <1506666838.7204.39.camel@inparadise.se> Date: Fri, 29 Sep 2017 08:54:28 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <7F95ECD3-3284-4808-9182-AF3FE5970333@gmail.com> References: <12ed413d-00d1-4182-bdd5-0e5230d46f01@email.android.com> <4D304428-E4AF-45D4-AB2E-D4B73A7FEE87@gmail.com> <1506664468.7204.37.camel@inparadise.se> <454261D6-0061-4DB9-8184-BDD7061870FB@gmail.com> <1506666838.7204.39.camel@inparadise.se> To: Freebsd fs X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 06:54:32 -0000 > On 29 Sep 2017, at 08:33, Karli Sj=C3=B6berg = wrote: >=20 > On fre, 2017-09-29 at 08:26 +0200, Ben RUBSON wrote: >>>=20 >>> On 29 Sep 2017, at 07:54, Karli Sj=C3=B6berg >>> wrote: >>>=20 >>> On tor, 2017-09-28 at 22:16 +0200, Ben RUBSON wrote: >>>>=20 >>>>>=20 >>>>>=20 >>>>> On 28 Sep 2017, at 20:48, Karli Sj=C3=B6berg >>>>> wrote: >>>>>=20 >>>>> Den 28 sep. 2017 6:47 em skrev Ben RUBSON >>>>> : >>>>>>=20 >>>>>>=20 >>>>>> On 28 Sep 2017, at 18:34, Bob Eager wrote: >>>>>>=20 >>>>>> On Thu, 28 Sep 2017 17:26:09 +0200 >>>>>> Fabian Keil wrote: >>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>> Ben RUBSON wrote: >>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> I'm trying to make holes in files in C. >>>>>>>> Goal is to deallocate huge files on ZFS while (randomly) >>>>>>>> reading >>>>>>>> them.=20 >>>>>>> My interpretation of the above is that you want to create >>>>>>> holes >>>>>>> without changing the file size and without affecting data >>>>>>> that >>>>>>> is located before or after the holes that you want to >>>>>>> create. >>>>>>>=20 >>>>>>> Otherwise you could simply "deallocate" the content with >>>>>>> truncate(1). >>>>>> If he doesn't mind copying the files, dd(1) will do the job. >>>>>> However, I >>>>>> expect that doesn't meet his criteria. >>>>> Thank you Bob for your suggestion. >>>>> You're right goal is to avoid copying data : free space would >>>>> not >>>>> necessarily >>>>> allow this, and as the files I'm working on are some hundreds >>>>> of GB >>>>> in size, >>>>> it would really be a counterproductive long stressing storage >>>>> operation. >>>>>=20 >>>>> Well, correct me if I'm wrong, but wouldn't dd with "seek" >>>>> mitigate >>>>> the issue of writing out all of the data from beginning to end. >>>>> If >>>>> you seek from the beginning of the file to the point you want >>>>> to >>>>> start writing from and use bs to specify how large of a "hole" >>>>> you >>>>> want, the operation wouldn't take long at all. You would, in my >>>>> opinion, achieve exactly what you want, to "create holes in >>>>> files". >>>>> Am I wrong? >>>> I would have liked to do this in C. >>> Yepp, I understand that, I was just bringing up the idea. But hey, >>> if >>> dd can do it, just look in the source of how it does it? >> Of course yes, you're right :) >>=20 >>>=20 >>>>=20 >>>> And the holes I need to create are not necessarily at the >>>> beginning >>>> of a file. >>> No, you _do not_ have to create holes in the beginning of a file, >>> you >>> can do it anywhere in the file you like. E.g. it=C2=B4s the trick I = use >>> to >>> "clean" hardrives to make them look empty, by only erasing the >>> first >>> and last 10 MB of the drives, a whole JBOD only takes seconds to >>> clean. >>> Say you have a 10 MB large file and you want to make a 1 MB large >>> hole >>> somewhere in the middle of it, you do it like this: >>>=20 >>> # dd if=3D/dev/zero of=3D/foo/bar.bin bs=3D1M seek=3D7 >>>=20 >>> It=C2=B4ll make the file look like this inside (hope the ASCII gods = are >>> with >>> me): >>> ____________ >>>>=20 >>>> _______|_|__| >> Unfortunately here storage blocks will be set/written to 0, >> they will not be freed/recovered, a file hole will not be created. >> Below are some commands which demonstrate this. >> The last 3/4 commands show a _newly_ created file with a hole. >>=20 >> # zfs userspace home >> POSIX User root 36.5K none >>=20 >> # dd if=3D/dev/random of=3Df bs=3D1M count=3D4 >> 4+0 records out >>=20 >> # zfs userspace home >> POSIX User root 4.04M none >>=20 >> # dd if=3D/dev/zero of=3Df bs=3D1M count=3D2 seek=3D2 >> 2+0 records out >>=20 >> # zfs userspace home >> POSIX User root 4.04M none >>=20 >> # rm f >>=20 >> # dd if=3D/dev/zero of=3Df bs=3D1M count=3D2 seek=3D2 >> 2+0 records out >>=20 >> # zfs userspace home >> POSIX User root 2.04M none >>=20 >> # ls -lh f >> -rw------- 1 root wheel 4.0M 29 Sep 08:19 f >=20 > Ah, I see, nice demonstration! But wouldn=C2=B4t TRIM take of that = though > (if that=C2=B4s available)? Good question ! I'm however not sure TRIM would notify ZFS about space not consumed. In addition I'm mainly on non-SSD drives :-/ Ben From owner-freebsd-fs@freebsd.org Fri Sep 29 06:59:42 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA7BFE25454 for ; Fri, 29 Sep 2017 06:59:42 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from mail.inparadise.se (h-246-50.A444.priv.bahnhof.se [155.4.246.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 70A3F80A71 for ; Fri, 29 Sep 2017 06:59:41 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 195764B827; Fri, 29 Sep 2017 08:59:39 +0200 (CEST) Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id oXke98adK440; Fri, 29 Sep 2017 08:59:38 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id A3BA04BCAA; Fri, 29 Sep 2017 08:59:38 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.inparadise.se A3BA04BCAA X-Virus-Scanned: amavisd-new at inparadise.se Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id yW0PHW-_J_h1; Fri, 29 Sep 2017 08:59:38 +0200 (CEST) Received: from vpnpool190-35.anst.uu.se (vpnpool190-35.anst.uu.se [130.238.190.35]) by mail.inparadise.se (Postfix) with ESMTPSA id 89CF84B827; Fri, 29 Sep 2017 08:59:38 +0200 (CEST) Message-ID: <1506668378.7204.41.camel@inparadise.se> Subject: Re: How to create holes in files ? From: Karli =?ISO-8859-1?Q?Sj=F6berg?= Reply-To: karli@inparadise.se To: Ben RUBSON , Freebsd fs Date: Fri, 29 Sep 2017 08:59:38 +0200 In-Reply-To: <7F95ECD3-3284-4808-9182-AF3FE5970333@gmail.com> References: <12ed413d-00d1-4182-bdd5-0e5230d46f01@email.android.com> <4D304428-E4AF-45D4-AB2E-D4B73A7FEE87@gmail.com> <1506664468.7204.37.camel@inparadise.se> <454261D6-0061-4DB9-8184-BDD7061870FB@gmail.com> <1506666838.7204.39.camel@inparadise.se> <7F95ECD3-3284-4808-9182-AF3FE5970333@gmail.com> Organization: InParadise Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.18.5.2-0ubuntu3.2 Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 06:59:42 -0000 On fre, 2017-09-29 at 08:54 +0200, Ben RUBSON wrote: > >=20 > > On 29 Sep 2017, at 08:33, Karli Sj=C3=B6berg > > wrote: > >=20 > > On fre, 2017-09-29 at 08:26 +0200, Ben RUBSON wrote: > > >=20 > > > >=20 > > > >=20 > > > > On 29 Sep 2017, at 07:54, Karli Sj=C3=B6berg > > > > wrote: > > > >=20 > > > > On tor, 2017-09-28 at 22:16 +0200, Ben RUBSON wrote: > > > > >=20 > > > > >=20 > > > > > >=20 > > > > > >=20 > > > > > >=20 > > > > > > On 28 Sep 2017, at 20:48, Karli Sj=C3=B6berg > > > > > e> > > > > > > wrote: > > > > > >=20 > > > > > > Den 28 sep. 2017 6:47 em skrev Ben RUBSON > > > > > .com > > > > > > >=20 > > > > > > > : > > > > > > >=20 > > > > > > >=20 > > > > > > > On 28 Sep 2017, at 18:34, Bob Eager wrote: > > > > > > >=20 > > > > > > > On Thu, 28 Sep 2017 17:26:09 +0200 > > > > > > > Fabian Keil wrote: > > > > > > >=20 > > > > > > > >=20 > > > > > > > >=20 > > > > > > > >=20 > > > > > > > > Ben RUBSON wrote: > > > > > > > >=20 > > > > > > > > >=20 > > > > > > > > >=20 > > > > > > > > >=20 > > > > > > > > > I'm trying to make holes in files in C. > > > > > > > > > Goal is to deallocate huge files on ZFS while > > > > > > > > > (randomly) > > > > > > > > > reading > > > > > > > > > them.=C2=A0 > > > > > > > > My interpretation of the above is that you want to > > > > > > > > create > > > > > > > > holes > > > > > > > > without changing the file size and without affecting > > > > > > > > data > > > > > > > > that > > > > > > > > is located before or after the holes that you want to > > > > > > > > create. > > > > > > > >=20 > > > > > > > > Otherwise you could simply "deallocate" the content > > > > > > > > with > > > > > > > > truncate(1). > > > > > > > If he doesn't mind copying the files, dd(1) will do the > > > > > > > job. > > > > > > > However, I > > > > > > > expect that doesn't meet his criteria. > > > > > > Thank you Bob for your suggestion. > > > > > > You're right goal is to avoid copying data : free space > > > > > > would > > > > > > not > > > > > > necessarily > > > > > > allow this, and as the files I'm working on are some > > > > > > hundreds > > > > > > of GB > > > > > > in size, > > > > > > it would really be a counterproductive long stressing > > > > > > storage > > > > > > operation. > > > > > >=20 > > > > > > Well, correct me if I'm wrong, but wouldn't dd with "seek" > > > > > > mitigate > > > > > > the issue of writing out all of the data from beginning to > > > > > > end. > > > > > > If > > > > > > you seek from the beginning of the file to the point you > > > > > > want > > > > > > to > > > > > > start writing from and use bs to specify how large of a > > > > > > "hole" > > > > > > you > > > > > > want, the operation wouldn't take long at all. You would, > > > > > > in my > > > > > > opinion, achieve exactly what you want, to "create holes in > > > > > > files". > > > > > > Am I wrong? > > > > > I would have liked to do this in C. > > > > Yepp, I understand that, I was just bringing up the idea. But > > > > hey, > > > > if > > > > dd can do it, just look in the source of how it does it? > > > Of course yes, you're right :) > > >=20 > > > >=20 > > > >=20 > > > > >=20 > > > > >=20 > > > > > And the holes I need to create are not necessarily at the > > > > > beginning > > > > > of a file. > > > > No, you _do not_ have to create holes in the beginning of a > > > > file, > > > > you > > > > can do it anywhere in the file you like. E.g. it=C2=B4s the trick= I > > > > use > > > > to > > > > "clean" hardrives to make them look empty, by only erasing the > > > > first > > > > and last 10 MB of the drives, a whole JBOD only takes seconds > > > > to > > > > clean. > > > > Say you have a 10 MB large file and you want to make a 1 MB > > > > large > > > > hole > > > > somewhere in the middle of it, you do it like this: > > > >=20 > > > > # dd if=3D/dev/zero of=3D/foo/bar.bin bs=3D1M seek=3D7 > > > >=20 > > > > It=C2=B4ll make the file look like this inside (hope the ASCII go= ds > > > > are > > > > with > > > > me): > > > > =C2=A0____________ > > > > >=20 > > > > >=20 > > > > > _______|_|__| > > > Unfortunately here storage blocks will be set/written to 0, > > > they will not be freed/recovered, a file hole will not be > > > created. > > > Below are some commands which demonstrate this. > > > The last 3/4 commands show a _newly_ created file with a hole. > > >=20 > > > # zfs userspace home > > > POSIX User=C2=A0=C2=A0root=C2=A0=C2=A036.5K=C2=A0=C2=A0=C2=A0none > > >=20 > > > # dd if=3D/dev/random of=3Df bs=3D1M count=3D4 > > > 4+0 records out > > >=20 > > > # zfs userspace home > > > POSIX User=C2=A0=C2=A0root=C2=A0=C2=A04.04M=C2=A0=C2=A0=C2=A0none > > >=20 > > > # dd if=3D/dev/zero of=3Df bs=3D1M count=3D2 seek=3D2 > > > 2+0 records out > > >=20 > > > # zfs userspace home > > > POSIX User=C2=A0=C2=A0root=C2=A0=C2=A04.04M=C2=A0=C2=A0=C2=A0none > > >=20 > > > # rm f > > >=20 > > > # dd if=3D/dev/zero of=3Df bs=3D1M count=3D2 seek=3D2 > > > 2+0 records out > > >=20 > > > # zfs userspace home > > > POSIX User=C2=A0=C2=A0root=C2=A0=C2=A02.04M=C2=A0=C2=A0=C2=A0none > > >=20 > > > # ls -lh f > > > -rw-------=C2=A0=C2=A01 root=C2=A0=C2=A0wheel=C2=A0=C2=A0=C2=A04.0M= 29 Sep 08:19 f > > Ah, I see, nice demonstration! But wouldn=C2=B4t TRIM take of that > > though > > (if that=C2=B4s available)? > Good question ! I'm however not sure TRIM would notify ZFS about > space not consumed. In addition I'm mainly on non-SSD drives :-/ OK, yeah it was more of a general question, not concerning your specific setup. Maybe someone else has an idea? Anyhow, I hope you get a better answer to your specific question. /K >=20 > Ben >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Fri Sep 29 07:08:27 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D71E6E257C0 for ; Fri, 29 Sep 2017 07:08:27 +0000 (UTC) (envelope-from rde@tavi.co.uk) Received: from kipling.tavi.co.uk (kipling.tavi.co.uk [81.187.145.130]) by mx1.freebsd.org (Postfix) with ESMTP id 91A7980EB0 for ; Fri, 29 Sep 2017 07:08:27 +0000 (UTC) (envelope-from rde@tavi.co.uk) Received: from kipling.tavi.co.uk (localhost [127.0.0.1]) by kipling.tavi.co.uk (Postfix) with ESMTP id 38AFB892C0 for ; Fri, 29 Sep 2017 08:08:24 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=tavi.co.uk; h=date:from:to :subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=selector1; bh=oDQ4MnY rYbHcuxSpPUxsF4RFdr8=; b=T4Bi/s9ERtky6/hxtr+SzFe+9l3kCbiCeO5IONt nG4Ff8TfKuZ0fZ7AiWmtFs/Oo37gJE1FfrHefj7YA6toD5b2zbankrLVodbdgpSi bxStMbE7ttGDU3FLgCdgLIUCcJIhZ1WG+Bsi9UxwppHoV2n/jX/NmyZx9AzTvsb9 OD4M= Received: from raksha.tavi.co.uk (raksha.tavi.co.uk [81.187.145.139]) (Authenticated sender: rde@tavi.co.uk) by kipling.tavi.co.uk (Postfix) with ESMTPA id F3DF2892BA for ; Fri, 29 Sep 2017 08:08:23 +0100 (BST) Date: Fri, 29 Sep 2017 08:08:23 +0100 From: Bob Eager To: freebsd-fs@freebsd.org Subject: Re: How to create holes in files ? Message-ID: <20170929080823.175d6abc@raksha.tavi.co.uk> In-Reply-To: <454261D6-0061-4DB9-8184-BDD7061870FB@gmail.com> References: <12ed413d-00d1-4182-bdd5-0e5230d46f01@email.android.com> <4D304428-E4AF-45D4-AB2E-D4B73A7FEE87@gmail.com> <1506664468.7204.37.camel@inparadise.se> <454261D6-0061-4DB9-8184-BDD7061870FB@gmail.com> X-Mailer: Claws Mail 3.15.0 (GTK+ 2.24.31; i386-portbld-freebsd11.0) Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAMFBMVEUwXjFLc0vD0cS7y7zw9PDZ4tkWSRaVrZZ+m39qi2tXfVj////7+/utwK4IPggAOAAJUUA7AAABKklEQVQ4jWPYjQMwDFYJp0NKEKCNJmEf9h8CsimXiL2e33s3/e7F7K2Cs3f3dCMkQkMKj4YuCY3K3iR+e7fMaiSjvkX0/5cFGrWpe2uLzOpaExUVqMS/8PX/Re5ey960OLBTZpFA8+IlSBKPQ92zNyUUBsosN58uIY0k8f+/ONCoYytkVuhWzVwNkYiYbqk5M3NmOVBi41YZ8RsGF7shEtFb5KJ3r969CyixM7OTPeFUxG2IxLO8/9/SvqXlc+/x3h295YzLlj2nIRJQj//nRvc5TEIal8RsXBLVuCQwIgoq/u80DomP6HEOk/iOS+IJLonZOCT+ReOQ+Lkbh0QKLonbOCR+7MYhsRqHBJrVcIl/1TgklqKLQyQ+tGKIgyQOqXpjig94diZRAgAXmDX6jyWafAAAAABJRU5ErkJggg====== MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 07:08:28 -0000 On Fri, 29 Sep 2017 08:26:13 +0200 Ben RUBSON wrote: > Unfortunately here storage blocks will be set/written to 0, > they will not be freed/recovered, a file hole will not be created. You need to add: conv=sparse to the dd command to turn zero blocks into unallocated blocks. From owner-freebsd-fs@freebsd.org Fri Sep 29 07:21:52 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EC9B2E25A77 for ; Fri, 29 Sep 2017 07:21:52 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (ns1.bitblocks.com [173.228.5.8]) by mx1.freebsd.org (Postfix) with ESMTP id D765D81390 for ; Fri, 29 Sep 2017 07:21:52 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 68594156E525; Fri, 29 Sep 2017 00:21:16 -0700 (PDT) From: Bakul Shah To: Ben RUBSON cc: Freebsd fs Subject: Re: How to create holes in files ? In-reply-to: Your message of "Thu, 28 Sep 2017 16:39:28 +0200." <44E32501-4213-4A09-992C-92DB4EF33C0C@gmail.com> References: <44E32501-4213-4A09-992C-92DB4EF33C0C@gmail.com> Comments: In-reply-to Ben RUBSON message dated "Thu, 28 Sep 2017 16:39:28 +0200." MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <46107.1506669676.1@bitblocks.com> Date: Fri, 29 Sep 2017 00:21:16 -0700 Message-Id: <20170929072131.68594156E525@mail.bitblocks.com> X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 07:21:53 -0000 On Thu, 28 Sep 2017 16:39:28 +0200 Ben RUBSON wrote: Ben RUBSON writes: > Hi, > > I'm trying to make holes in files in C. > Goal is to deallocate huge files on ZFS while (randomly) reading them. > > The only thing I found is on Linux, fallocate(2) with FALLOC_FL_PUNCH_HOLE. > > What about FreeBSD ? You can copy such a file using rsync with -S or --sparse option or dd with conv=sparse option. I don't know if these programs use seek with SEEK_HOLE and SEEK_DATA to avoid reading existing holes in the source file as zeroes. Someone mentioned TRIM support but AFAIK this is only used (if available) when a file is freed. Replacing a zero block write with lseek will impact every write (you have to check if the block is all zeroes) so not worth doing in the general case. From owner-freebsd-fs@freebsd.org Fri Sep 29 07:32:02 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A177E25C98 for ; Fri, 29 Sep 2017 07:32:02 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wr0-x22c.google.com (mail-wr0-x22c.google.com [IPv6:2a00:1450:400c:c0c::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EF9C48171D for ; Fri, 29 Sep 2017 07:32:01 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wr0-x22c.google.com with SMTP id r74so833252wrb.13 for ; Fri, 29 Sep 2017 00:32:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=kVHsxbixiwVRfGiSXHFJmkwEeJDS6xLdczq/oyvqjts=; b=n4unpW+OODTJHpxYavZSOJzhA5wTNQPIm9fvOMDrLkLHppHn2LQ6iwak0YMM+XcBrn 8rxFp4Jtzig/cPDB/F7i7pRX8P/RgE9zTIA6LI+K2PluPMR2n3Is4yHZ6HEBPxmiEk3A 3dA8+HJPsXjLvc5PQMzZWJXjSRVGw0BZyoFNQiMJM3j7W6FZ9Elld8AxBDvmLpsAPx0j iguUHcEx6vJly+FgXTPLnL2uxudi3evjMujAQbNJK0Q3t/dgjiyA8vV+QQNezHRrVpjj dsmiXHhnvwzaDrI2viPpgDF0Un9gurLmR2I8sjtuBQConUXhh5gSZCThuchzaHgx7Vi3 JnGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=kVHsxbixiwVRfGiSXHFJmkwEeJDS6xLdczq/oyvqjts=; b=Zx8BcgHXLh68IOL8C7CQIKgL3rZeC9C3QMs8eIwgKq5r2AdLlDtvMQu3ttJeskUQip Kd7nt9LaUTLZrMHxRp9VNNvY8HYbnTUsazANNyfQfb+3UiV8iLNlO661taI8JShq8H5D /AwIo4TKzwS/vd/2OdLE0y75VoZ/ZRKQICekIWQVEGufcZCIrEMTEioz7Zzjw0c+poqa CcYKmNCRlUmLpOKCIH7fwJUYimT4yyygRujc4QG2tyFyw4Q9b1n9XNsBBNzZoQX/cQ8Z WpxD3cSaBLVskuThN7xpRRBv7A1vAF2gtAjXKJtOqLZkp6icaMpDzgjNOcjt37B2wB5j kwHg== X-Gm-Message-State: AHPjjUgnbaf5hnK2hje2cWReJ16wc7ZI8uaaV8+hU/2ntisTntMKI2G9 HsI/D3o1mVmGJJAP/+jKxXZU3B7y X-Google-Smtp-Source: AOwi7QDTvEDb+cb1UcVh6QG9PTDgRwfxHK85q+pOcOfuCKp/K9epcgkQl26HlIIRY8aCJh8fnFdt6Q== X-Received: by 10.223.174.227 with SMTP id y90mr6609648wrc.205.1506670320105; Fri, 29 Sep 2017 00:32:00 -0700 (PDT) Received: from bens-mac.home (LFbn-MAR-1-445-220.w2-15.abo.wanadoo.fr. [2.15.38.220]) by smtp.gmail.com with ESMTPSA id n9sm4038925wmd.12.2017.09.29.00.31.59 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 29 Sep 2017 00:31:59 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: How to create holes in files ? From: Ben RUBSON In-Reply-To: <20170929080823.175d6abc@raksha.tavi.co.uk> Date: Fri, 29 Sep 2017 09:31:58 +0200 Content-Transfer-Encoding: 7bit Message-Id: <7E8BB109-C72C-4B5D-A2EF-630C6B34D8E6@gmail.com> References: <12ed413d-00d1-4182-bdd5-0e5230d46f01@email.android.com> <4D304428-E4AF-45D4-AB2E-D4B73A7FEE87@gmail.com> <1506664468.7204.37.camel@inparadise.se> <454261D6-0061-4DB9-8184-BDD7061870FB@gmail.com> <20170929080823.175d6abc@raksha.tavi.co.uk> To: Freebsd fs X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 07:32:02 -0000 > On 29 Sep 2017, at 09:08, Bob Eager wrote: > > On Fri, 29 Sep 2017 08:26:13 +0200 > Ben RUBSON wrote: > >> Unfortunately here storage blocks will be set/written to 0, >> they will not be freed/recovered, a file hole will not be created. > > You need to add: conv=sparse to the dd command to turn zero blocks > into unallocated blocks. You're right Bob it works : # dd if=/dev/random of=f bs=1M count=4 4+0 records out # zfs userspace home POSIX User root 4.04M none # dd if=/dev/zero of=f bs=1M count=2 seek=2 conv=sparse 2+0 records out # zfs userspace home POSIX User root 2.04M none # ls -lh f -rw------- 1 root wheel 4.0M 29 Sep 09:25 f Unfortunately, using "conv=notrunc,sparse" to make a hole in the middle of a file does not seem to work, file is truncated : # rm f # dd if=/dev/random of=f bs=1M count=4 4+0 records out # dd if=/dev/zero of=f bs=1M count=1 seek=2 conv=notrunc,sparse 1+0 records out # ls -lh f -rw------- 1 root wheel 3.0M 29 Sep 09:26 f # zfs userspace home POSIX User root 3.04M none Anyway thank you for this tip, this is something to analyse ! Ben From owner-freebsd-fs@freebsd.org Fri Sep 29 07:33:25 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1D8B6E25E73 for ; Fri, 29 Sep 2017 07:33:25 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wr0-x241.google.com (mail-wr0-x241.google.com [IPv6:2a00:1450:400c:c0c::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A555981936 for ; Fri, 29 Sep 2017 07:33:24 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wr0-x241.google.com with SMTP id n64so534545wrb.2 for ; Fri, 29 Sep 2017 00:33:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=mjjF+ecIyrIXUrgdysWIULQDsIvkxntlzcFfqRkX9dc=; b=OPMKZxavYBruT4/PXgpB1VpqXNZOLgr9VWczzIH1pMGX/riLR4G3ag9ZLbo9PsvI/2 QDYAjl+YdNPscjS2ccKSatAiNqoG5v1jAv6JAVOhjqKHcEk7Hz6773D4ImT1btKR1N55 qm/QOIc1Ur5geeZCHEk4grihcLxsMDRv3ZXVvWCXgkRemW79tr4jDu7zMhpcOiQZjvff Qocqr6K4mT6UGZ9ZhwUHTUmRZkmhxU7ZJnhhS9/r1ButQGLioZqph50muak0OBGjn/if QK6eTSg3I4Yu4pQ4HpCxnFmkkIc9s/+vdClPGpxonLgv1F36NhQ3XQhTDZhel9+JsiX4 Dhfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=mjjF+ecIyrIXUrgdysWIULQDsIvkxntlzcFfqRkX9dc=; b=tSsbdHhxYjkYbBRzLqNsVXBUa4dCe9H2tvSBu8ScXAXjnfkweSkq+y6IRZQTtVpcE4 HxyhyKQxSYNlX0pHZNkw3ocwwAWmNfULV3ot7Bravsa5uJF9Q4MivWaX80412cR4NcEK whSX730SMrg+r7FBz4/QRMz+HB0pgG+qgYwnxklGeVaNuxWuOCK8FTX6juJoD7dfManF pxZLaRR3S380aYy7B+W0hX59JiNlFW02BhsLefLvsDmIW9LF8MV3qvm9LFObgPrWDHQh 6mAsgJAkVNWocH+zaU6MJS7OUEHT04Q1FCaGYuRrcsRwlpvOPogfnzD9mm+s5oCBLvbJ EjUg== X-Gm-Message-State: AHPjjUh6aPDKINIIyrM/kv/fgWywvU9IwJUe3e/cpuZFJ2kDSZm29b8Z CaixGyU0vo4TEhawVEWIZ2VThH7F X-Google-Smtp-Source: AOwi7QCXIQlIlLf5NFtl3Y1+7C2nKul+FqwjOvhmvoVTVHi0k/m+5RZ5hoSrhZcrv72zwfaD4ob2cQ== X-Received: by 10.223.135.53 with SMTP id a50mr7615150wra.229.1506670402930; Fri, 29 Sep 2017 00:33:22 -0700 (PDT) Received: from bens-mac.home (LFbn-MAR-1-445-220.w2-15.abo.wanadoo.fr. [2.15.38.220]) by smtp.gmail.com with ESMTPSA id m138sm3779538wmd.29.2017.09.29.00.33.22 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 29 Sep 2017 00:33:22 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: How to create holes in files ? From: Ben RUBSON In-Reply-To: <20170929072131.68594156E525@mail.bitblocks.com> Date: Fri, 29 Sep 2017 09:33:21 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <713DA316-D2DA-4F28-9879-C4278E122F5E@gmail.com> References: <44E32501-4213-4A09-992C-92DB4EF33C0C@gmail.com> <20170929072131.68594156E525@mail.bitblocks.com> To: Freebsd fs X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 07:33:25 -0000 > On 29 Sep 2017, at 09:21, Bakul Shah wrote: >=20 > On Thu, 28 Sep 2017 16:39:28 +0200 Ben RUBSON = wrote: > Ben RUBSON writes: >> Hi, >>=20 >> I'm trying to make holes in files in C. >> Goal is to deallocate huge files on ZFS while (randomly) reading = them. >>=20 >> The only thing I found is on Linux, fallocate(2) with = FALLOC_FL_PUNCH_HOLE. >>=20 >> What about FreeBSD ? >=20 > You can copy such a file using rsync with -S or --sparse > option or dd with conv=3Dsparse option. Unfortunately Bakul I would have liked to avoid copying data. From owner-freebsd-fs@freebsd.org Fri Sep 29 18:22:47 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D0656E327C8 for ; Fri, 29 Sep 2017 18:22:47 +0000 (UTC) (envelope-from citrin@citrin.ru) Received: from hz.citrin.ru (hz.citrin.ru [IPv6:2a01:4f8:d16:10c3::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9D0D370BDB for ; Fri, 29 Sep 2017 18:22:47 +0000 (UTC) (envelope-from citrin@citrin.ru) Received: from x220.lan (unknown [IPv6:2601:18a:c680:9888:9477:d48c:3bae:c7eb]) by hz.citrin.ru (Postfix) with ESMTPSA id 1FB212C1653 for ; Fri, 29 Sep 2017 18:22:45 +0000 (UTC) To: freebsd-fs@freebsd.org From: Anton Yuzhaninov Subject: Panic: vm_fault: fault on nofault entry Message-ID: <35aef1ac-a1dd-4526-46f8-599b4fb3b0f4@citrin.ru> Date: Fri, 29 Sep 2017 14:22:43 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrin.ru; s=s0; t=1506709365; bh=6BtTk5+IwFGJepWBaJGLWNaQBbpnTPf456tWAoL3SNQ=; h=To:From:Subject:Message-ID:Date:MIME-Version:Content-Type:Content-Transfer-Encoding; b=sUajXfrx+41cKecauDPMglD4OKY//96Mc7F+k9SzQDq9Rg5/w2EwhYoBw5+YVPTdw0wd64y1snTZVYvHkHirgyXnxWk/ip6FamQp3h/TsWKurVYq9m8DihbIKFGr67K/nXHQ1+xc1GMx24+yU5ULlkJtK73LglIykLCYuiSrEuY= X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 18:22:47 -0000 Hi All, I have panics when I try to use nginx on my notebook. When sendfile is enabled it is easy to trigger panic, after disabling sendfile I've got no panics so far and I think that panic is triggered by using sendfile. Root filesystem - UFS on GELI whith 8k sectorsize. GELI with sectorsize large than page size is non-standard configuration, and I've got problem because of this in the past: https://lists.freebsd.org/pipermail/freebsd-fs/2016-October/023957.html but it was fixed in: https://svnweb.freebsd.org/changeset/base/307626 Probably I've not used sendfile on this filesystem since then. Panic string is: "vm_fault: fault on nofault entry, addr: fffffe0220ab6000" 0xfffffe0220ab6000 is the address for bzero in vnode_pager_generic_getpages_done (kgdb) f 17 #17 0xffffffff807ee290 in vnode_pager_generic_getpages_done (bp=0xfffffe01ea816490) at /spool/src/sys/vm/vnode_pager.c:1049 1049 bzero(bp->b_data + bp->b_bcount, (kgdb) l 1044 if (!buf_mapped(bp)) { 1045 bp->b_data = bp->b_kvabase; 1046 pmap_qenter((vm_offset_t)bp->b_data, bp->b_pages, 1047 bp->b_npages); 1048 } 1049 bzero(bp->b_data + bp->b_bcount, 1050 PAGE_SIZE * bp->b_npages - bp->b_bcount); 1051 } 1052 if (buf_mapped(bp)) { 1053 pmap_qremove((vm_offset_t)bp->b_data, bp->b_npages); (kgdb) p bp->b_data + bp->b_bcount $4 = 0xfffffe0220ab6000
bp->b_data is a pointer to data from a file served using sendfile. and bp->b_bcount is 8192 (GELI sector size is also 8192) Full backtrace: http://termbin.com/dqpl OS version: 11.1-STABLE #0 r322995 -- Best Regards, Anton Yuzhaninov From owner-freebsd-fs@freebsd.org Sat Sep 30 08:18:50 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EFC19E157C5 for ; Sat, 30 Sep 2017 08:18:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 99628661C4 for ; Sat, 30 Sep 2017 08:18:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v8U8IaRW046583 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 30 Sep 2017 11:18:36 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v8U8IaRW046583 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v8U8IZ48046582; Sat, 30 Sep 2017 11:18:35 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 30 Sep 2017 11:18:35 +0300 From: Konstantin Belousov To: Anton Yuzhaninov Cc: freebsd-fs@freebsd.org Subject: Re: Panic: vm_fault: fault on nofault entry Message-ID: <20170930081835.GH95911@kib.kiev.ua> References: <35aef1ac-a1dd-4526-46f8-599b4fb3b0f4@citrin.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <35aef1ac-a1dd-4526-46f8-599b4fb3b0f4@citrin.ru> User-Agent: Mutt/1.9.1 (2017-09-22) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2017 08:18:51 -0000 On Fri, Sep 29, 2017 at 02:22:43PM -0400, Anton Yuzhaninov wrote: > Hi All, > > I have panics when I try to use nginx on my notebook. When sendfile is > enabled it is easy to trigger panic, after disabling sendfile I've got > no panics so far and I think that panic is triggered by using sendfile. > > Root filesystem - UFS on GELI whith 8k sectorsize. > > GELI with sectorsize large than page size is non-standard configuration, > and I've got problem because of this in the past: > https://lists.freebsd.org/pipermail/freebsd-fs/2016-October/023957.html > > but it was fixed in: > https://svnweb.freebsd.org/changeset/base/307626 Well, it was fixed by replacing the code path for normal page in by the less buggy pager. The sendfile path was not touched, it still uses the generic pager with all its bugs sitting in place. There is no way around it, you cannot use block size > page size for sendfile, and there are more issues, e.g. with the simultaneous file truncation. > > Probably I've not used sendfile on this filesystem since then. > > Panic string is: > "vm_fault: fault on nofault entry, addr: fffffe0220ab6000" > > 0xfffffe0220ab6000 is the address for bzero in > vnode_pager_generic_getpages_done > > (kgdb) f 17 > #17 0xffffffff807ee290 in vnode_pager_generic_getpages_done > (bp=0xfffffe01ea816490) at /spool/src/sys/vm/vnode_pager.c:1049 > 1049 bzero(bp->b_data + bp->b_bcount, > (kgdb) l > 1044 if (!buf_mapped(bp)) { > 1045 bp->b_data = bp->b_kvabase; > 1046 pmap_qenter((vm_offset_t)bp->b_data, bp->b_pages, > 1047 bp->b_npages); > 1048 } > 1049 bzero(bp->b_data + bp->b_bcount, > 1050 PAGE_SIZE * bp->b_npages - bp->b_bcount); > 1051 } > 1052 if (buf_mapped(bp)) { > 1053 pmap_qremove((vm_offset_t)bp->b_data, bp->b_npages); > (kgdb) p bp->b_data + bp->b_bcount > $4 = 0xfffffe0220ab6000
> > bp->b_data is a pointer to data from a file served using sendfile. > and bp->b_bcount is 8192 (GELI sector size is also 8192) > > Full backtrace: http://termbin.com/dqpl > OS version: 11.1-STABLE #0 r322995 > > -- > Best Regards, > Anton Yuzhaninov > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Sat Sep 30 18:53:18 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B6500E2CEAB for ; Sat, 30 Sep 2017 18:53:18 +0000 (UTC) (envelope-from citrin+bsd@citrin.ru) Received: from hz.citrin.ru (hz.citrin.ru [IPv6:2a01:4f8:d16:10c3::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7AD3E7C514 for ; Sat, 30 Sep 2017 18:53:18 +0000 (UTC) (envelope-from citrin+bsd@citrin.ru) Received: from x220.lan (unknown [IPv6:2601:18a:c680:9888:9477:d48c:3bae:c7eb]) by hz.citrin.ru (Postfix) with ESMTPSA id 6919F2C1653 for ; Sat, 30 Sep 2017 18:53:15 +0000 (UTC) Subject: Re: Panic: vm_fault: fault on nofault entry To: freebsd-fs@freebsd.org References: <35aef1ac-a1dd-4526-46f8-599b4fb3b0f4@citrin.ru> <20170930081835.GH95911@kib.kiev.ua> From: Anton Yuzhaninov Message-ID: <1eaf07ba-1928-a162-16fe-1908d350ed46@citrin.ru> Date: Sat, 30 Sep 2017 14:53:13 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20170930081835.GH95911@kib.kiev.ua> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrin.ru; s=s0; t=1506797595; bh=JIB+uRqLV+ifCLXWFv1hgez08wA2DXVJExRyh6jb8YE=; h=Subject:To:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=MtuISQlEG5r04d3U+iuGtj5oq9TQPU55jGM/VicZcSU+fC3pVNAH+1TmPS48ZUTZoFLyva50K/CFsLNuh/OQny0Qf0W7YNplznkzfRylMvZgnJZEmmpmod8fIVQnd24wKB/lzSt9Khig42MmNGZbtRPZhL22nxNc1EnwbyP8Tso= X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2017 18:53:18 -0000 On 09/30/17 04:18, Konstantin Belousov wrote: >> I have panics when I try to use nginx on my notebook. When sendfile is >> enabled it is easy to trigger panic, after disabling sendfile I've got >> no panics so far and I think that panic is triggered by using sendfile. >> >> Root filesystem - UFS on GELI whith 8k sectorsize. >> >> GELI with sectorsize large than page size is non-standard configuration, >> and I've got problem because of this in the past: >> https://lists.freebsd.org/pipermail/freebsd-fs/2016-October/023957.html >> >> but it was fixed in: >> https://svnweb.freebsd.org/changeset/base/307626 > Well, it was fixed by replacing the code path for normal page in by > the less buggy pager. The sendfile path was not touched, it still uses > the generic pager with all its bugs sitting in place. > > There is no way around it, you cannot use block size > page size for > sendfile, and there are more issues, e.g. with the simultaneous file > truncation. > Thanks for explaining. On a desktop I can easily avoid using sendfile(2). Hard part for me was to find out that panic was triggered by sendfile (and to something else).