From owner-freebsd-current@freebsd.org Sun Apr 1 16:33:57 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E6AC3F6BBF2 for ; Sun, 1 Apr 2018 16:33:56 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22c.google.com (mail-io0-x22c.google.com [IPv6:2607:f8b0:4001:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8320C73881 for ; Sun, 1 Apr 2018 16:33:56 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22c.google.com with SMTP id d7so15655712ioc.11 for ; Sun, 01 Apr 2018 09:33:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=IEtnYDlLaO2bRDi87Wl8zM94lejg2GmXwvblf8vwD1Y=; b=IcB5iz68xI6PFUXoC2bQYjkdOlzZvm2LMrKdecC47PEgTe1hLtx+nAdvF+YQ6NqROd kbTbO9XAN9FyrvoZemK0txCDY5RJLB0Yu228G8K6zui+w2I8ua8Puga01zP3pWSSKhUU rMCHKm6n2dSy38KaNK4r/OFPwHE5q3Cp+xdoEzauTQH25fIKDpWG1KPZi1iRQa2jejIq QczEnDAeUdtggVUnWEc0oxYe4ylusZYcJdVT4sAoY2p2iEMDHyRMYxrb4aCBlfKZxdS8 s+BGQThdffvZZLTw3dWgFFAolXLTZIhfjc+oTH5anFwup0updPLUmqXrXNVvCezMY74+ pHDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=IEtnYDlLaO2bRDi87Wl8zM94lejg2GmXwvblf8vwD1Y=; b=VefLKAid+47NYXtgo54Nbk43udZXJARkEqvZoreg0gpARk7BCltThQbNDeeVzp2Dc8 LXs7ywPxjnzuvPXt7MgNUtDYQWqDgwaIGpJWlCwFlGQgwXswZVEvevrFvTh47BApr3bM XocxQBDUwyGCIxm4u/YKlPbAu+XaX1Cdy+m9aZ+Kg+DDR1odbRoQEiQed2I/FbktDWgk WuK0uqa4wOWmSmIrwRRKr4wB1X7aJbhkvceyATSVoR1DKeraVUfCGkh98LiKobt4AAv1 PWLOA0XmybxCurLbuGlTpigF1tuN0sl9iaUZQyCF3qljk3e1jHUwQ52vRJ4JS5ZuiodR lOBQ== X-Gm-Message-State: ALQs6tDf7d90Drl81jLWxQG5zNUdh1fCNyX6eSMM0yLNnwLcM0fs7/K6 tVr5vCwuAPBJslB81C+8YQUlDoq5H17TGfYNVhdIvQ== X-Google-Smtp-Source: AIpwx4+I9VigJ1Z6XcN14QnbFCSNLkJQvcwk8xhMSspc1lRrqs4MKUT+VbLg4xBNN3RjH1/1JXJTN9QQoUMiuU5/TWk= X-Received: by 10.107.162.146 with SMTP id l140mr5499256ioe.39.1522600435488; Sun, 01 Apr 2018 09:33:55 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.203.196 with HTTP; Sun, 1 Apr 2018 09:33:54 -0700 (PDT) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: References: From: Warner Losh Date: Sun, 1 Apr 2018 10:33:54 -0600 X-Google-Sender-Auth: iiJYRfyjdqkdg1KRTJfzG2R9Fio Message-ID: Subject: Re: Extremely low disk throughput under high compute load To: Stefan Esser Cc: FreeBSD Current Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Apr 2018 16:33:57 -0000 On Sun, Apr 1, 2018 at 9:18 AM, Stefan Esser wrote: > My i7-2600K based system with 24 GB RAM was in the midst of a buildworld > -j8 > (starting from a clean state) which caused a load average of 12 for more > than > 1 hour, when I decided to move a directory structure holding some 10 GB to > its > own ZFS file system. File sizes varied, but were mostly in the range 0f > 500KB. > > I had just thrown away /usr/obj, but /usr/src was cached in ARC and thus > there > was nearly no disk activity caused by the buildworld. > > The copying proceeded at a rate of at most 10 MB/s, but most of the time > less > than 100 KB/s were transferred. The "cp" process had a PRIO of 20 and thus > a > much better priority than the compute bound compiler processes, but it got > just 0.2% to 0.5% of 1 CPU core. Apparently, the copy process was scheduled > at such a low rate, that it only managed to issue a few controller writes > per > second. > > The system is healthy and does not show any problems or anomalies under > normal use (e.g., file copies are fast, without the high compute load). > > This was with SCHED_ULE on a -CURRENT without WITNESS or malloc debugging. > > Is this a regression in -CURRENT? > Does 'sync' push a lot of I/O to the disk? Is the effective throughput of CP tiny or large? It's tiny, if I read right, and the I/O is slow (as opposed to it all buffering in memory and being slow to drain own), right? Warner