From owner-freebsd-virtualization@FreeBSD.ORG Tue Sep 2 22:15:30 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 268D34CF for ; Tue, 2 Sep 2014 22:15:30 +0000 (UTC) Received: from mail-vc0-x22b.google.com (mail-vc0-x22b.google.com [IPv6:2607:f8b0:400c:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DB43C1C35 for ; Tue, 2 Sep 2014 22:15:29 +0000 (UTC) Received: by mail-vc0-f171.google.com with SMTP id id10so7883132vcb.30 for ; Tue, 02 Sep 2014 15:15:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=rGqKiJks2q0OrLhmLDFL88Cf+Uf3qFSoAm9+tuTzuvM=; b=AcoEM6Bv+CapAzdAPv/Q3aCTFln/STkYn4T+kt9hCLJk+VeB+imY20sF4xqy0L+U1A 3z8bUC7cMclWFhkHD18ibs1bX0NzPkF5qTWJUOFURnghK4zVxdnKOBTiGz9EsM7qbLCG RTjPKUvPATmhxfujxtxvsxa4E4PNIKWVCkXbUn+P/Kjc2KgNQ9vz/L/NiAq4nJJ1EC8a YhGgIcVCdPrleMN5PYST5MvYEBHF8mXLjrpScV7g1MvnUBtqAYSj74RuwyNSzOpknqTR w6FySzQwtbmp+XiWKRzHPQrWqNXo2Hkp2/2UAYruXWR9M4+Yt4nT4XSURp/vd7SHVmKv BLig== MIME-Version: 1.0 X-Received: by 10.221.6.201 with SMTP id ol9mr31929356vcb.2.1409696128939; Tue, 02 Sep 2014 15:15:28 -0700 (PDT) Received: by 10.221.46.133 with HTTP; Tue, 2 Sep 2014 15:15:28 -0700 (PDT) In-Reply-To: <20140902133950.GA19480@potato.growveg.org> References: <20140902133950.GA19480@potato.growveg.org> Date: Wed, 3 Sep 2014 00:15:28 +0200 Message-ID: Subject: Re: bhyve and zfs and speed From: Nikolay Denev To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Sep 2014 22:15:30 -0000 On Tue, Sep 2, 2014 at 3:39 PM, John wrote: > Hello list, > > I'm thinking of re-doing a host server from virtualbox to bhyve in an effort to > speed things up. The server is a Dell and the cpu is E5-2650L @ 1.80GHz > with 32 CPUs and 192GB ram. It runs freebsd 10-stable r267848 generic > kernel with ZFS throughout (10TB). Unfortunately there's no SSD. There will > probably be 10 at the most websites on this server as guests, all needing > MySQL. > > The problem I'm having is that with bhyve (and to a lesser extent, ZFS) > the relevant advice for what I need is kind of scattered. Basically, I > need to know: > > 0. any ballpark figures for zfs tunables? Right now, it's running > with defaults. top shows this, with one freebsd and one ubuntu guest, on vbox: > > Mem: 50M Active, 784M Inact, 78G Wired, 6132K Cache, 108G Free > ARC: 38G Total, 5778M MFU, 27G MRU, 16K Anon, 4540M Header, 1910M Other > Swap: 192G Total, 192G Free > > 1. does zfs need tuning for bhyve? > > 2. does bhyve need tuning for zfs? > > 3. do either need tuning for MySQL (or even apache) ? > > many thanks for advice (or pointers to documentation) > -- > John > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" Hi, I see you have 108G free memory. Is this due to a recent reboot and the ARC hasn't grown much yet, or you are limiting it? As for ZFS tuning for bhyve, I'm not aware of anything specific, but you should consider that values you want to put for the zvols for primarycache and secondarycache. (In your case only primarycache as you don't have L2ARC). Leaving this to the default might be faster but might result in double caching: Once in the ARC on the host system, and then on the buffer cache of the UFS guest. For MySQL and databases in general, as far as I can remember the general advice was to run them on zfs datasets with lower reclen, like 8k for example. Here you have zvols which default to 8k but the vtbd device reports 512b sectors, not sure if you try to trick it into using larger values here wouldn't help. Also different UFS block sizes for different zvol recordlens might be interesting test. --Nikolay