From owner-freebsd-virtualization@freebsd.org Sat Jul 22 00:50:57 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31AB5C31D37 for ; Sat, 22 Jul 2017 00:50:57 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-wm0-x232.google.com (mail-wm0-x232.google.com [IPv6:2a00:1450:400c:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B90A263C82; Sat, 22 Jul 2017 00:50:56 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by mail-wm0-x232.google.com with SMTP id w126so23660548wme.0; Fri, 21 Jul 2017 17:50:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=+VZgIKdWSUgngl5pJjtNQhVontGunUJ2E1ds1wQP3JQ=; b=Mgel6c5bgIGjQssBKf2tCSPherHEV7qZCe6GDLIWT0mV81EELfeivAvV/DmOTkn9VD nlkyUBwDe4363TENz8HvReoPdjqDFNIRHQj1SmgBlsKPTYRESTntZNC33zRn6dM6o3j9 S7fn7oH+jPn/1n8GvOAvoeZUeLIe9PKq89liDKO/imTbOpDJANfABcG9MMj0IDMQwvK2 cEzJyAUQIQ9rfPUY8IvY1SGP8sUhNl/hMT2lQOfB+ZJhsn5Y4BwnBJ2RB6w2V91KlaBq KPC5KcmEitTCqRooo/gb0oJ0rwq0Dczw0ZxqXLI5N1y6aWG5KEM9uSK6dn5thzVCzBAi V6Dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=+VZgIKdWSUgngl5pJjtNQhVontGunUJ2E1ds1wQP3JQ=; b=cxyEIlmO7DSq7ZxWCg0ReDOjtjcuUSOoYaLTKKLIssjj7anI+WbhvwgaOF+WscXjCc aOLbPpu94LSYvu+/b5/kYhxrLfDtSZCGZqG11PQ2mLrXKs/1ZAsWuS/FW/vmU1i6MTKS LVLGIy4loltAtkLGBGTNhm3OZ12C0rQagiWgGUzHXzkb6jgVJmGm7r2woHPP/NPdgQju AEKhc8KvbW4bziPGiWB68SCcuj6jHWowGDpKs3Nf+0EPTaeEunvuD4gXnWrpAIl2EGK5 OclY/mx3qDJ9Lqz6ofeaZIILc6ols9wFeHuMULHTccC/8AUDaYmCO9VntmjZT69GGN1G WORw== X-Gm-Message-State: AIVw111/FGdoV7e+HaAGIrlF4Dk90dd+2VAwLWvPhfGD+1GKGhzIFSnT blvizM0DtsflAqJJ2fD5vrHIqqvwxvek X-Received: by 10.80.154.131 with SMTP id p3mr7285043edb.181.1500684655221; Fri, 21 Jul 2017 17:50:55 -0700 (PDT) MIME-Version: 1.0 Received: by 10.80.220.8 with HTTP; Fri, 21 Jul 2017 17:50:54 -0700 (PDT) In-Reply-To: <792ad5df-9e03-a332-c9a1-0b04ba69786a@freebsd.org> References: <792ad5df-9e03-a332-c9a1-0b04ba69786a@freebsd.org> From: Zaphod Beeblebrox Date: Fri, 21 Jul 2017 20:50:54 -0400 Message-ID: Subject: Re: Bhyve Broken: whose fault (AMD, FreeBSD, ZFS ...?) To: Peter Grehan Cc: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Jul 2017 00:50:57 -0000 remember that the quoted problem is spit out by the guest, not the host. That said, the 'top' line on the frozen bhyve was: 29380 root 22 20 0 1060M 928M kqread 5 218:32 399.30% bhyve ... indicating that the bhyve had almost all it's memory... and the system had also 500M free when I checked it. On Fri, Jul 21, 2017 at 7:03 PM, Peter Grehan wrote: > oh ... and ... the console spit out: >> >> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 4522, size: 8192 >> >> (swap is on a separate zVol). >> > > Ok - you may have hit a separate issue. Is ZFS ARC limited on your setup > ? If bhyve and ZFS (and other consumers) end up fighting for memory, > everyone loses :( A general rule of thumb is to limit ARC to less than the > bhyve VM usage + a few additional gig for the base system. > > The FreeBSD default of giving ZFS all memory minus 1GB doesn't work too > well when running VMs. > > later, > > Peter. >