From owner-freebsd-virtualization@freebsd.org Mon May 3 10:32:22 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BD3C662E3FE for ; Mon, 3 May 2021 10:32:22 +0000 (UTC) (envelope-from freebsd@grem.de) Received: from mail.evolve.de (mail.evolve.de [213.239.217.29]) (using TLSv1.3 with cipher TLS_CHACHA20_POLY1305_SHA256 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail.evolve.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FYfRn4shrz4gXf for ; Mon, 3 May 2021 10:32:21 +0000 (UTC) (envelope-from freebsd@grem.de) Received: by mail.evolve.de (OpenSMTPD) with ESMTP id 6636d580 for ; Mon, 3 May 2021 10:32:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=grem.de; h=date:from:to :subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=20180501; bh=Me1j4YRX QVcoRZv0FOdApvaYPKc=; b=nZESgw/xmNxLaZsuiKnqAVO9Zxe4Ug9JWVgJL2El lz4zceQ3JliExheh44uV8g5eF3wxT6RWqGgfPXz5f2Wp5NYf4PggplvTCiBOKEpq hBAJflnh4CNbga8Rd1H2Lmdb7oxzm1dUGrJEXpqydtDNLmuYo9OdWk23bIbvlmoS RcwRSb1ArUvvRS7KhtIOK9e+WfR1ZOHvwCP15mfDq8nUq3oesVXCmruBP//VQ0A1 04KWcZtuJLPn+HVEqYXBlkYljdb/9hNfnP8FwbHU1lxAt6FA4YNw2xTJou9lpNJQ kpk9zkvM6HDU40fJcKGR3+g7XbFv+8Qhty7B6zOKRdcVaQ== DomainKey-Signature: a=rsa-sha1; c=nofws; d=grem.de; h=date:from:to :subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; q=dns; s=20180501; b=Yj qCWANK+b2NCXusKS7OIWKc5y0egWjuK4WJNkkAJbTG1xFZ3H0EDiBtuazsADUL+Y 6hDrt8OqOPiFFo5/+n0xmp44TzS+HwdpUzz8f2OqBz93KkNYJcCb7+liv5quN2v3 XQL1P7nssKM/YZrNEMUqxs1fp1yXjmYdjryRsaXU6OJZjh7odFifPMSGaqEHZrBP 0N1h8Xg1zW+iwTTYfgQIYCehTBy+6Ojk0G1pG8VY5EJple84VHMb9FhhpZuxxs9B ygHcjWKKl8Drwf/Y7mdQhhBqWku4mkr9NdWvhkCfGSDrG49Wln/+jF4cEkXNg2zr DZxkfDHq3T+56z50Wa0Q== Received: by mail.evolve.de (OpenSMTPD) with ESMTPSA id 40eefae8 (TLSv1.3:AEAD-CHACHA20-POLY1305-SHA256:256:NO) for ; Mon, 3 May 2021 10:32:14 +0000 (UTC) Date: Mon, 3 May 2021 12:31:44 +0200 From: Michael Gmelin To: freebsd-virtualization@freebsd.org Subject: Re: FreeBSD 13 on bhyve freezes under load Message-ID: <20210503123144.134c209c@bsd64.grem.de> In-Reply-To: <587b3653-0433-ef46-a53d-d6ab0841f993@borderworlds.dk> References: <587b3653-0433-ef46-a53d-d6ab0841f993@borderworlds.dk> X-Face: $wrgCtfdVw_H9WAY?S&9+/F"!41z'L$uo*WzT8miX?kZ~W~Lr5W7v?j0Sde\mwB&/ypo^}> +a'4xMc^^KroE~+v^&^#[B">soBo1y6(TW6#UZiC]o>C6`ej+i Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAJFBMVEWJBwe5BQDl LASZU0/LTEWEfHbyj0Txi32+sKrp1Mv944X8/fm1rS+cAAAACXBIWXMAAAsTAAAL EwEAmpwYAAAAB3RJTUUH3wESCxwC7OBhbgAAACFpVFh0Q29tbWVudAAAAAAAQ3Jl YXRlZCB3aXRoIFRoZSBHSU1QbbCXAAAAAghJREFUOMu11DFvEzEUAGCfEhBVFzuq AKkLd0O6VrIQsLXVSZXoWE5N1K3DobBBA9fQpRWc8OkWouaIjedWKiyREOKs+3PY fvalCNjgLVHeF7/3bMtBzV8C/VsQ8tecEgCcDgrzjekwKZ7TwsJZd/ywEKwwP+ZM 8P3drTsAwWn2mpWuDDuYiK1bFs6De0KUUFw0tWxm+D4AIhuuvZqtyWYeO7jQ4Aea 7jUqI+ixhQoHex4WshEvSXdood7stlv4oSuFOC4tqGcr0NjEqXgV4mMJO38nld4+ xKNxRDon7khyKVqY7YR4d+Cg0OMrkWXZOM7YDkEfKiilCn1qYv4mighZiynuHHOA Wq9QJq+BIES7lMFUtcikMnkDGHUoncA+uHgrP0ctIEqfwLHzeSo+eUA66AqzwN6n 2ZHJhw6Qh/PoyC/QENyEyC/AyNjq74Bs+3UH0xYwzDUC4B97HgLocg1QLYgDDO1v f3UX9Y307Ew4AHh67YAFFsxEpkXwpXY3eIgMhAAE3R19L919nNnuD2wlPcDE3UeT L2ytEICQib9BXgS2fU8PrD82ToYO1OEmMSnYTjSqSv9wdC0tPYC+rQRQD9ESnldF CyqfmiYW+tlALt8gH2xrMdC/youbjzPXEun+/ReXsMCDyve3dZc09fn2Oas8oXGc Jj6/fOeK5UmSMPmf/jL+GD8BEj0k/Fn6IO4AAAAASUVORK5CYII= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4FYfRn4shrz4gXf X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=grem.de header.s=20180501 header.b=nZESgw/x; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd@grem.de designates 213.239.217.29 as permitted sender) smtp.mailfrom=freebsd@grem.de X-Spamd-Result: default: False [-1.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[grem.de:s=20180501]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:213.239.217.29/32]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[213.239.217.29:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[grem.de]; NEURAL_SPAM_SHORT(1.00)[1.000]; DKIM_TRACE(0.00)[grem.de:+]; NEURAL_HAM_LONG(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[213.239.217.29:from]; ASN(0.00)[asn:24940, ipnet:213.239.192.0/18, country:DE]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2021 10:32:22 -0000 On Mon, 3 May 2021 11:14:45 +0200 Christian St=C3=A6rk wrote: > Hello, >=20 > I recently upgraded one of my bhyve guests to FreeBSD 13.0-RELEASE. > The host is running 12.2-RELEASE-p4. >=20 > The guest in question is used for building packages with poudriere > and is always the first one to get upgraded to new releases. >=20 > The guest freezes consistently every time poudriere tries to build > the thunderbird package. Although it is not a complete freeze. It > responds to ping and programs like top and systat continue updating. > However, all I/O operations seem to hang. >=20 > systat looks like this when it happens:=20 > https://borderworlds.dk/~xi/FreeBSD%2013%20frozen.png=20 > >=20 > What is the best way to debug this? Sharing top output might help (memory/swap and ZFS arc usage). Also, make sure you're not low on diskspace (these are the two most common issues I encountered with hanging bhyve VMs - out of non-virtual memory and out of diskspace of the host machine filesystem). Some basic info on the host and your VM configuration might also help others to reproduce the problem. -m --=20 Michael Gmelin