From owner-freebsd-virtualization@freebsd.org Sun May 2 21:00:03 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5404B5F99E0 for ; Sun, 2 May 2021 21:00:03 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4FYJQW1pGVz3KFL for ; Sun, 2 May 2021 21:00:03 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.nyi.freebsd.org (Postfix) id 3DCD25F96D8; Sun, 2 May 2021 21:00:03 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3D99F5F9BD9 for ; Sun, 2 May 2021 21:00:03 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FYJQW1GrPz3KNc for ; Sun, 2 May 2021 21:00:03 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1EDA511E0C for ; Sun, 2 May 2021 21:00:03 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 142L03hl084545 for ; Sun, 2 May 2021 21:00:03 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 142L03to084544 for virtualization@FreeBSD.org; Sun, 2 May 2021 21:00:03 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202105022100.142L03to084544@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: virtualization@FreeBSD.org Subject: Problem reports for virtualization@FreeBSD.org that need special attention Date: Sun, 2 May 2021 21:00:03 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 May 2021 21:00:03 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- In Progress | 247208 | mpt(4): VMWare virtualized LSI controller panics New | 240945 | [hyper-v] [netvsc] hn network driver incorrectly 2 problems total for which you should take action. From owner-freebsd-virtualization@freebsd.org Mon May 3 09:14:49 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4BA4262C60D for ; Mon, 3 May 2021 09:14:49 +0000 (UTC) (envelope-from xi@borderworlds.dk) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FYckJ1ltwz4cdh for ; Mon, 3 May 2021 09:14:47 +0000 (UTC) (envelope-from xi@borderworlds.dk) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 791085C00A1 for ; Mon, 3 May 2021 05:14:47 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Mon, 03 May 2021 05:14:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=borderworlds.dk; h=to:from:subject:message-id:date:mime-version:content-type :content-transfer-encoding; s=fm2; bh=YPCNKc5hmojXNevheEKIBfuEH0 EbR1yqcuempNuOb8s=; b=cyVNuvfJmtQJFLYHiY3GipFqgivlbzAGkbppuK5W+l VfrUC6MShjniNVukS7j92MCF9gNktL42DPHtgWLrI6x4M2spluhxirrgERvWb4y4 WtqvF4jsbDqld0hXKsram4f7PFB18jymaVgE3VC2Ox7O+PjcQ5HaWm2o1OvQV4fA f71z4+GxWfZi9IWcZ4QmSmz+SL62tsuICCr5FwBDdqOFUMELaj3WFfWugtxaiGyX gkF1ue+c7pZpcnhodTUW2DOK26YQnaoN7Shavb+ogtFHUi/+Yp7v4YR621E+IyDg LllnjUMJvfaBAsYUhrdZZi1p3SnRlxqymXuFIWjg0SgA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=YPCNKc 5hmojXNevheEKIBfuEH0EbR1yqcuempNuOb8s=; b=uUiRabFhoUfYi49vuQhnyl 22sbLo8uDjFSh362u/UUxSP0rjGd20/6lDr9eh57E51mFrzMdUuEJ1J7uMGaPGhh E+iEs80y/Madc/TBkLZmxyZW+fQkdGkvLVJDa1UkLV9d3iFQYwzc3qSC6VAQFbIK E5zWcz/BRgavHDfiihYZGotmPteKgiA8URLLEc8IlkhAneXQlpMXjn3RVBjSClot 6hJuKeu9D7rYs6nIWEq/cFYFw9Ji9EAI72zRBjy/xRGrKyh9zQO3/O3qFzLwxT+m t+BeKQ4L9I8Wae596X2xr+VB9keFQ8VQipg/uHuIsoKt2buSCzD7QL8bJuOXPjPw == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvdefgedgudefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefvhffuohfkffgfgggtgfesthejre dttdefjeenucfhrhhomhepvehhrhhishhtihgrnhgpufhtoohrkhcuoeigihessghorhgu vghrfihorhhlughsrdgukheqnecuggftrfgrthhtvghrnhepuedvtdfggfduveetfedtfe etudfhgeekgfdtteejledttdeuteehvdejgefgtdeinecuffhomhgrihhnpegsohhruggv rhifohhrlhgushdrughknecukfhppeeivddruddtjedrjedrudeivdenucevlhhushhtvg hrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeigihessghorhguvghrfiho rhhlughsrdgukh X-ME-Proxy: Received: from baku.borderworlds.dk (3e6b07a2.rev.stofanet.dk [62.107.7.162]) by mail.messagingengine.com (Postfix) with ESMTPA for ; Mon, 3 May 2021 05:14:46 -0400 (EDT) To: freebsd-virtualization@freebsd.org From: =?UTF-8?Q?Christian_St=c3=a6rk?= Subject: FreeBSD 13 on bhyve freezes under load Organization: The Border Worlds Message-ID: <587b3653-0433-ef46-a53d-d6ab0841f993@borderworlds.dk> Date: Mon, 3 May 2021 11:14:45 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Rspamd-Queue-Id: 4FYckJ1ltwz4cdh X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=borderworlds.dk header.s=fm2 header.b=cyVNuvfJ; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=uUiRabFh; dmarc=none; spf=pass (mx1.freebsd.org: domain of xi@borderworlds.dk designates 66.111.4.29 as permitted sender) smtp.mailfrom=xi@borderworlds.dk X-Spamd-Result: default: False [-3.04 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.29]; TO_DN_NONE(0.00)[]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[borderworlds.dk:+,messagingengine.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.29:from]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; R_MIXED_CHARSET(0.56)[subject]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[borderworlds.dk:s=fm2,messagingengine.com:s=fm2]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; DMARC_NA(0.00)[borderworlds.dk]; RCPT_COUNT_ONE(0.00)[1]; RECEIVED_SPAMHAUS_PBL(0.00)[62.107.7.162:received]; RWL_MAILSPIKE_VERYGOOD(0.00)[66.111.4.29:from]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2021 09:14:49 -0000 Hello, I recently upgraded one of my bhyve guests to FreeBSD 13.0-RELEASE. The host is running 12.2-RELEASE-p4. The guest in question is used for building packages with poudriere and is always the first one to get upgraded to new releases. The guest freezes consistently every time poudriere tries to build the thunderbird package. Although it is not a complete freeze. It responds to ping and programs like top and systat continue updating. However, all I/O operations seem to hang. systat looks like this when it happens: https://borderworlds.dk/~xi/FreeBSD%2013%20frozen.png What is the best way to debug this? Best regards Christian From owner-freebsd-virtualization@freebsd.org Mon May 3 10:32:22 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BD3C662E3FE for ; Mon, 3 May 2021 10:32:22 +0000 (UTC) (envelope-from freebsd@grem.de) Received: from mail.evolve.de (mail.evolve.de [213.239.217.29]) (using TLSv1.3 with cipher TLS_CHACHA20_POLY1305_SHA256 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail.evolve.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FYfRn4shrz4gXf for ; Mon, 3 May 2021 10:32:21 +0000 (UTC) (envelope-from freebsd@grem.de) Received: by mail.evolve.de (OpenSMTPD) with ESMTP id 6636d580 for ; Mon, 3 May 2021 10:32:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=grem.de; h=date:from:to :subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=20180501; bh=Me1j4YRX QVcoRZv0FOdApvaYPKc=; b=nZESgw/xmNxLaZsuiKnqAVO9Zxe4Ug9JWVgJL2El lz4zceQ3JliExheh44uV8g5eF3wxT6RWqGgfPXz5f2Wp5NYf4PggplvTCiBOKEpq hBAJflnh4CNbga8Rd1H2Lmdb7oxzm1dUGrJEXpqydtDNLmuYo9OdWk23bIbvlmoS RcwRSb1ArUvvRS7KhtIOK9e+WfR1ZOHvwCP15mfDq8nUq3oesVXCmruBP//VQ0A1 04KWcZtuJLPn+HVEqYXBlkYljdb/9hNfnP8FwbHU1lxAt6FA4YNw2xTJou9lpNJQ kpk9zkvM6HDU40fJcKGR3+g7XbFv+8Qhty7B6zOKRdcVaQ== DomainKey-Signature: a=rsa-sha1; c=nofws; d=grem.de; h=date:from:to :subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; q=dns; s=20180501; b=Yj qCWANK+b2NCXusKS7OIWKc5y0egWjuK4WJNkkAJbTG1xFZ3H0EDiBtuazsADUL+Y 6hDrt8OqOPiFFo5/+n0xmp44TzS+HwdpUzz8f2OqBz93KkNYJcCb7+liv5quN2v3 XQL1P7nssKM/YZrNEMUqxs1fp1yXjmYdjryRsaXU6OJZjh7odFifPMSGaqEHZrBP 0N1h8Xg1zW+iwTTYfgQIYCehTBy+6Ojk0G1pG8VY5EJple84VHMb9FhhpZuxxs9B ygHcjWKKl8Drwf/Y7mdQhhBqWku4mkr9NdWvhkCfGSDrG49Wln/+jF4cEkXNg2zr DZxkfDHq3T+56z50Wa0Q== Received: by mail.evolve.de (OpenSMTPD) with ESMTPSA id 40eefae8 (TLSv1.3:AEAD-CHACHA20-POLY1305-SHA256:256:NO) for ; Mon, 3 May 2021 10:32:14 +0000 (UTC) Date: Mon, 3 May 2021 12:31:44 +0200 From: Michael Gmelin To: freebsd-virtualization@freebsd.org Subject: Re: FreeBSD 13 on bhyve freezes under load Message-ID: <20210503123144.134c209c@bsd64.grem.de> In-Reply-To: <587b3653-0433-ef46-a53d-d6ab0841f993@borderworlds.dk> References: <587b3653-0433-ef46-a53d-d6ab0841f993@borderworlds.dk> X-Face: $wrgCtfdVw_H9WAY?S&9+/F"!41z'L$uo*WzT8miX?kZ~W~Lr5W7v?j0Sde\mwB&/ypo^}> +a'4xMc^^KroE~+v^&^#[B">soBo1y6(TW6#UZiC]o>C6`ej+i Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAJFBMVEWJBwe5BQDl LASZU0/LTEWEfHbyj0Txi32+sKrp1Mv944X8/fm1rS+cAAAACXBIWXMAAAsTAAAL EwEAmpwYAAAAB3RJTUUH3wESCxwC7OBhbgAAACFpVFh0Q29tbWVudAAAAAAAQ3Jl YXRlZCB3aXRoIFRoZSBHSU1QbbCXAAAAAghJREFUOMu11DFvEzEUAGCfEhBVFzuq AKkLd0O6VrIQsLXVSZXoWE5N1K3DobBBA9fQpRWc8OkWouaIjedWKiyREOKs+3PY fvalCNjgLVHeF7/3bMtBzV8C/VsQ8tecEgCcDgrzjekwKZ7TwsJZd/ywEKwwP+ZM 8P3drTsAwWn2mpWuDDuYiK1bFs6De0KUUFw0tWxm+D4AIhuuvZqtyWYeO7jQ4Aea 7jUqI+ixhQoHex4WshEvSXdood7stlv4oSuFOC4tqGcr0NjEqXgV4mMJO38nld4+ xKNxRDon7khyKVqY7YR4d+Cg0OMrkWXZOM7YDkEfKiilCn1qYv4mighZiynuHHOA Wq9QJq+BIES7lMFUtcikMnkDGHUoncA+uHgrP0ctIEqfwLHzeSo+eUA66AqzwN6n 2ZHJhw6Qh/PoyC/QENyEyC/AyNjq74Bs+3UH0xYwzDUC4B97HgLocg1QLYgDDO1v f3UX9Y307Ew4AHh67YAFFsxEpkXwpXY3eIgMhAAE3R19L919nNnuD2wlPcDE3UeT L2ytEICQib9BXgS2fU8PrD82ToYO1OEmMSnYTjSqSv9wdC0tPYC+rQRQD9ESnldF CyqfmiYW+tlALt8gH2xrMdC/youbjzPXEun+/ReXsMCDyve3dZc09fn2Oas8oXGc Jj6/fOeK5UmSMPmf/jL+GD8BEj0k/Fn6IO4AAAAASUVORK5CYII= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4FYfRn4shrz4gXf X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=grem.de header.s=20180501 header.b=nZESgw/x; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd@grem.de designates 213.239.217.29 as permitted sender) smtp.mailfrom=freebsd@grem.de X-Spamd-Result: default: False [-1.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[grem.de:s=20180501]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:213.239.217.29/32]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[213.239.217.29:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[grem.de]; NEURAL_SPAM_SHORT(1.00)[1.000]; DKIM_TRACE(0.00)[grem.de:+]; NEURAL_HAM_LONG(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[213.239.217.29:from]; ASN(0.00)[asn:24940, ipnet:213.239.192.0/18, country:DE]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2021 10:32:22 -0000 On Mon, 3 May 2021 11:14:45 +0200 Christian St=C3=A6rk wrote: > Hello, >=20 > I recently upgraded one of my bhyve guests to FreeBSD 13.0-RELEASE. > The host is running 12.2-RELEASE-p4. >=20 > The guest in question is used for building packages with poudriere > and is always the first one to get upgraded to new releases. >=20 > The guest freezes consistently every time poudriere tries to build > the thunderbird package. Although it is not a complete freeze. It > responds to ping and programs like top and systat continue updating. > However, all I/O operations seem to hang. >=20 > systat looks like this when it happens:=20 > https://borderworlds.dk/~xi/FreeBSD%2013%20frozen.png=20 > >=20 > What is the best way to debug this? Sharing top output might help (memory/swap and ZFS arc usage). Also, make sure you're not low on diskspace (these are the two most common issues I encountered with hanging bhyve VMs - out of non-virtual memory and out of diskspace of the host machine filesystem). Some basic info on the host and your VM configuration might also help others to reproduce the problem. -m --=20 Michael Gmelin From owner-freebsd-virtualization@freebsd.org Mon May 3 11:14:57 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E009C62F345 for ; Mon, 3 May 2021 11:14:57 +0000 (UTC) (envelope-from xi@borderworlds.dk) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FYgNw65vXz4jTZ for ; Mon, 3 May 2021 11:14:56 +0000 (UTC) (envelope-from xi@borderworlds.dk) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id E1F0A5C0085 for ; Mon, 3 May 2021 07:14:55 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Mon, 03 May 2021 07:14:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=borderworlds.dk; h=subject:to:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm2; bh=u c1rAcnIH+rwjiHGGaghdlUXVWc9hCqox2duLTH7odg=; b=OtlicXL+KBiGHZR73 qLuze4IecGE2rBA2emOwCSiWgPNwZvXplknvcmUrfNnD61oMj7EH68qx6qi/1bBO 3bgR7/GBUaKoeuAnRNZ3dN5JqXPl8jnsCdevi1WnQxCF6yKlQSvD5rOzq5j3WQIa eGS/bCvk1Ik0uOTfEsZ/8fcg2IjDh3EsdSEsbJ6ENw3LCKiGlwmzdEk/Rx5M3gOd wq0HGgyRr8Z87I1qy6oZDNcfld3Zv7/x9qjXnjlhkh1nR/gqZyi+eyZV4tZ8L/U6 6ckDNJkqOk65xJTA7ZYcYPrGT5afzPL6j1VjpRdohr8hA7y+q9HbSKmsf+JPiSX6 RvnLg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=uc1rAcnIH+rwjiHGGaghdlUXVWc9hCqox2duLTH7o dg=; b=N3PibY+npE/myBeu856wOenNY130myVdnion+kOXnkfqj4DRqvHhvPnT/ yyz7p/UsoYwyhoaqpx6tUF5Bb+StejvSmJKfGjtGTDFJpnUyEJivJCNfIBNNpzXU U+/PfYsm111kZRTCm3yepZGYKqpNyYemUtWNCVtCohPw/jcmtzuNlmNMfgO+i103 AvqSLUPFf2bjfkGdW7QUD6mdzXy3zrKlulw5sDMv6uMYhX8bh3lD3gOnzbjodZWm KEzs8SBS0uUgUhdriqCdBfencH8EgrozgRTdYIpgJucsClfEteYr8AInlykpzld4 ZpP4K9iP3Hd+HtjAUhV1rOomIm0yw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvdefgedgfeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefuvfhfhfhokffffgggjggtgfesth ekredttdefjeenucfhrhhomhepvehhrhhishhtihgrnhgpufhtoohrkhcuoeigihessgho rhguvghrfihorhhlughsrdgukheqnecuggftrfgrthhtvghrnhephffffeeluedtfefhje ekleevtddtvdehgfehfefhudetkefggfduueeuffevvdegnecuffhomhgrihhnpegsohhr uggvrhifohhrlhgushdrughknecukfhppeeivddruddtjedrjedrudeivdenucevlhhush htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeigihessghorhguvghr fihorhhlughsrdgukh X-ME-Proxy: Received: from baku.borderworlds.dk (3e6b07a2.rev.stofanet.dk [62.107.7.162]) by mail.messagingengine.com (Postfix) with ESMTPA for ; Mon, 3 May 2021 07:14:55 -0400 (EDT) Subject: Re: FreeBSD 13 on bhyve freezes under load To: freebsd-virtualization@freebsd.org References: <587b3653-0433-ef46-a53d-d6ab0841f993@borderworlds.dk> <20210503123144.134c209c@bsd64.grem.de> From: =?UTF-8?Q?Christian_St=c3=a6rk?= Organization: The Border Worlds Message-ID: <9ddd42be-e316-c813-8132-b7b90b694f34@borderworlds.dk> Date: Mon, 3 May 2021 13:14:53 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 In-Reply-To: <20210503123144.134c209c@bsd64.grem.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4FYgNw65vXz4jTZ X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=borderworlds.dk header.s=fm2 header.b=OtlicXL+; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=N3PibY+n; dmarc=none; spf=pass (mx1.freebsd.org: domain of xi@borderworlds.dk designates 66.111.4.25 as permitted sender) smtp.mailfrom=xi@borderworlds.dk X-Spamd-Result: default: False [-1.60 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[66.111.4.25:from]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.25:c]; TO_DN_NONE(0.00)[]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[borderworlds.dk:+,messagingengine.com:+]; RECEIVED_SPAMHAUS_PBL(0.00)[62.107.7.162:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[66.111.4.25:from]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[borderworlds.dk:s=fm2,messagingengine.com:s=fm2]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.25:from]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; DMARC_NA(0.00)[borderworlds.dk]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[66.111.4.25:from:127.0.2.255]; NEURAL_SPAM_SHORT(1.00)[1.000]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2021 11:14:57 -0000 Hi Michael, I am not using ZFS and have plenty of free diskspace: Filesystem     Size    Used   Avail Capacity Mounted on /dev/ada0p2    190G     92G     83G    52%    / top looks like this on a not-hung system: last pid: 38292;  load averages:  0.91, 0.87,  0.88    up 0+01:13:16  13:12:12 32 processes:  2 running, 29 sleeping, 1 zombie CPU:  5.5% user,  0.0% nice,  7.8% system,  7.0% interrupt, 79.7% idle Mem: 59M Active, 3549M Inact, 1090M Wired, 610M Buf, 2250M Free Swap: 24G Total, 24G Free   PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME    WCPU COMMAND 38079 root          1  22    0    99M    43M RUN      0:03  14.02% pkg-static 38285 xi            1  20    0    14M  3612K RUN      0:00   0.10% top 37889 root          1  22    0    15M  3680K nanslp   0:02   0.05% sh   759 root          1  20    0    17M  6200K select   0:04   0.01% tmux 37881 xi            1  20    0    21M  9252K select   0:00   0.00% sshd   600 ntpd          1  20    0    17M  4980K select   0:00   0.00% ntpd   749 xi            1  20    0    21M  8804K select   0:03   0.00% sshd 37882 xi            1  20    0    15M  5584K pause    0:00   0.00% zsh 37603 root          1  20    0    15M  3900K select   0:02   0.00% sh 78529 root          1  26    0    15M  3892K piperd   0:00   0.00% sh 36203 root          1  47    0    23M    10M wait     0:00   0.00% perl   750 xi            1  25    0    15M  5296K pause    0:00   0.00% zsh   542 root          1  20    0    13M  2632K select   0:00   0.00% syslogd   341 root          1  20    0    11M  1424K select   0:00   0.00% devd   753 root          1  20    0    16M  4428K pause    0:00   0.00% csh 37826 root          1  20    0    21M  9228K select   0:00   0.00% sshd 37790 root          1  52    0    15M  3896K wait     0:00   0.00% sh It occurred to me that I could try using ahci-hd instead of virtio-blk, so I am trying that now to see if it makes a difference. Best regards Christian On 5/3/21 12:31 PM, Michael Gmelin wrote: > > > On Mon, 3 May 2021 11:14:45 +0200 > Christian Stærk wrote: > >> Hello, >> >> I recently upgraded one of my bhyve guests to FreeBSD 13.0-RELEASE. >> The host is running 12.2-RELEASE-p4. >> >> The guest in question is used for building packages with poudriere >> and is always the first one to get upgraded to new releases. >> >> The guest freezes consistently every time poudriere tries to build >> the thunderbird package. Although it is not a complete freeze. It >> responds to ping and programs like top and systat continue updating. >> However, all I/O operations seem to hang. >> >> systat looks like this when it happens: >> https://borderworlds.dk/~xi/FreeBSD%2013%20frozen.png >> >> >> What is the best way to debug this? > Sharing top output might help (memory/swap and ZFS arc usage). Also, > make sure you're not low on diskspace (these are the two most common > issues I encountered with hanging bhyve VMs - out of non-virtual memory > and out of diskspace of the host machine filesystem). > > Some basic info on the host and your VM configuration might also help > others to reproduce the problem. > > -m > From owner-freebsd-virtualization@freebsd.org Mon May 3 11:57:36 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 68B19630690 for ; Mon, 3 May 2021 11:57:36 +0000 (UTC) (envelope-from freebsd@grem.de) Received: from mail.evolve.de (mail.evolve.de [213.239.217.29]) (using TLSv1.3 with cipher TLS_CHACHA20_POLY1305_SHA256 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail.evolve.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FYhL73nGdz4l42 for ; Mon, 3 May 2021 11:57:35 +0000 (UTC) (envelope-from freebsd@grem.de) Received: by mail.evolve.de (OpenSMTPD) with ESMTP id 585ed4e3; Mon, 3 May 2021 11:57:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=grem.de; h=date:from:to:cc :subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=20180501; bh=zKCwpZ98 vL0kT0xspYhZm8Q/zX0=; b=lP86WYhBZJuo4CfdlqvKAWC55ZFuiFjkzekvMXN5 cYW25Q8hQGGfwc2njP9sYxZjotz5c9G7w9PkaE+fjsV0mD6G+vkJq1IXAi9Dmbkr s/r5EZjoH0yxB+FyrqjUQi7G4kTtSzer6rbGOH17q0kVHkHAAFs/tBJKQDga7zZL L6r5N1xtJNFgh1lR2hmP2hzeylBnlRoE7ktRmk8sH93s3xiGJ6Qmuxt6LAAxVfqP wT774FEIMHFPPUDmapZueDDZqMIZM2xU7Zi3xmGXFr3AxB82QMa8jyATyEVQinyo SyH05U9fD6LzL3MGf3xbgwk654HMtJM0vqx+I4gvOY8jkw== DomainKey-Signature: a=rsa-sha1; c=nofws; d=grem.de; h=date:from:to:cc :subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; q=dns; s=20180501; b=hC +1m4L4Hf4b56SrpebvW4/rPJaMRC9p5s14ok6VTdimb8MV+Evy+FasCuKXoZDTsf 7vZmE3Ae2o7LFzNTYZdYqtFyVrwAoRqaaJDncrILKMfIS/bX1l7zg6NwEGM342ba o//MQZSzjgzfh1pFGeieL6bYk4ocluElPLFMlOB1aJ8bE/Nbf88tUTAZamgP7UgS VHaC7/6ELbrFbmakrIf3RQiZ3y1TlW4NJCSyWUQZ7rz8lXaA8f0JhboWS9ckLYRO cTiTgjWIsNkXLBh3auw5BrL9ikbhXF5n28Z93ITjiye/aF8rz04ZVbyVtnavhNZb tsZA1MPKD4wh68vcFURQ== Received: by mail.evolve.de (OpenSMTPD) with ESMTPSA id 028f26c8 (TLSv1.3:AEAD-CHACHA20-POLY1305-SHA256:256:NO); Mon, 3 May 2021 11:57:28 +0000 (UTC) Date: Mon, 3 May 2021 13:56:58 +0200 From: Michael Gmelin To: Christian =?UTF-8?B?U3TDpnJr?= Cc: freebsd-virtualization@freebsd.org Subject: Re: FreeBSD 13 on bhyve freezes under load Message-ID: <20210503135658.4a388e62@bsd64.grem.de> In-Reply-To: <9ddd42be-e316-c813-8132-b7b90b694f34@borderworlds.dk> References: <587b3653-0433-ef46-a53d-d6ab0841f993@borderworlds.dk> <20210503123144.134c209c@bsd64.grem.de> <9ddd42be-e316-c813-8132-b7b90b694f34@borderworlds.dk> X-Face: $wrgCtfdVw_H9WAY?S&9+/F"!41z'L$uo*WzT8miX?kZ~W~Lr5W7v?j0Sde\mwB&/ypo^}> +a'4xMc^^KroE~+v^&^#[B">soBo1y6(TW6#UZiC]o>C6`ej+i Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAJFBMVEWJBwe5BQDl LASZU0/LTEWEfHbyj0Txi32+sKrp1Mv944X8/fm1rS+cAAAACXBIWXMAAAsTAAAL EwEAmpwYAAAAB3RJTUUH3wESCxwC7OBhbgAAACFpVFh0Q29tbWVudAAAAAAAQ3Jl YXRlZCB3aXRoIFRoZSBHSU1QbbCXAAAAAghJREFUOMu11DFvEzEUAGCfEhBVFzuq AKkLd0O6VrIQsLXVSZXoWE5N1K3DobBBA9fQpRWc8OkWouaIjedWKiyREOKs+3PY fvalCNjgLVHeF7/3bMtBzV8C/VsQ8tecEgCcDgrzjekwKZ7TwsJZd/ywEKwwP+ZM 8P3drTsAwWn2mpWuDDuYiK1bFs6De0KUUFw0tWxm+D4AIhuuvZqtyWYeO7jQ4Aea 7jUqI+ixhQoHex4WshEvSXdood7stlv4oSuFOC4tqGcr0NjEqXgV4mMJO38nld4+ xKNxRDon7khyKVqY7YR4d+Cg0OMrkWXZOM7YDkEfKiilCn1qYv4mighZiynuHHOA Wq9QJq+BIES7lMFUtcikMnkDGHUoncA+uHgrP0ctIEqfwLHzeSo+eUA66AqzwN6n 2ZHJhw6Qh/PoyC/QENyEyC/AyNjq74Bs+3UH0xYwzDUC4B97HgLocg1QLYgDDO1v f3UX9Y307Ew4AHh67YAFFsxEpkXwpXY3eIgMhAAE3R19L919nNnuD2wlPcDE3UeT L2ytEICQib9BXgS2fU8PrD82ToYO1OEmMSnYTjSqSv9wdC0tPYC+rQRQD9ESnldF CyqfmiYW+tlALt8gH2xrMdC/youbjzPXEun+/ReXsMCDyve3dZc09fn2Oas8oXGc Jj6/fOeK5UmSMPmf/jL+GD8BEj0k/Fn6IO4AAAAASUVORK5CYII= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4FYhL73nGdz4l42 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=grem.de header.s=20180501 header.b=lP86WYhB; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd@grem.de designates 213.239.217.29 as permitted sender) smtp.mailfrom=freebsd@grem.de X-Spamd-Result: default: False [-1.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[grem.de:s=20180501]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:213.239.217.29/32]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grem.de]; NEURAL_SPAM_SHORT(1.00)[1.000]; SPAMHAUS_ZRD(0.00)[213.239.217.29:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[grem.de:+]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[213.239.217.29:from]; ASN(0.00)[asn:24940, ipnet:213.239.192.0/18, country:DE]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2021 11:57:36 -0000 On Mon, 3 May 2021 13:14:53 +0200 Christian St=C3=A6rk wrote: > Hi Michael, >=20 > I am not using ZFS and have plenty of free diskspace: > Filesystem=C2=A0=C2=A0=C2=A0=C2=A0 Size=C2=A0=C2=A0=C2=A0 Used=C2=A0=C2= =A0 Avail Capacity Mounted on > /dev/ada0p2=C2=A0=C2=A0=C2=A0 190G=C2=A0=C2=A0=C2=A0=C2=A0 92G=C2=A0=C2= =A0=C2=A0=C2=A0 83G=C2=A0=C2=A0=C2=A0 52%=C2=A0=C2=A0=C2=A0 / >=20 > top looks like this on a not-hung system: >=20 > last pid: 38292;=C2=A0 load averages:=C2=A0 0.91, 0.87,=C2=A0 0.88=C2=A0= =C2=A0=C2=A0 up 0+01:13:16=C2=A0=20 > 13:12:12 > 32 processes:=C2=A0 2 running, 29 sleeping, 1 zombie > CPU:=C2=A0 5.5% user,=C2=A0 0.0% nice,=C2=A0 7.8% system,=C2=A0 7.0% inte= rrupt, 79.7% idle > Mem: 59M Active, 3549M Inact, 1090M Wired, 610M Buf, 2250M Free > Swap: 24G Total, 24G Free >=20 > =C2=A0 PID USERNAME=C2=A0=C2=A0=C2=A0 THR PRI NICE=C2=A0=C2=A0 SIZE=C2= =A0=C2=A0=C2=A0 RES STATE=C2=A0=C2=A0=C2=A0 TIME=C2=A0=C2=A0=C2=A0 WCPU > COMMAND 38079 root=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = 1=C2=A0 22=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0 99M=C2=A0=C2=A0=C2=A0 43M = RUN=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0:03 > 14.02% pkg-static Based on hanging pkg-static process I assume that top output is from within the VM though, isn't it? Or is this on the host? Best Michael --=20 Michael Gmelin From owner-freebsd-virtualization@freebsd.org Mon May 3 17:27:54 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 183076380AB for ; Mon, 3 May 2021 17:27:54 +0000 (UTC) (envelope-from rebecca@bsdio.com) Received: from out03.mta.xmission.com (out03.mta.xmission.com [166.70.13.233]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FYqgD4p67z3HGq for ; Mon, 3 May 2021 17:27:52 +0000 (UTC) (envelope-from rebecca@bsdio.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=simple/simple; d=bsdio.com; s=xmission; h=Subject:Content-Transfer-Encoding:Content-Type:In-Reply-To: MIME-Version:Date:Message-ID:From:References:To:Sender:Reply-To:Cc:Content-ID :Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To: Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe :List-Post:List-Owner:List-Archive; bh=Py2NJkEBLEg2dli2tw8JJoH9qy69rvzQoHjgjNAPYpk=; b=WVAUD+JqZ0vbyuE9P+1CjadsDr FsBuqb2C4m2olFcdm2kunWBrKaqq/WZhuvlbvFgph2vBZYFSLRWAghBRON8u/e/l202XsxnPwGuGZ 9fabBe9vj0L34rXzuj8rtFK4vPhkmWwhHMrXImoiewNjDnZm8S0bQK/USO2kM5CZGO2E=; Received: from in02.mta.xmission.com ([166.70.13.52]) by out03.mta.xmission.com with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.93) (envelope-from ) id 1ldcMH-008R9o-J7; Mon, 03 May 2021 11:27:49 -0600 Received: from mta4.zcs.xmission.com ([166.70.13.68]) by in02.mta.xmission.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.93) (envelope-from ) id 1ldcMG-00DdSz-Ed; Mon, 03 May 2021 11:27:49 -0600 Received: from localhost (localhost [127.0.0.1]) by mta4.zcs.xmission.com (Postfix) with ESMTP id B6028500E78; Mon, 3 May 2021 11:27:47 -0600 (MDT) X-Amavis-Modified: Mail body modified (using disclaimer) - mta4.zcs.xmission.com Received: from mta4.zcs.xmission.com ([127.0.0.1]) by localhost (mta4.zcs.xmission.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id cm-zYc-T3aeC; Mon, 3 May 2021 11:27:47 -0600 (MDT) Received: from photon.int.bluestop.org (c-174-52-16-57.hsd1.ut.comcast.net [174.52.16.57]) by mta4.zcs.xmission.com (Postfix) with ESMTPSA id 8952D500DC3; Mon, 3 May 2021 11:27:47 -0600 (MDT) To: =?UTF-8?Q?Christian_St=c3=a6rk?= , freebsd-virtualization@freebsd.org References: <587b3653-0433-ef46-a53d-d6ab0841f993@borderworlds.dk> <20210503123144.134c209c@bsd64.grem.de> <9ddd42be-e316-c813-8132-b7b90b694f34@borderworlds.dk> From: Rebecca Cran Message-ID: Date: Mon, 3 May 2021 11:27:47 -0600 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: <9ddd42be-e316-c813-8132-b7b90b694f34@borderworlds.dk> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-XM-SPF: eid=1ldcMG-00DdSz-Ed; ; ; mid=; ; ; hst=in02.mta.xmission.com; ; ; ip=166.70.13.68; ; ; frm=rebecca@bsdio.com; ; ; spf=none X-SA-Exim-Connect-IP: 166.70.13.68 X-SA-Exim-Mail-From: rebecca@bsdio.com X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on sa07.xmission.com X-Spam-Level: * X-Spam-Status: No, score=1.6 required=8.0 tests=ALL_TRUSTED,BAYES_50, DCC_CHECK_NEGATIVE,NICE_REPLY_A,T_TM2_M_HEADER_IN_MSG,TooManyTo_001, XMNoVowels,XM_B_Unicode,XM_B_Unicode3 autolearn=disabled version=3.4.2 X-Spam-Report: * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP * 0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60% * [score: 0.5060] * 0.3 TooManyTo_001 Multiple "To" Header Recipients 2x (uncommon) * 1.5 XMNoVowels Alpha-numberic number with no vowels * 0.0 T_TM2_M_HEADER_IN_MSG BODY: No description available. * 0.0 XM_B_Unicode3 BODY: Testing for specific types of unicode * 0.0 XM_B_Unicode BODY: Testing for specific types of unicode * -0.0 DCC_CHECK_NEGATIVE Not listed in DCC * [sa07 1397; IP=ok Body=1 Fuz1=1] [Fuz2=1] * -0.0 NICE_REPLY_A Looks like a legit reply (A) X-Spam-DCC: XMission; sa07 1397; IP=ok Body=1 Fuz1=1 Fuz2=1 X-Spam-Combo: =?ISO-8859-1?Q?*;Christian St=c3=a6rk , freebsd-virtualization@freebsd.org?= X-Spam-Relay-Country: X-Spam-Timing: total 517 ms - load_scoreonly_sql: 0.03 (0.0%), signal_user_changed: 11 (2.1%), b_tie_ro: 9 (1.8%), parse: 0.95 (0.2%), extract_message_metadata: 3.4 (0.7%), get_uri_detail_list: 0.29 (0.1%), tests_pri_-1000: 2.5 (0.5%), tests_pri_-950: 1.29 (0.2%), tests_pri_-900: 1.13 (0.2%), tests_pri_-90: 250 (48.4%), check_bayes: 248 (48.0%), b_tokenize: 4.2 (0.8%), b_tok_get_all: 4.1 (0.8%), b_comp_prob: 1.55 (0.3%), b_tok_touch_all: 235 (45.4%), b_finish: 1.08 (0.2%), tests_pri_0: 228 (44.0%), check_dkim_signature: 0.51 (0.1%), check_dkim_adsp: 42 (8.2%), poll_dns_idle: 35 (6.8%), tests_pri_10: 3.3 (0.6%), tests_pri_500: 14 (2.6%), rewrite_mail: 0.00 (0.0%) Subject: Re: FreeBSD 13 on bhyve freezes under load X-SA-Exim-Version: 4.2.1 (built Sat, 08 Feb 2020 21:53:50 +0000) X-SA-Exim-Scanned: Yes (on in02.mta.xmission.com) X-Rspamd-Queue-Id: 4FYqgD4p67z3HGq X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdio.com header.s=xmission header.b=WVAUD+Jq; dmarc=none; spf=none (mx1.freebsd.org: domain of rebecca@bsdio.com has no SPF policy when checking 166.70.13.233) smtp.mailfrom=rebecca@bsdio.com X-Spamd-Result: default: False [-3.40 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[6]; R_DKIM_ALLOW(-0.20)[bsdio.com:s=xmission]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[bsdio.com]; RBL_DBL_DONT_QUERY_IPS(0.00)[166.70.13.233:from]; SPAMHAUS_ZRD(0.00)[166.70.13.233:from:127.0.2.255]; RECEIVED_SPAMHAUS_PBL(0.00)[174.52.16.57:received]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bsdio.com:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:6315, ipnet:166.70.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_IN_DNSWL_LOW(-0.10)[166.70.13.233:from] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2021 17:27:54 -0000 On 5/3/21 5:14 AM, Christian Stærk wrote: > > It occurred to me that I could try using ahci-hd instead of > virtio-blk, so I am trying that now to see if it makes a difference. You might also want to try nvme, since it's higher performance than ahci-hd. -- Rebecca Cran From owner-freebsd-virtualization@freebsd.org Wed May 5 06:30:51 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 81FF7631BD2 for ; Wed, 5 May 2021 06:30:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4FZn0C34kWz3D3h for ; Wed, 5 May 2021 06:30:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 67E04631B4C; Wed, 5 May 2021 06:30:51 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 67A9F63146B for ; Wed, 5 May 2021 06:30:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZn0C2RByz3CqZ for ; Wed, 5 May 2021 06:30:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 472292003D for ; Wed, 5 May 2021 06:30:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1456UpjK085136 for ; Wed, 5 May 2021 06:30:51 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1456UpfS085135 for virtualization@FreeBSD.org; Wed, 5 May 2021 06:30:51 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 255571] vmx mtu 9000 performance issues Date: Wed, 05 May 2021 06:30:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2021 06:30:51 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D255571 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |virtualization@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Wed May 5 15:19:51 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BA30B5FA466 for ; Wed, 5 May 2021 15:19:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Fb0kb4kLDz4SF9 for ; Wed, 5 May 2021 15:19:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id A23525FA465; Wed, 5 May 2021 15:19:51 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A1F915FA24B for ; Wed, 5 May 2021 15:19:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fb0kb45ZYz4S26 for ; Wed, 5 May 2021 15:19:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7FB3826FEB for ; Wed, 5 May 2021 15:19:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 145FJp2W057537 for ; Wed, 5 May 2021 15:19:51 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 145FJpV5057536 for virtualization@FreeBSD.org; Wed, 5 May 2021 15:19:51 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 255571] vmx mtu 9000 no transmission Date: Wed, 05 May 2021 15:19:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: jcaplan@blackberry.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2021 15:19:51 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D255571 jcaplan@blackberry.com changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|vmx mtu 9000 performance |vmx mtu 9000 no |issues |transmission --- Comment #1 from jcaplan@blackberry.com --- After a bit more investigating, all segments are discarded on the target. in vmxnet3_isc_rxd_pkt_get(), this condition is reached for all incoming packets with large MTU: if (__predict_false(rxcd->error)) { rxc->vxcr_pkt_errors++; for (i =3D 0; i < nfrags; i++) { frag =3D &ri->iri_frags[i]; frag->irf_len =3D 0; } } As far as I can tell I've done everything on the host side with vmware to configure MTU properly. --=20 You are receiving this mail because: You are the assignee for the bug.=