From nobody Mon Oct 14 22:52:09 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XSCD55snYz5ZJ2X for ; Mon, 14 Oct 2024 22:52:21 +0000 (UTC) (envelope-from SRS0=YIY8=RK=freebsd.org=grehan@iredmail.onthenet.com.au) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XSCD201lTz44PJ for ; Mon, 14 Oct 2024 22:52:15 +0000 (UTC) (envelope-from SRS0=YIY8=RK=freebsd.org=grehan@iredmail.onthenet.com.au) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=nt.com.au header.s=dkim header.b=jdJbE1vf; spf=pass (mx1.freebsd.org: domain of "SRS0=YIY8=RK=freebsd.org=grehan@iredmail.onthenet.com.au" designates 203.13.68.12 as permitted sender) smtp.mailfrom="SRS0=YIY8=RK=freebsd.org=grehan@iredmail.onthenet.com.au"; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=freebsd.org (policy=none) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id 930C820B49A6 for ; Tue, 15 Oct 2024 08:52:09 +1000 (AEST) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 8CDD62127CB9 for ; Tue, 15 Oct 2024 08:52:09 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=nt.com.au; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:content-language:references:to:subject:subject :user-agent:mime-version:date:date:message-id; s=dkim; t= 1728946329; x=1731538330; bh=E+VvW8Rx9ayBaZ9PmlakWAO6cfMME/GTdGh XkV8IFsY=; b=jdJbE1vfTHynz6n9iA8ATraimRLlHzlJC+26y5emw1gkEy7CYH4 w3TA7NcjpVUm4b4mLki0J7w8/5spDgmCfmmjkzgPbxRD2LwZiyLa4GVUk5Wu1Tc4 k8I6Tlza76D4rzbPfnqtqrsvGY0NXbyXZnOvWJeEKtbMlpEa/JvuVK84= Received: from iredmail.onthenet.com.au ([127.0.0.1]) by iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ibX1fXl7u5Y7 for ; Tue, 15 Oct 2024 08:52:09 +1000 (AEST) Received: from [192.168.1.101] (otn-120-29-24-249.broadband.onthenet.net [120.29.24.249]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 68ED42127CB7; Tue, 15 Oct 2024 08:52:09 +1000 (AEST) Message-ID: <106b8500-a0ef-4095-af20-8c0f110ea739@freebsd.org> Date: Tue, 15 Oct 2024 08:52:09 +1000 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Running Mezzano in bhyve To: Vasily Postnicov Cc: freebsd-virtualization@freebsd.org References: <17f4077d-647d-4848-9d6f-97f9886ef636@freebsd.org> <8b249b64-d041-4f12-b6cb-fdb528837f22@freebsd.org> Content-Language: en-US From: Peter Grehan In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.4 cv=Fu4D/Xrq c=1 sm=1 tr=0 ts=670da099 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=0UtLtLgMz4NvdyIsuxvgLw==:17 a=IkcTkHD0fZMA:10 a=DAUX931o1VcA:10 a=bi0XHdcepdgA:10 a=H7MLjYanAAAA:8 a=1Wr8Lj4pOnmtSuG4wvkA:9 a=QEXdDO2ut3YA:10 a=bgSlMGY6Cwri7Y-TTqhu:22 X-Spamd-Result: default: False [-3.29 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; FORGED_SENDER(0.30)[grehan@freebsd.org,SRS0=YIY8=RK=freebsd.org=grehan@iredmail.onthenet.com.au]; R_SPF_ALLOW(-0.20)[+ip4:203.13.68.0/24]; R_DKIM_ALLOW(-0.20)[nt.com.au:s=dkim]; RCVD_IN_DNSWL_LOW(-0.10)[203.13.68.12:from]; RWL_MAILSPIKE_GOOD(-0.10)[203.13.68.12:from]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), DKIM not aligned (relaxed),none]; XM_UA_NO_VERSION(0.01)[]; ARC_NA(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[grehan@freebsd.org,SRS0=YIY8=RK=freebsd.org=grehan@iredmail.onthenet.com.au]; DKIM_TRACE(0.00)[nt.com.au:+]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; TAGGED_RCPT(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:9313, ipnet:203.13.68.0/24, country:AU]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[] X-Rspamd-Queue-Id: 4XSCD201lTz44PJ X-Spamd-Bar: --- > 1) The problem with PIT. Can be solved as you proposed or by > patching Mezzano. The bhyve patch would be the best option for that: it's useful for other older o/s's (DOS). > 2) Mezzano assumes that Intel AHCI controllers report no more than 6 > ports. Can be solved by patching Mezzano or defining MAX_PORTS to be > 6 in usr.sbin/bhyve/pci_ahci.c A Mezzano patch would be best for that. The bhyve man page has an example with 8 disks attached so reducing the limit to 6 could hit existing users. > 3) According to > https://wiki.osdev.org/PCI#Message_Signaled_Interrupts > , interrupt > line config register must be RW. Bhyve does not support writing to > it. I do not know a correct fix, this [1] workaround helps, however. Bhyve does support writing to that - your patch disables that, and my guess is that when Mezzano sees this as zero (ie invalid) it then looks for the irq line via the ACPI MADT (or other means). A quick look at Mezzano shows that it is still using the 8259 PIC for interrupts. At the minimum it should be using the IOAPIC, or excessive interrupt sharing will result, and possibly incorrect behaviour when this happens. I think IOAPIC support could be added without a large amount of effort, compared to e.g. MSI/MSI-x. > 4) Finally, I had a random deadlock in interrupt handling for the > virtio-net device. Likewise, I do not know how to fix it correctly, > but this [2] patch helped. Hmmm that seems strange: MSI interrupts aren't generated if they haven't been setup/enabled by a guest. Commenting out the lock/unlock code would seem to indicate a larger bug in play. Would it possible to get some tracing on that segment of code e.g. a dtrace log ? > Do you have any ideas how to make proper patches for bhyve from > these workarounds? The first one can be put in a phab diff, which I'll do. I think there's still some more work involved for the others. later, Peter.