From nobody Wed Oct 9 06:25:52 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XNjZR0WSyz5Ytpr for ; Wed, 09 Oct 2024 06:26:07 +0000 (UTC) (envelope-from shamaz.mazum@gmail.com) Received: from mail-lj1-x22b.google.com (mail-lj1-x22b.google.com [IPv6:2a00:1450:4864:20::22b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XNjZQ0q37z3wrm for ; Wed, 9 Oct 2024 06:26:06 +0000 (UTC) (envelope-from shamaz.mazum@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=FnBvO4uH; spf=pass (mx1.freebsd.org: domain of shamaz.mazum@gmail.com designates 2a00:1450:4864:20::22b as permitted sender) smtp.mailfrom=shamaz.mazum@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-lj1-x22b.google.com with SMTP id 38308e7fff4ca-2fad100dd9eso63784911fa.3 for ; Tue, 08 Oct 2024 23:26:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1728455164; x=1729059964; darn=freebsd.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=gKLcAORBEtp1gj4rIA+sUewr92awumFuDiAzA+Idbl4=; b=FnBvO4uHXSrfzrCtrL/D5s73lI6TsmgyE/BIfBVTcKZBVVlM/q2zWXzUVYmDlRyG6z Beu+/VxXGVuSBAlZejQs5dBXalSOlnXt0mRTV2svpvaUexX18e0mtbN5DN6qdB0jzisL 1V3NXub+bgQFDVwXt+C2nEC8pCDARiOo33OzdIz/GoNzD3CXo3ek7WeWsAyvmnuPFWYX ywz1/6EPJiWyYTLf1Cu8ay1xE2Wl5uUCBMtFlIy6prio1Qbt0zPatcOAAruro74VQo4o aZX/PjdvTiNWYfcNbe14pdrAkgm4040KkT23zUUZKgkniyrXgXMX93O3igiu9Z92X82e 4AQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728455164; x=1729059964; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=gKLcAORBEtp1gj4rIA+sUewr92awumFuDiAzA+Idbl4=; b=UeXACb3NLv23FL2Q0s3sIPCSbACuwBhrvd06O0LV2TulWQ1J5jjFdvQfGIREUNduTm 3BS3IW/ohCHkKywE3mqIWic258yKNjB/Zi2MRSTDQv38jCmK6gerqFhMjJccEmR4TL5Y EjtWcoVdKMFU5lntvAUWOsyswSPSd4JOUaJL/2RUJw5ULKGib7fwB99OxvyKfbldruTf KZh3QeXa+uqyH0tN54EeBFcw2y2BCjCukf+nusrq7XY0iKA/4FFsEnsTCORX0iZ0ZIQm SSzTNy508tJHeJ63bQC2LqPcbJGNPEgTZWHKqe0FYuelUzEugA91vG61aBGtqNTZO207 gI2Q== X-Gm-Message-State: AOJu0YwOoDK4lAsvtPztvwqiLn1kWkKigZC6oaT62z2JHZBnPnqgNJpw WPHgkKe6I7av07NJLS6qAt6i2UbOzSuCBj1fIttEefWAlhC8QqJ4VsaoKerMcjNK8cGQfiu2dXE HVPD5W0DVKFf7oK+l5es/HyDTtCSRQfb0vlwHSg== X-Google-Smtp-Source: AGHT+IFETO2HoVqpLYQaRvhdh4etfXxnSlIcSB7L4h74TRaRwo8RgDiH4t7OIVnTUHBSX7FfX4vsu0zZ23SwyDwb8ew= X-Received: by 2002:a05:6512:b93:b0:52c:d905:9645 with SMTP id 2adb3069b0e04-539c48989a8mr649856e87.13.1728455163534; Tue, 08 Oct 2024 23:26:03 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 From: Vasily Postnicov Date: Wed, 9 Oct 2024 06:25:52 +0000 Message-ID: Subject: Running Mezzano in bhyve To: freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="00000000000008657506240554fa" X-Spamd-Result: default: False [-3.99 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-0.99)[-0.987]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FREEMAIL_ENVFROM(0.00)[gmail.com]; TAGGED_FROM(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FROM_HAS_DN(0.00)[]; MISSING_XM_UA(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::22b:from] X-Rspamd-Queue-Id: 4XNjZQ0q37z3wrm X-Spamd-Bar: --- --00000000000008657506240554fa Content-Type: text/plain; charset="UTF-8" Hi! Mezzano is an operating system written in Common lisp. I would like to test it in bhyve. When I launch it, it spins infinitely in a loop measuring how many CPU cycles it makes in a unit of time (I understood it by adding various debug messages). The timer it uses is something called a PIT: https://wiki.osdev.org/Programmable_Interval_Timer The problem is, once the timer is configured it fires an interrupt only once, while it obviously must fire interrupts constantly at some frequency (100 Hz, as I understand from the code). Is the PIT supported by bhyve? Does FreeBSD use it anywhere in its kernel? BTW, Mezzano boots up fine on bare metal and in qemu, so the project is in a working state, despite being abandoned. --00000000000008657506240554fa Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi! Mezzano is an operating system written in Common lisp.= I would like to test it in bhyve. When I launch it, it spins infinitely in= a loop measuring how many CPU cycles it makes in a unit of time (I underst= ood it by adding various debug messages).

The timer it u= ses is something called a PIT:=C2=A0https://wiki.osdev.org/Programmable_Interval_Timer<= /a>



--00000000000008657506240554fa-- From nobody Wed Oct 9 11:20:17 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XNr651mTWz5YQ5s for ; Wed, 09 Oct 2024 11:20:29 +0000 (UTC) (envelope-from SRS0=jGWV=RF=freebsd.org=grehan@iredmail.onthenet.com.au) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XNr614PXmz4Pr0 for ; Wed, 9 Oct 2024 11:20:24 +0000 (UTC) (envelope-from SRS0=jGWV=RF=freebsd.org=grehan@iredmail.onthenet.com.au) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=nt.com.au header.s=dkim header.b=qQ8Rpf50; spf=pass (mx1.freebsd.org: domain of "SRS0=jGWV=RF=freebsd.org=grehan@iredmail.onthenet.com.au" designates 203.13.68.12 as permitted sender) smtp.mailfrom="SRS0=jGWV=RF=freebsd.org=grehan@iredmail.onthenet.com.au"; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=freebsd.org (policy=none) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id 592C020B49AC for ; Wed, 9 Oct 2024 21:20:18 +1000 (AEST) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 535D52127CB9 for ; Wed, 9 Oct 2024 21:20:18 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=nt.com.au; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:references:to:content-language:subject:subject :user-agent:mime-version:date:date:message-id; s=dkim; t= 1728472818; x=1731064819; bh=wlNrALY7DURxv7Xij0Y5eQ/k6wSkYNTfcMB zSO1Rfb4=; b=qQ8Rpf50LbSdIriPelD+qPcBmsqUdUHmhj/mCBjlHWiVYWL3tR+ LwxQAVMhu7nsjuG6q4tu1mQ/nT+mhIsgABGz0bXgl9hIftGQA6vHcB0zp+QdQHWa dQwpFbmlwsFVqN2O1AOq0JBAFrF2m5yyyt2ISok+vg4D82XuUfBwxYM0= Received: from iredmail.onthenet.com.au ([127.0.0.1]) by iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id q7pGFfOcO9YS for ; Wed, 9 Oct 2024 21:20:18 +1000 (AEST) Received: from [192.168.1.101] (otn-120-29-24-249.broadband.onthenet.net [120.29.24.249]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 2CC0D2127CB7; Wed, 9 Oct 2024 21:20:18 +1000 (AEST) Message-ID: <17f4077d-647d-4848-9d6f-97f9886ef636@freebsd.org> Date: Wed, 9 Oct 2024 21:20:17 +1000 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Running Mezzano in bhyve Content-Language: en-US To: Vasily Postnicov References: Cc: freebsd-virtualization@freebsd.org From: Peter Grehan In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.4 cv=Fu4D/Xrq c=1 sm=1 tr=0 ts=670666f2 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=0UtLtLgMz4NvdyIsuxvgLw==:17 a=IkcTkHD0fZMA:10 a=DAUX931o1VcA:10 a=bi0XHdcepdgA:10 a=H7MLjYanAAAA:8 a=NEAV23lmAAAA:8 a=6I5d2MoRAAAA:8 a=mIvVm0WCNU7gLV3VHwkA:9 a=QEXdDO2ut3YA:10 a=f_Ahs_8vY64A:10 a=bgSlMGY6Cwri7Y-TTqhu:22 X-Spamd-Result: default: False [-3.28 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.992]; FORGED_SENDER(0.30)[grehan@freebsd.org,SRS0=jGWV=RF=freebsd.org=grehan@iredmail.onthenet.com.au]; R_SPF_ALLOW(-0.20)[+ip4:203.13.68.0/24]; R_DKIM_ALLOW(-0.20)[nt.com.au:s=dkim]; RCVD_IN_DNSWL_LOW(-0.10)[203.13.68.12:from]; RWL_MAILSPIKE_GOOD(-0.10)[203.13.68.12:from]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), DKIM not aligned (relaxed),none]; XM_UA_NO_VERSION(0.01)[]; ARC_NA(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[grehan@freebsd.org,SRS0=jGWV=RF=freebsd.org=grehan@iredmail.onthenet.com.au]; DKIM_TRACE(0.00)[nt.com.au:+]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; TAGGED_RCPT(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:9313, ipnet:203.13.68.0/24, country:AU]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[] X-Rspamd-Queue-Id: 4XNr614PXmz4Pr0 X-Spamd-Bar: --- > Hi! Mezzano is an operating system written in Common lisp. I would like > to test it in bhyve. When I launch it, it spins infinitely in a loop > measuring how many CPU cycles it makes in a unit of time (I understood > it by adding various debug messages). > > The timer it uses is something called a PIT: > https://wiki.osdev.org/Programmable_Interval_Timer > > > The problem is, once the timer is configured it fires an interrupt only > once, while it obviously must fire interrupts constantly at some > frequency (100 Hz, as I understand from the code). > > Is the PIT supported by bhyve? Does FreeBSD use it anywhere in its kernel? Yes, though mode 3 of the timer isn't supported by the device emulation. It appears that Mezzano uses that mode https://github.com/froggey/Mezzano/blob/master/supervisor/x86-64/time.lisp#L20 Try this patch to the PIT code to see if it helps https://people.freebsd.org/~grehan/bhyve_vatpit.diff later, Peter. From nobody Wed Oct 9 12:18:42 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XNsPY5MMvz5YSVs for ; Wed, 09 Oct 2024 12:18:57 +0000 (UTC) (envelope-from shamaz.mazum@gmail.com) Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com [IPv6:2a00:1450:4864:20::129]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XNsPY12nPz4X50; Wed, 9 Oct 2024 12:18:57 +0000 (UTC) (envelope-from shamaz.mazum@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-x129.google.com with SMTP id 2adb3069b0e04-5398cc2fcb7so7749165e87.1; Wed, 09 Oct 2024 05:18:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1728476334; x=1729081134; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=PWpOdIZaM5a0jnC4dowOPP2AZ2HtGqeeLzeGJV8AgAs=; b=cI5RfuPHc9Oi4WslYA5UBYZ7r5bgh6sTS03TBo3mmHgVmQ4ZJIrZI9n9Lh4H/rAZys 8iXUK9a6OMdaJjkzmC7xIKnD8QGxKOljppJ8862IG+9pwB3SWoxJqlw1tlWbJffy/j1E PcKd9ENuM0RfMh4cvkTTBa2zi3Dp6q+RKtEralSeqUn6TnxjBv76VqCMWx6/vvHWAl+X rv9cBn3q1hIJtlckA+HmI68p8P0QsfnVUWzd3obBlvqtuhhe1hgllD31fwUM0BoLwor5 1s252VbDZWk+BUGD82WoCpQDX4wpP+oYmBvONfdpGfG42PvrE3VOtAR8eCJSvVR+9vok ndrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728476334; x=1729081134; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=PWpOdIZaM5a0jnC4dowOPP2AZ2HtGqeeLzeGJV8AgAs=; b=CRe5TmD3vpzJh36tICyJtQM5og+TF0Hgi/alq0DjiPcksDBs87wXQHXsZi/rtxeimr /6DhKw6dO7vKa5KhkzftQaXeIXaP6LA/nbEWRaMOpNVVH0AnaKo2fwPSkwqCpvzIdPVW osXqZVVqnzZJjdIUWAmLAs/i72JdJOiMhIQgfLg75col54c/ZaInwJ+alFy5r0qvDr69 5PpKDfzAuBKEEAvIgAGzMXZKOospL2azI2vtf9LioFkAnW0ElwO4M13SiXZMkgRkjFPj rRWKA1Dpcv2jCo8WFIGv+5MuYbNRPY/ppfJbAsYo5v6O1T4EtHOMNUFHwh6ZhQJF3w7o oR1Q== X-Gm-Message-State: AOJu0YzsVPalHvp7kX67arleCMClrztdG0sV4k4Pb8vsi79eUp8bqc8c Iz2DA1xS8FeRiJTKc7lTJd5Do3TnHztMSAAMnHtuOzZviuRBN+ZazM9wsrU6zj+4SixmJhpbHtd s5FnkH3qTORSqGwuwLG3vr6sS3UTcP/koacc= X-Google-Smtp-Source: AGHT+IHFejBCVBbhg1pio6x+VqWvAWnDTAUAq/gaf/VKHzPU+gCjqHans89gl2ffA/iAPWsbSfMu12rQZsHmUjDyx/A= X-Received: by 2002:a05:6512:a8f:b0:535:d4e9:28c1 with SMTP id 2adb3069b0e04-539c48c47a2mr1236137e87.20.1728476333930; Wed, 09 Oct 2024 05:18:53 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 References: <17f4077d-647d-4848-9d6f-97f9886ef636@freebsd.org> In-Reply-To: <17f4077d-647d-4848-9d6f-97f9886ef636@freebsd.org> From: Vasily Postnicov Date: Wed, 9 Oct 2024 12:18:42 +0000 Message-ID: Subject: Re: Running Mezzano in bhyve To: Peter Grehan Cc: freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="000000000000e3012f06240a41fd" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4XNsPY12nPz4X50 X-Spamd-Bar: ---- --000000000000e3012f06240a41fd Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks! It worked, but Mezanno crashes after a short period of time: Begin PCI scan. PCI:0:1F:0 8086:7000 Intel Corporation - 82371SB PIIX3 ISA [Natoma/Triton II] 6:1:0 rid: 0 hdr: 0 intr: FF PCI:0:1D:0 FB5D:40FB NIL - NIL 3:0:0 rid: 0 hdr: 0 intr: FF 40: MSI 84 PCI:0:5:0 1AF4:1000 Red Hat, Inc - Virtio network device 2:0:0 rid: 0 hdr: 0 intr: A 40: Unknown capability 11 4C: MSI 80 PCI:0:3:0 8086:2821 Intel Corporation - 82801HR/HO/HH (ICH8R/DO/DH) 6 port SATA Controller [AHCI mode] 1:6:1 rid: 0 hdr: 0 intr: B 40: MSI 88 Detected AHCI ABAR at C1002000 AHCI IRQ is B Host Capabilities FF30FF3F Global Host Control 80000000 Interrupt Status 0 Ports Implemented 1 Version 10300 Command Completion Coalescing Control 0 Command Completion Coalescing Ports 0 Enclosure Management Location 0 Enclosure Management Control 0 Host Capabilities Extended 4 BIOS/OS Handoff Control and Status 0 AHCI HBA version 1.300 Handler: 0 Config register: 17 ----- PANIC ----- Early page fault on address 80000035B0 Local CPU is #<7A4AF9> Run queues: Thread #<1468079 Thread Initial thread> holds the world Run queue #<1498139>/SUPERVISOR: Run queue #<1498159>/HIGH: Run queue #<1498179>/NORMAL: Run queue #<1498199>/LOW: IRQ state: IRQ #<1497DF9 Irq :Number 0> - 0 (86 delivered) #<1499119> PIT [exclusive] IRQ #<1497E29 Irq :Number 1> - 1 (1 delivered) #<1499699> INTEL-8042-KEYBOARD [exclusive] IRQ #<1498009 Irq :Number B> - B (0 delivered) #<149C959> #<149C909> IRQ #<1498039 Irq :Number C> - C (0 delivered) #<14993D9> INTEL-8042-AUXILIARY [exclusive] Active timers: (current time is 147260) Thread #<1468079 Thread Initial thread> ACTIVE #<1498669 Wait-Queue Heartbeat wait queue> TIFH: 1 TPFH: NIL 20000121FB90 7FFF892B07 DEBUG-DUMP 20000121FD20 7FFF800FAE (LAMBDA IN PANIC-1) 20000121FE30 7FFF80073F PANIC-1 20000121FE50 7FFF800633 PANIC 20000121FEC0 7FFF8820B1 FATAL-PAGE-FAULT 20000121FEF0 7FFF88114B %PAGE-FAULT-HANDLER 20000121FF40 7FFF87F3BB %%INTERRUPT-SERVICE-ROUTINES 20000121FFD0 80000035B0 # 200000A1FFF0 7FFF800841 %CALL-ON-WIRED-STACK-WITHOUT-INTERRUPTS 20000221FE60 7FFF958E90 (SETF PCI-CONFIG/16) 20000221FE90 7FFF9B5670 AHCI-PCI-REGISTER 20000221FEF0 7FFF95B9C9 (FLET DX-LAMBDA IN PCI-DETECT) 20000221FF50 7FFF95B22E MAP-PCI-DEVICES 20000221FF90 7FFF955671 PCI-DETECT 20000221FFA0 7FFF93C8F4 INITIALIZE-PLATFORM 20000221FFC0 7FFF8D7F03 BOOTLOADER-ENTRY-POINT Looks like it executes an interrupt handler with calling something "on a wired stack without interrupts". Postmortem rflags: root@vonbraun:~ # bhyvectl --vm=3Dubuntu --get-rflags rflags[0] 0x0000000000000046 I do not know what to think, that %cal-on-wired-stack-without-interrupts seems to be legit to me: https://github.com/froggey/Mezzano/blob/master/supervisor/x86-64/interrupts= .lisp Anyway, thanks again for the help! =D1=81=D1=80, 9 =D0=BE=D0=BA=D1=82. 2024=E2=80=AF=D0=B3. =D0=B2 11:20, Pete= r Grehan : > > Hi! Mezzano is an operating system written in Common lisp. I would like > > to test it in bhyve. When I launch it, it spins infinitely in a loop > > measuring how many CPU cycles it makes in a unit of time (I understood > > it by adding various debug messages). > > > > The timer it uses is something called a PIT: > > https://wiki.osdev.org/Programmable_Interval_Timer > > > > > > The problem is, once the timer is configured it fires an interrupt only > > once, while it obviously must fire interrupts constantly at some > > frequency (100 Hz, as I understand from the code). > > > > Is the PIT supported by bhyve? Does FreeBSD use it anywhere in its > kernel? > > Yes, though mode 3 of the timer isn't supported by the device > emulation. It appears that Mezzano uses that mode > > > https://github.com/froggey/Mezzano/blob/master/supervisor/x86-64/time.lis= p#L20 > > Try this patch to the PIT code to see if it helps > https://people.freebsd.org/~grehan/bhyve_vatpit.diff > > later, > > Peter. > > --000000000000e3012f06240a41fd Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks! It worked, but Mezanno crashes af= ter a short=C2=A0period of time:

Begin PCI scan.
PCI:= 0:1F:0 8086:7000 Intel Corporation - 82371SB PIIX3 ISA [Natoma/Triton II] 6= :1:0 rid: 0 hdr: 0 intr: FF
PCI:0:1D:0 FB5D:40FB NIL - NIL 3:0:0 rid: 0 = hdr: 0 intr: FF
=C2=A0 =C2=A0 40: MSI 84
PCI:0:5:0 1AF4:1000 Red Hat,= Inc - Virtio network device 2:0:0 rid: 0 hdr: 0 intr: A
=C2=A0 =C2=A0 4= 0: Unknown capability 11
=C2=A0 =C2=A0 4C: MSI 80
PCI:0:3:0 8086:2821= Intel Corporation - 82801HR/HO/HH (ICH8R/DO/DH) 6 port SATA Controller [AH= CI mode] 1:6:1 rid: 0 hdr: 0 intr: B
=C2=A0 =C2=A0 40: MSI 88
Detecte= d AHCI ABAR at C1002000
AHCI IRQ is B
Host Capabilities FF30FF3F
G= lobal Host Control 80000000
Interrupt Status 0
Ports Implemented 1Version 10300
Command Completion Coalescing Control 0
Command Comple= tion Coalescing Ports 0
Enclosure Management Location 0
Enclosure Man= agement Control 0
Host Capabilities Extended 4
BIOS/OS Handoff Contro= l and Status 0
AHCI HBA version 1.300
Handler: 0
Config register: = 17
----- PANIC -----
Early page fault on address 80000035B0
Local = CPU is #<7A4AF9>
Run queues:
Thread #<1468079 Thread Initial= thread> holds the world
Run queue #<1498139>/SUPERVISOR:
Ru= n queue #<1498159>/HIGH:
Run queue #<1498179>/NORMAL:
Run= queue #<1498199>/LOW:
IRQ state:
IRQ #<1497DF9 Irq :Number = 0> - 0 (86 delivered)
=C2=A0 #<1499119> PIT [exclusive]
IRQ = #<1497E29 Irq :Number 1> - 1 (1 delivered)
=C2=A0 #<1499699>= INTEL-8042-KEYBOARD [exclusive]
IRQ #<1498009 Irq :Number B> - B = (0 delivered)
=C2=A0 #<149C959> #<149C909>
IRQ #<14980= 39 Irq :Number C> - C (0 delivered)
=C2=A0 #<14993D9> INTEL-804= 2-AUXILIARY [exclusive]
Active timers: (current time is 147260)
Threa= d #<1468079 Thread Initial thread> ACTIVE #<1498669 Wait-Queue Hea= rtbeat wait queue>
TIFH: 1 TPFH: NIL
20000121FB90 7FFF892B07 DEBUG= -DUMP
20000121FD20 7FFF800FAE (LAMBDA IN PANIC-1)
20000121FE30 7FFF80= 073F PANIC-1
20000121FE50 7FFF800633 PANIC
20000121FEC0 7FFF8820B1 FA= TAL-PAGE-FAULT
20000121FEF0 7FFF88114B %PAGE-FAULT-HANDLER
20000121FF= 40 7FFF87F3BB %%INTERRUPT-SERVICE-ROUTINES
20000121FFD0 80000035B0 #<= unknown>
200000A1FFF0 7FFF800841 %CALL-ON-WIRED-STACK-WITHOUT-INTERRU= PTS
20000221FE60 7FFF958E90 (SETF PCI-CONFIG/16)
20000221FE90 7FFF9B5= 670 AHCI-PCI-REGISTER
20000221FEF0 7FFF95B9C9 (FLET DX-LAMBDA IN PCI-DET= ECT)
20000221FF50 7FFF95B22E MAP-PCI-DEVICES
20000221FF90 7FFF955671 = PCI-DETECT
20000221FFA0 7FFF93C8F4 INITIALIZE-PLATFORM
20000221FFC0 7= FFF8D7F03 BOOTLOADER-ENTRY-POINT

Looks like it= executes an interrupt handler with calling something "on a wired stac= k without interrupts". Postmortem rflags:

=D1=81=D1=80, 9 =D0= =BE=D0=BA=D1=82. 2024=E2=80=AF=D0=B3. =D0=B2 11:20, Peter Grehan <grehan@freebsd.org>:
> Hi! Mezzano is an operating system written in Common lisp. = I would like
> to test it in bhyve. When I launch it, it spins infinitely in a loop <= br> > measuring how many CPU cycles it makes in a unit of time (I understood=
> it by adding various debug messages).
>
> The timer it uses is something called a PIT:
> https://wiki.osdev.org/Programmable_Interval_= Timer
> <https://wiki.osdev.org/Programmable_Inter= val_Timer>
>
> The problem is, once the timer is configured it fires an interrupt onl= y
> once, while it obviously must fire interrupts constantly at some
> frequency (100 Hz, as I understand from the code).
>
> Is the PIT supported by bhyve? Does FreeBSD use it anywhere in its ker= nel?

=C2=A0 Yes, though mode 3 of the timer isn't supported by the device emulation. It appears that Mezzano uses that mode

https://github.com/fro= ggey/Mezzano/blob/master/supervisor/x86-64/time.lisp#L20

=C2=A0 Try this patch to the PIT code to see if it helps
=C2=A0 =C2=A0 =C2=A0https://people.freebsd.org/~= grehan/bhyve_vatpit.diff

later,

Peter.

--000000000000e3012f06240a41fd-- From nobody Wed Oct 9 15:22:43 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XNxTv52zjz5Yfp9 for ; Wed, 09 Oct 2024 15:22:59 +0000 (UTC) (envelope-from shamaz.mazum@gmail.com) Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com [IPv6:2a00:1450:4864:20::131]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XNxTs10jXz4p7n; Wed, 9 Oct 2024 15:22:57 +0000 (UTC) (envelope-from shamaz.mazum@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=kzERx3Wu; spf=pass (mx1.freebsd.org: domain of shamaz.mazum@gmail.com designates 2a00:1450:4864:20::131 as permitted sender) smtp.mailfrom=shamaz.mazum@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-lf1-x131.google.com with SMTP id 2adb3069b0e04-53988c54ec8so8229238e87.0; Wed, 09 Oct 2024 08:22:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1728487375; x=1729092175; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=mhdPuyLC0+7VYwR8lKpMmcoDbmwduvCzZV8i3kDuNPE=; b=kzERx3Wuy90mPtqRnyTup7akdXZpT2K6ieQlDoS05INg8BxUEJZiJEFTvqm2nCW/jN OgIAJVO6tpqhutUbT6/Mp5soDLYXTkm5nzK8SwaLlQ+rZqVU4YQmXRifF/8mouc+h3eY GQIia1x9gLPbQH0npqJZWeCvjayxOMdNgv0gqqKRdm7j4bU6Uiq0cW5Diymqd+PxNVTV WFMEJPZpyHKyFSZ6Uoqimk0VWT9Xo0CObAawF8F+RPLRvfSNi/rDyr7czZtiDrUVMjDE SXtpnrSO/k+3uTxjcewASlar8B+ulBOUvpWazkLp1S44yu2OF+7HSHw56LZkUooffqK0 lQag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728487375; x=1729092175; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=mhdPuyLC0+7VYwR8lKpMmcoDbmwduvCzZV8i3kDuNPE=; b=gHa5EKekkJedHqjrIAy+HkGFieBoht5HEdLzKol2xcG0EfWVt2GSMozj1/FufY6uCH mk6IN3oW0ZJnTrzzVQ4J/NbmQr2h2ycOPGsEUYUE4/Vwgn5f5N3yhIM7jKKIs9zSNWho 0dfxurHAv8ZHdVjhEUyQCvia78b2qJIscA5p/VQPaFZUd9akGE4926G1mv6ev9EZv0mH xLJz0fQyZ+ap1HMLLF7boCT8hKu0GQC+lH1fz7MgPY41OPKtf75p3bIT+yv1k7a8AoxV mvawbEaNAYXMzuxJpR/2BuKUmVHF4WMRisgZ419EwwazLfTSILsDaF7AoYku80ipYwl6 Q0Vg== X-Gm-Message-State: AOJu0Yx5CeDRVI5HEAzorPhEetTgT1FBL8/C4x/w/+qVFnS1FHtxy7ih PtI/xJ506QU7MHGLWrJZo4nO1mvaCUWGuAWfo5u+G3NvFFMIEpwhEpaWsOMc8+uJbDoft8BK/DQ +2xsrpJ/dN5EcDKCZnfAnC7fl05yhp4XL X-Google-Smtp-Source: AGHT+IGi007blXc5amWrxeayWhuE93/NXk5iCCBgegYt5bsMHBgQjNtydRzciMYmeFMwvZhbI/+oQycZIjWlObUncno= X-Received: by 2002:a05:6512:1096:b0:539:959e:f0e8 with SMTP id 2adb3069b0e04-539c4897565mr1614471e87.21.1728487374341; Wed, 09 Oct 2024 08:22:54 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 References: <17f4077d-647d-4848-9d6f-97f9886ef636@freebsd.org> In-Reply-To: From: Vasily Postnicov Date: Wed, 9 Oct 2024 15:22:43 +0000 Message-ID: Subject: Re: Running Mezzano in bhyve To: Peter Grehan Cc: freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="000000000000f2501306240cd3e0" X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.995]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; ARC_NA(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; RCVD_TLS_LAST(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROMTLD(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; TAGGED_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MISSING_XM_UA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::131:from] X-Rspamd-Queue-Id: 4XNxTs10jXz4p7n X-Spamd-Bar: --- --000000000000f2501306240cd3e0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Turns out, panic always happens in the same place when Mezzano is trying to initialize an AHCI controller. Qemu uses an old PATA controller by default, which works. When forcing SATA, it fails just like bhyve. Virtio block devices are also claimed to work, but using virtio-blk option in bhyve also results in panic. When I tried Mezzano on my PC (bare metal), I used a USB stick. Can a USB stick be emulated in bhyve? I guess, only using USB controller passthrough? =D1=81=D1=80, 9 =D0=BE=D0=BA=D1=82. 2024=E2=80=AF=D0=B3. =D0=B2 12:18, Vasi= ly Postnicov : > Thanks! It worked, but Mezanno crashes after a short period of time: > > Begin PCI scan. > PCI:0:1F:0 8086:7000 Intel Corporation - 82371SB PIIX3 ISA [Natoma/Triton > II] 6:1:0 rid: 0 hdr: 0 intr: FF > PCI:0:1D:0 FB5D:40FB NIL - NIL 3:0:0 rid: 0 hdr: 0 intr: FF > 40: MSI 84 > PCI:0:5:0 1AF4:1000 Red Hat, Inc - Virtio network device 2:0:0 rid: 0 hdr= : > 0 intr: A > 40: Unknown capability 11 > 4C: MSI 80 > PCI:0:3:0 8086:2821 Intel Corporation - 82801HR/HO/HH (ICH8R/DO/DH) 6 por= t > SATA Controller [AHCI mode] 1:6:1 rid: 0 hdr: 0 intr: B > 40: MSI 88 > Detected AHCI ABAR at C1002000 > AHCI IRQ is B > Host Capabilities FF30FF3F > Global Host Control 80000000 > Interrupt Status 0 > Ports Implemented 1 > Version 10300 > Command Completion Coalescing Control 0 > Command Completion Coalescing Ports 0 > Enclosure Management Location 0 > Enclosure Management Control 0 > Host Capabilities Extended 4 > BIOS/OS Handoff Control and Status 0 > AHCI HBA version 1.300 > Handler: 0 > Config register: 17 > ----- PANIC ----- > Early page fault on address 80000035B0 > Local CPU is #<7A4AF9> > Run queues: > Thread #<1468079 Thread Initial thread> holds the world > Run queue #<1498139>/SUPERVISOR: > Run queue #<1498159>/HIGH: > Run queue #<1498179>/NORMAL: > Run queue #<1498199>/LOW: > IRQ state: > IRQ #<1497DF9 Irq :Number 0> - 0 (86 delivered) > #<1499119> PIT [exclusive] > IRQ #<1497E29 Irq :Number 1> - 1 (1 delivered) > #<1499699> INTEL-8042-KEYBOARD [exclusive] > IRQ #<1498009 Irq :Number B> - B (0 delivered) > #<149C959> #<149C909> > IRQ #<1498039 Irq :Number C> - C (0 delivered) > #<14993D9> INTEL-8042-AUXILIARY [exclusive] > Active timers: (current time is 147260) > Thread #<1468079 Thread Initial thread> ACTIVE #<1498669 Wait-Queue > Heartbeat wait queue> > TIFH: 1 TPFH: NIL > 20000121FB90 7FFF892B07 DEBUG-DUMP > 20000121FD20 7FFF800FAE (LAMBDA IN PANIC-1) > 20000121FE30 7FFF80073F PANIC-1 > 20000121FE50 7FFF800633 PANIC > 20000121FEC0 7FFF8820B1 FATAL-PAGE-FAULT > 20000121FEF0 7FFF88114B %PAGE-FAULT-HANDLER > 20000121FF40 7FFF87F3BB %%INTERRUPT-SERVICE-ROUTINES > 20000121FFD0 80000035B0 # > 200000A1FFF0 7FFF800841 %CALL-ON-WIRED-STACK-WITHOUT-INTERRUPTS > 20000221FE60 7FFF958E90 (SETF PCI-CONFIG/16) > 20000221FE90 7FFF9B5670 AHCI-PCI-REGISTER > 20000221FEF0 7FFF95B9C9 (FLET DX-LAMBDA IN PCI-DETECT) > 20000221FF50 7FFF95B22E MAP-PCI-DEVICES > 20000221FF90 7FFF955671 PCI-DETECT > 20000221FFA0 7FFF93C8F4 INITIALIZE-PLATFORM > 20000221FFC0 7FFF8D7F03 BOOTLOADER-ENTRY-POINT > > Looks like it executes an interrupt handler with calling something "on a > wired stack without interrupts". Postmortem rflags: > root@vonbraun:~ # bhyvectl --vm=3Dubuntu --get-rflags > rflags[0] 0x0000000000000046 > > I do not know what to think, that %cal-on-wired-stack-without-interrupts > seems to be legit to me: > https://github.com/froggey/Mezzano/blob/master/supervisor/x86-64/interrup= ts.lisp > Anyway, thanks again for the help! > > =D1=81=D1=80, 9 =D0=BE=D0=BA=D1=82. 2024=E2=80=AF=D0=B3. =D0=B2 11:20, Pe= ter Grehan : > >> > Hi! Mezzano is an operating system written in Common lisp. I would lik= e >> > to test it in bhyve. When I launch it, it spins infinitely in a loop >> > measuring how many CPU cycles it makes in a unit of time (I understood >> > it by adding various debug messages). >> > >> > The timer it uses is something called a PIT: >> > https://wiki.osdev.org/Programmable_Interval_Timer >> > >> > >> > The problem is, once the timer is configured it fires an interrupt onl= y >> > once, while it obviously must fire interrupts constantly at some >> > frequency (100 Hz, as I understand from the code). >> > >> > Is the PIT supported by bhyve? Does FreeBSD use it anywhere in its >> kernel? >> >> Yes, though mode 3 of the timer isn't supported by the device >> emulation. It appears that Mezzano uses that mode >> >> >> https://github.com/froggey/Mezzano/blob/master/supervisor/x86-64/time.li= sp#L20 >> >> Try this patch to the PIT code to see if it helps >> https://people.freebsd.org/~grehan/bhyve_vatpit.diff >> >> later, >> >> Peter. >> >> --000000000000f2501306240cd3e0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Turns out, panic always happens in the same place when Mez= zano is trying to initialize an AHCI controller. Qemu uses an old PATA cont= roller by=C2=A0default, which works. When forcing SATA, it fails just like = bhyve. Virtio block devices are also claimed to work, but using virtio-blk = option in bhyve also results in panic. When I tried Mezzano on my PC (bare = metal), I used a USB stick. Can a USB stick be emulated in bhyve? I guess, = only using USB controller passthrough?

=
=D1=81=D1=80, 9 =D0=BE=D0=BA=D1=82. 2= 024=E2=80=AF=D0=B3. =D0=B2 12:18, Vasily Postnicov <shamaz.mazum@gmail.com>:
Thanks! It worked, but Mezanno crash= es after a short=C2=A0period of time:

Begin PCI scan.PCI:0:1F:0 8086:7000 Intel Corporation - 82371SB PIIX3 ISA [Natoma/Triton = II] 6:1:0 rid: 0 hdr: 0 intr: FF
PCI:0:1D:0 FB5D:40FB NIL - NIL 3:0:0 ri= d: 0 hdr: 0 intr: FF
=C2=A0 =C2=A0 40: MSI 84
PCI:0:5:0 1AF4:1000 Red= Hat, Inc - Virtio network device 2:0:0 rid: 0 hdr: 0 intr: A
=C2=A0 =C2= =A0 40: Unknown capability 11
=C2=A0 =C2=A0 4C: MSI 80
PCI:0:3:0 8086= :2821 Intel Corporation - 82801HR/HO/HH (ICH8R/DO/DH) 6 port SATA Controlle= r [AHCI mode] 1:6:1 rid: 0 hdr: 0 intr: B
=C2=A0 =C2=A0 40: MSI 88
De= tected AHCI ABAR at C1002000
AHCI IRQ is B
Host Capabilities FF30FF3F=
Global Host Control 80000000
Interrupt Status 0
Ports Implemented= 1
Version 10300
Command Completion Coalescing Control 0
Command C= ompletion Coalescing Ports 0
Enclosure Management Location 0
Enclosur= e Management Control 0
Host Capabilities Extended 4
BIOS/OS Handoff C= ontrol and Status 0
AHCI HBA version 1.300
Handler: 0
Config regis= ter: 17
----- PANIC -----
Early page fault on address 80000035B0
L= ocal CPU is #<7A4AF9>
Run queues:
Thread #<1468079 Thread In= itial thread> holds the world
Run queue #<1498139>/SUPERVISOR:<= br>Run queue #<1498159>/HIGH:
Run queue #<1498179>/NORMAL:Run queue #<1498199>/LOW:
IRQ state:
IRQ #<1497DF9 Irq :Nu= mber 0> - 0 (86 delivered)
=C2=A0 #<1499119> PIT [exclusive]IRQ #<1497E29 Irq :Number 1> - 1 (1 delivered)
=C2=A0 #<149969= 9> INTEL-8042-KEYBOARD [exclusive]
IRQ #<1498009 Irq :Number B>= - B (0 delivered)
=C2=A0 #<149C959> #<149C909>
IRQ #<= 1498039 Irq :Number C> - C (0 delivered)
=C2=A0 #<14993D9> INTE= L-8042-AUXILIARY [exclusive]
Active timers: (current time is 147260)
= Thread #<1468079 Thread Initial thread> ACTIVE #<1498669 Wait-Queu= e Heartbeat wait queue>
TIFH: 1 TPFH: NIL
20000121FB90 7FFF892B07 = DEBUG-DUMP
20000121FD20 7FFF800FAE (LAMBDA IN PANIC-1)
20000121FE30 7= FFF80073F PANIC-1
20000121FE50 7FFF800633 PANIC
20000121FEC0 7FFF8820= B1 FATAL-PAGE-FAULT
20000121FEF0 7FFF88114B %PAGE-FAULT-HANDLER
20000= 121FF40 7FFF87F3BB %%INTERRUPT-SERVICE-ROUTINES
20000121FFD0 80000035B0 = #<unknown>
200000A1FFF0 7FFF800841 %CALL-ON-WIRED-STACK-WITHOUT-IN= TERRUPTS
20000221FE60 7FFF958E90 (SETF PCI-CONFIG/16)
20000221FE90 7F= FF9B5670 AHCI-PCI-REGISTER
20000221FEF0 7FFF95B9C9 (FLET DX-LAMBDA IN PC= I-DETECT)
20000221FF50 7FFF95B22E MAP-PCI-DEVICES
20000221FF90 7FFF95= 5671 PCI-DETECT
20000221FFA0 7FFF93C8F4 INITIALIZE-PLATFORM
20000221F= FC0 7FFF8D7F03 BOOTLOADER-ENTRY-POINT

Looks li= ke it executes an interrupt handler with calling something "on a wired= stack without interrupts". Postmortem rflags:
root@vonbraun= :~ # bhyvectl --vm=3Dubuntu --get-rflags
rflags[0] 0x0000000000000046

I do not know what to think, that %cal-on-wired-= stack-without-interrupts seems to be legit to me:=C2=A0https://github.com/froggey/Mezzano/blob/master/supervisor/x= 86-64/interrupts.lisp
Anyway, thanks again for the help!

=D1=81=D1=80, 9 =D0=BE=D0=BA=D1=82. 2024=E2=80=AF=D0=B3. =D0=B2 11:20, Pet= er Grehan <greha= n@freebsd.org>:
> Hi! Mezzano is an ope= rating system written in Common lisp. I would like
> to test it in bhyve. When I launch it, it spins infinitely in a loop <= br> > measuring how many CPU cycles it makes in a unit of time (I understood=
> it by adding various debug messages).
>
> The timer it uses is something called a PIT:
> https://wiki.osdev.org/Programmable_Interval_= Timer
> <https://wiki.osdev.org/Programmable_Inter= val_Timer>
>
> The problem is, once the timer is configured it fires an interrupt onl= y
> once, while it obviously must fire interrupts constantly at some
> frequency (100 Hz, as I understand from the code).
>
> Is the PIT supported by bhyve? Does FreeBSD use it anywhere in its ker= nel?

=C2=A0 Yes, though mode 3 of the timer isn't supported by the device emulation. It appears that Mezzano uses that mode

https://github.com/fro= ggey/Mezzano/blob/master/supervisor/x86-64/time.lisp#L20

=C2=A0 Try this patch to the PIT code to see if it helps
=C2=A0 =C2=A0 =C2=A0https://people.freebsd.org/~= grehan/bhyve_vatpit.diff

later,

Peter.

--000000000000f2501306240cd3e0-- From nobody Thu Oct 10 02:33:09 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XPDMM6Fv0z5ZM16 for ; Thu, 10 Oct 2024 02:33:19 +0000 (UTC) (envelope-from SRS0=SYqd=RG=freebsd.org=grehan@iredmail.onthenet.com.au) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XPDMJ5CF8z49WR for ; Thu, 10 Oct 2024 02:33:14 +0000 (UTC) (envelope-from SRS0=SYqd=RG=freebsd.org=grehan@iredmail.onthenet.com.au) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=nt.com.au header.s=dkim header.b=Ee+BWgZw; spf=pass (mx1.freebsd.org: domain of "SRS0=SYqd=RG=freebsd.org=grehan@iredmail.onthenet.com.au" designates 203.13.68.12 as permitted sender) smtp.mailfrom="SRS0=SYqd=RG=freebsd.org=grehan@iredmail.onthenet.com.au"; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=freebsd.org (policy=none) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id A198D20B49B3 for ; Thu, 10 Oct 2024 12:33:10 +1000 (AEST) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 9829C2127CBD for ; Thu, 10 Oct 2024 12:33:10 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=nt.com.au; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:references:to:content-language:subject:subject :user-agent:mime-version:date:date:message-id; s=dkim; t= 1728527590; x=1731119591; bh=TENrsGQFgqcLHz504/hursvFcqfwtNQvd4R 014kUUIE=; b=Ee+BWgZwBUly3rnhyD0Ce+lJsQoPbZZhNibhwIWibtajKyHgemy 8+2mOSG9M3McBIx4F8a0OwNKrImfvPBzyNWKmvwfVhcMCCcG9pwHFUueB2h8n+nj e669d62CArcyW7nNy099gbyLh7iY0uRgZ7i+7Pv2/2Jlf7akkHh3t6zw= Received: from iredmail.onthenet.com.au ([127.0.0.1]) by iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id XGj7kOil1gy8 for ; Thu, 10 Oct 2024 12:33:10 +1000 (AEST) Received: from [172.20.10.3] (unknown [1.146.52.193]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 5D4632127CB7; Thu, 10 Oct 2024 12:33:10 +1000 (AEST) Message-ID: <8b249b64-d041-4f12-b6cb-fdb528837f22@freebsd.org> Date: Thu, 10 Oct 2024 12:33:09 +1000 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Running Mezzano in bhyve Content-Language: en-US To: Vasily Postnicov Cc: freebsd-virtualization@freebsd.org References: <17f4077d-647d-4848-9d6f-97f9886ef636@freebsd.org> From: Peter Grehan In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-CMAE-Score: 0 X-CMAE-Analysis: v=2.4 cv=Fu4D/Xrq c=1 sm=1 tr=0 ts=67073ce6 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=Zfx7oXMFReXrHyuXUb3ZVQ==:17 a=IkcTkHD0fZMA:10 a=DAUX931o1VcA:10 a=bi0XHdcepdgA:10 a=2PKPKxbeqJBfNPpYSrMA:9 a=QEXdDO2ut3YA:10 X-Spamd-Result: default: False [-3.25 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.96)[-0.958]; FORGED_SENDER(0.30)[grehan@freebsd.org,SRS0=SYqd=RG=freebsd.org=grehan@iredmail.onthenet.com.au]; R_SPF_ALLOW(-0.20)[+ip4:203.13.68.0/24]; R_DKIM_ALLOW(-0.20)[nt.com.au:s=dkim]; RCVD_IN_DNSWL_LOW(-0.10)[203.13.68.12:from]; RWL_MAILSPIKE_GOOD(-0.10)[203.13.68.12:from]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), DKIM not aligned (relaxed),none]; XM_UA_NO_VERSION(0.01)[]; ARC_NA(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[grehan@freebsd.org,SRS0=SYqd=RG=freebsd.org=grehan@iredmail.onthenet.com.au]; DKIM_TRACE(0.00)[nt.com.au:+]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; TAGGED_RCPT(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:9313, ipnet:203.13.68.0/24, country:AU]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[] X-Rspamd-Queue-Id: 4XPDMJ5CF8z49WR X-Spamd-Bar: --- > Turns out, panic always happens in the same place when Mezzano is tryin= g=20 > to initialize an AHCI controller. Qemu uses an old PATA controller=20 > by=C2=A0default, which works. When forcing SATA, it fails just like bhy= ve.=20 > Virtio block devices are also claimed to work, but using virtio-blk=20 > option in bhyve also results in panic. What's the panic with virtio-blk ? That may be an easier=20 fix/workaround than AHCI, and it may help with other virtio devices. > When I tried Mezzano on my PC=20 > (bare metal), I used a USB stick. Can a USB stick be emulated in bhyve?= =20 > I guess, only using USB controller passthrough? That's correct currently. later, Peter. From nobody Thu Oct 10 17:12:59 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XPbtf6zxPz5YKk6 for ; Thu, 10 Oct 2024 17:13:14 +0000 (UTC) (envelope-from shamaz.mazum@gmail.com) Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com [IPv6:2a00:1450:4864:20::133]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XPbtf4ld9z3wbp; Thu, 10 Oct 2024 17:13:14 +0000 (UTC) (envelope-from shamaz.mazum@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-x133.google.com with SMTP id 2adb3069b0e04-53993c115cfso1566883e87.2; Thu, 10 Oct 2024 10:13:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1728580391; x=1729185191; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=V0bF1PPnpsvAfUnX8UGS2a/26N4qeNJPshFt2UAwcx8=; b=JSHiVZIJ3c5+DxJoqOkrJSsAZVdb7X/QGoX0EPXxvAwD3C9hN9t1E0E6u7uI6Or4BQ CF68l9CJd69zZuYhJSTqikFKqMPXsNiYeekOupalKa497v3McmM2VGYUjAya17DLqpQy RPyyR7uHxtAnDUvQ7/MmGkj4Pu67H0F24Qr/LpHxQeJYOKOaMnnALyuljfeHJ1TfnPuN BO47xJOXjMtaLMoGqJ39swxW2/JcAj2OaGYRTP5OHi+hVTnUcoI8onC5yQ0nLERS06Vm BPfzYCledB5nl3SfPx3YNDF75g3LRfxvJ+1yzF1BZTVmjubbCWXwGzYDGrbH3OczuUYy TkuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728580391; x=1729185191; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=V0bF1PPnpsvAfUnX8UGS2a/26N4qeNJPshFt2UAwcx8=; b=u1VrMveFfeGn7L5xoTDWH9jd47WpaO8BvOuHfrgAeIncppQedEmRHZYN7BKu3lE1Rn eyHbscHIdg691I8LYMIJ4/uRCAxXaG86rnEQX8pXVQIRoSkvCYeCz9/WNDiBeC7vWXbr yp6o7ZdxVZN/KMRab64VLb+YKYPvkjGU6aKYhOrxVpN/t/8iykYTAa1k3UD8yI76uY2/ S0fnA/9CZucsraAFxrnZtetjPBHZuUkKsr17sj273mTXIdijp6dJL0tkaFDjED3xCN/K r4M7GYDgtiR9WZRmmAbYK+r4I7+sOLgghznYuT5HnDrZkGK1ufWG7mijdi9nGH34FAR7 rztA== X-Gm-Message-State: AOJu0YwIYcFP8+9K7k3ex5ZWHfR7hl18DNlEkxcOINAn5D8jbnypTvmj qLCPBrg88tcpGLAFeKy0ZEQPLqDPUptgcrvE+Ec6N31GmKvM1XrlKf2QsUDYlHsqvSuVmRq3aEi F6Bw7AdFmkM7D1Sbs4U2bg4rLo+N3t7JyjcuuJg== X-Google-Smtp-Source: AGHT+IEUhXlpwJvSHnRHqkG/0q67LC8cG3tCIHqOiSHzSMQjHF4SXcMmClKZMHYB9GFh5GQAeQNY3iABLzDBDVFcUYM= X-Received: by 2002:ac2:4c49:0:b0:539:9548:41a6 with SMTP id 2adb3069b0e04-539c498ecd9mr4336733e87.61.1728580390779; Thu, 10 Oct 2024 10:13:10 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 References: <17f4077d-647d-4848-9d6f-97f9886ef636@freebsd.org> <8b249b64-d041-4f12-b6cb-fdb528837f22@freebsd.org> In-Reply-To: <8b249b64-d041-4f12-b6cb-fdb528837f22@freebsd.org> From: Vasily Postnicov Date: Thu, 10 Oct 2024 17:12:59 +0000 Message-ID: Subject: Re: Running Mezzano in bhyve To: Peter Grehan Cc: freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="0000000000002882f20624227cb0" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4XPbtf4ld9z3wbp X-Spamd-Bar: ---- --0000000000002882f20624227cb0 Content-Type: text/plain; charset="UTF-8" I was able to fix panics in both virtio and AHCI. This is what I found: 1) Virtio had a stupid bug, namely Mezzano tried to find an accessor to some IO port in the runtime doing something like (funcall (intern (format nil "~a-~a" bus-name slot-name)) ...). Surely, the creator made an error in the name of one of the accessors, so FUNCALL tried to call an unbound symbol, hence the page fault. 2) AHCI had the following code: ;; Magic hacks for Intel devices? ;; Set port enable bits in Port Control and Status on Intel controllers. (when (eql (pci:pci-config/16 location pci:+pci-config-vendorid+) #x8086) (let* ((n-ports (1+ (ldb (byte +ahci-CAP-NP-size+ +ahci-CAP-NP-position+) (ahci-global-register ahci +ahci-register-CAP+)))) (pcs (pci:pci-config/16 location #x92))) (setf (pci:pci-config/16 location #x92) (logior pcs (ash #xFF (- (- 8 n-ports))))))) I checked the value of N-PORTS, it's 20, so (ash #xff (- (- 8 n-ports))) is 1044480 which is bigger than 2^16-1. I recompiled bhyve with MAX_PORTS = 6 in bhyve/pci_ahci.c and the panic disappeared. Now I have this output: Detected AHCI ABAR at C1002000 AHCI IRQ is B Host Capabilities FF30FF25 Global Host Control 80000000 Interrupt Status 0 Ports Implemented 1 Version 10300 Command Completion Coalescing Control 0 Command Completion Coalescing Ports 0 Enclosure Management Location 0 Enclosure Management Control 0 Host Capabilities Extended 4 BIOS/OS Handoff Control and Status 0 AHCI HBA version 1.300 Handler: 0 Config register: 17 Port 0 Waiting for CR/FR to stop. Allocated port data at 105C33000 Command List at 105C33000 Received FIS at 105C33400 Command Tabl at 105C33500 Initializing device on port 0 Command List Base Address 5C33000 Command List Base Address Upper 32-bits 1 FIS Base Address 5C33400 FIS Base Address Upper 32-bits 1 Interrupt Status 0 Interrupt Enable 7D80003F Command and Status 1C017 Task File Data 50 Signature 101 SATA Status (SCR0: SStatus) 133 SATA Control (SCR2: SControl) 300 SATA Error (SCR1: SError) 0 SATA Active (SCR3: SActive) 0 Command Issue 0 SATA Notification (SCR4: SNotification) 0 FIS-based Switching Control 0 *** AHCI-RUN-COMMAND TIMEOUT EXPIRED! *** Command completed. 105C33600: 28A20040 100000 0 3F 105C33610: 0 59564248 4644452D 2D413239 105C33620: 382D4136 39433646 0 30300000 105C33630: 20203120 42482020 45205956 54415341 105C33640: 49532044 20204B20 20202020 20202020 105C33650: 20202020 20202020 20202020 80802020 105C33660: B000000 4000 60000 0 105C33670: 0 0 A00000 70000 105C33680: 780003 780078 40200078 0 105C33690: 0 1F0000 40010E 0 105C336A0: 2803F0 74004068 40684000 4000B400 105C336B0: 7F 0 0 0 105C336C0: 0 0 A00000 0 105C336D0: 10000 6008 0 0 105C336E0: 0 0 0 40080000 105C336F0: 4008 0 0 0 105C33700: 0 0 0 0 105C33710: 0 0 0 0 105C33720: 0 0 0 0 105C33730: 0 0 0 0 105C33740: 0 0 0 0 105C33750: 10000 0 0 0 105C33760: 0 0 0 0 105C33770: 0 0 0 0 105C33780: 0 0 0 0 105C33790: 0 0 0 0 105C337A0: 40000000 0 0 0 105C337B0: 0 0 0 1020 105C337C0: 0 0 0 0 105C337D0: 0 0 0 0 105C337E0: 0 0 0 0 105C337F0: 0 0 0 78A50000 Features (83): 7400 Sector size: 200 Sector count: A00000 Serial: BHYVE-FD29-AA68-6F9C Model: BHYVE SATA DISK Registered new R/W disk #<149CAC9> sectors:A00000 Host Capabilities FF30FF25 Global Host Control 80000002 Interrupt Status 1 Ports Implemented 1 Version 10300 Command Completion Coalescing Control 0 Command Completion Coalescing Ports 0 Enclosure Management Location 0 Enclosure Management Control 0 Host Capabilities Extended 4 BIOS/OS Handoff Control and Status 0 PCI:0:0:0 1022:7432 NIL - NIL 6:0:0 rid: 0 hdr: 0 intr: FF 40: Unknown capability 10 *** AHCI-RUN-COMMAND TIMEOUT EXPIRED! *** *** AHCI-RUN-COMMAND TIMEOUT EXPIRED! *** Detected MBR style parition table on disk #<149CAC9> Detected partition 0 on disk #<149CAC9>. Start: 800 size: 800 Registered new R/W disk #<149CCD9> sectors:800 Detected partition 1 on disk #<149CAC9>. Start: 1000 size: 800 Registered new R/W disk #<149CD89> sectors:800 Detected partition 2 on disk #<149CAC9>. Start: 2000 size: 9FE000 Registered new R/W disk #<149CE39> sectors:9FE000 Looking for paging disk with UUID 5C:F6:EE:79:2C:DF:45:E1:BA:2B:63:25:C4:1A:5F:10 *** AHCI-RUN-COMMAND TIMEOUT EXPIRED! *** Found image with UUID 5C:F6:EE:79:2C:DF:45:E1:BA:2B:63:25:C4:1A:5F:10 on disk #<149CE39> Found boot image on disk #<149CE39>! BML4 at -7FFFFFEFD000 Store freelist block is 2 It seems it is booting, but very very slowly with those "TIMEOUT EXPIRED" messages. For virtio-blk, it's almost the same with an exception that it hangs completely. I'll try to investigate further. Meanwhile, can you make any suggestions why those magic intel AHCI controller hacks are required and why sc->ports can get bigger than DEF_PORTS in pci_ahci_init in bhyve? --0000000000002882f20624227cb0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I was able to fix panics in both virtio and AHCI. This is = what=C2=A0I found:

1) Virtio had a stupid bug, namely Me= zzano tried to find an accessor to some IO port in the runtime doing someth= ing like (funcall (intern (format nil "~a-~a" bus-name slot-name)= ) ...). Surely, the creator made an error in the name of one of the accesso= rs, so FUNCALL tried to call an unbound symbol, hence the page fault.
=
2) AHCI had the following code:

;; Magic hack= s for Intel devices?
;; Set port enable bits in Port Control and Status = on Intel controllers.
(when (eql (pci:pci-config/16 location pci:+= pci-config-vendorid+) #x8086)
=C2=A0 (let* ((n-ports (1+ (ldb (byte +ahc= i-CAP-NP-size+ +ahci-CAP-NP-position+)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(ahci-glo= bal-register ahci +ahci-register-CAP+))))
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0(pcs (pci:pci-config/16 location #x92)))
=C2=A0 =C2=A0 (setf (pci:= pci-config/16 location #x92) (logior pcs
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 (ash #xFF (- (- 8 n-ports)))))))

I chec= ked the value of N-PORTS, it's 20, so (ash #xff (- (- 8 n-ports))) is= =C2=A01044480 which is bigger than 2^16-1. I recompiled bhyve with MAX_PORT= S =3D 6 in bhyve/pci_ahci.c and the panic disappeared. Now I have this outp= ut:

Detected AHCI ABAR at C1002000
AHCI IRQ is = B
Host Capabilities FF30FF25
Global Host Control 80000000
Interrup= t Status 0
Ports Implemented 1
Version 10300
Command Completion Co= alescing Control 0
Command Completion Coalescing Ports 0
Enclosure Ma= nagement Location 0
Enclosure Management Control 0
Host Capabilities = Extended 4
BIOS/OS Handoff Control and Status 0
AHCI HBA version 1.30= 0
Handler: 0
Config register: 17
Port 0
Waiting for CR/FR to st= op.
Allocated port data at 105C33000
Command List at 105C33000
Rec= eived FIS at 105C33400
Command Tabl at 105C33500
Initializing device = on port 0
=C2=A0Command List Base Address 5C33000
=C2=A0Command List = Base Address Upper 32-bits 1
=C2=A0FIS Base Address 5C33400
=C2=A0FIS= Base Address Upper 32-bits 1
=C2=A0Interrupt Status 0
=C2=A0Interrup= t Enable 7D80003F
=C2=A0Command and Status 1C017
=C2=A0Task File Data= 50
=C2=A0Signature 101
=C2=A0SATA Status (SCR0: SStatus) 133
=C2= =A0SATA Control (SCR2: SControl) 300
=C2=A0SATA Error (SCR1: SError) 0=C2=A0SATA Active (SCR3: SActive) 0
=C2=A0Command Issue 0
=C2=A0SAT= A Notification (SCR4: SNotification) 0
=C2=A0FIS-based Switching Control= 0
*** AHCI-RUN-COMMAND TIMEOUT EXPIRED! ***
Command completed.
10= 5C33600: 28A20040 100000 0 3F
105C33610: 0 59564248 4644452D 2D413239105C33620: 382D4136 39433646 0 30300000
105C33630: 20203120 42482020 45= 205956 54415341
105C33640: 49532044 20204B20 20202020 20202020
105C33= 650: 20202020 20202020 20202020 80802020
105C33660: B000000 4000 60000 0=
105C33670: 0 0 A00000 70000
105C33680: 780003 780078 40200078 0
1= 05C33690: 0 1F0000 40010E 0
105C336A0: 2803F0 74004068 40684000 4000B400=
105C336B0: 7F 0 0 0
105C336C0: 0 0 A00000 0
105C336D0: 10000 6008= 0 0
105C336E0: 0 0 0 40080000
105C336F0: 4008 0 0 0
105C33700: 0 = 0 0 0
105C33710: 0 0 0 0
105C33720: 0 0 0 0
105C33730: 0 0 0 0
= 105C33740: 0 0 0 0
105C33750: 10000 0 0 0
105C33760: 0 0 0 0
105C3= 3770: 0 0 0 0
105C33780: 0 0 0 0
105C33790: 0 0 0 0
105C337A0: 400= 00000 0 0 0
105C337B0: 0 0 0 1020
105C337C0: 0 0 0 0
105C337D0: 0 = 0 0 0
105C337E0: 0 0 0 0
105C337F0: 0 0 0 78A50000
Features (83): = 7400
Sector size: 200
Sector count: A00000
Serial: BHYVE-FD29-AA68= -6F9C
Model: BHYVE SATA DISK
Registered new R/W disk #<149CAC9>= sectors:A00000
Host Capabilities FF30FF25
Global Host Control 800000= 02
Interrupt Status 1
Ports Implemented 1
Version 10300
Command= Completion Coalescing Control 0
Command Completion Coalescing Ports 0Enclosure Management Location 0
Enclosure Management Control 0
Host= Capabilities Extended 4
BIOS/OS Handoff Control and Status 0
PCI:0:0= :0 1022:7432 NIL - NIL 6:0:0 rid: 0 hdr: 0 intr: FF
=C2=A0 =C2=A0 40: Un= known capability 10
*** AHCI-RUN-COMMAND TIMEOUT EXPIRED! ***
<= div>*** AHCI-RUN-COMMAND TIMEOUT EXPIRED! ***
Detected MBR st= yle parition table on disk #<149CAC9>
Detected partition 0 on disk= #<149CAC9>. Start: 800 size: 800
Registered new R/W disk #<149= CCD9> sectors:800
Detected partition 1 on disk #<149CAC9>. Star= t: 1000 size: 800
Registered new R/W disk #<149CD89> sectors:800Detected partition 2 on disk #<149CAC9>. Start: 2000 size: 9FE000Registered new R/W disk #<149CE39> sectors:9FE000
Looking for pa= ging disk with UUID 5C:F6:EE:79:2C:DF:45:E1:BA:2B:63:25:C4:1A:5F:10
*** AHCI-RUN-COMMAND TIMEOUT EXPIRED! ***
Found image with = UUID 5C:F6:EE:79:2C:DF:45:E1:BA:2B:63:25:C4:1A:5F:10 on disk #<149CE39&g= t;
Found boot image on disk #<149CE39>!
BML4 at -7FFFFFEFD000Store freelist block is 2

It seems it is booting, but = very very slowly with those "TIMEOUT EXPIRED" messages. For virti= o-blk, it's almost the same with an exception=C2=A0that=C2=A0it hangs c= ompletely. I'll try to investigate further. Meanwhile, can you make any= suggestions why those magic intel AHCI controller hacks are required and w= hy sc->ports can get bigger than DEF_PORTS in pci_ahci_init in bhyve?
--0000000000002882f20624227cb0-- From nobody Thu Oct 10 19:43:28 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XPgDH037jz5YWt6 for ; Thu, 10 Oct 2024 19:43:43 +0000 (UTC) (envelope-from shamaz.mazum@gmail.com) Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com [IPv6:2a00:1450:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XPgDF5cBRz4J5c; Thu, 10 Oct 2024 19:43:41 +0000 (UTC) (envelope-from shamaz.mazum@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=XXnF8IwB; spf=pass (mx1.freebsd.org: domain of shamaz.mazum@gmail.com designates 2a00:1450:4864:20::136 as permitted sender) smtp.mailfrom=shamaz.mazum@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-lf1-x136.google.com with SMTP id 2adb3069b0e04-539973829e7so1378660e87.0; Thu, 10 Oct 2024 12:43:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1728589420; x=1729194220; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Plwfv8DFoejiLnDijLXCD3eTe5ZuGBDiYsjackQlWdE=; b=XXnF8IwBn3BVMfVLRgkeKz0IP9v/o1MnS0lkocqD34hDg5cqHTSxmIT7XfIMiIr7hX mI8esKSbRuxZQ8qpQiqVnaJR2hEgHUT91Dm1zzGqX3W9fiyECt2p96QdZJwzijLL3ZA1 49cULsU61ly4VfA2gCiKxk65v6KhOm9X1FG9rtosQLtv5rURSe7/wJArMH+AzegfgApk RZb/TuM0XcHwTgaBeYu9vmf/hWa5MPRLLW9tt2mLpUN8Pp0t+oVmoxogVZ7rc6H0GnZX 1c5J8C80e7pPtQuMntOlJDNeABnRo7NvAMc63gS37BiDjaXNCB3FkokXnXWhWbUxLZdW 55Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728589420; x=1729194220; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Plwfv8DFoejiLnDijLXCD3eTe5ZuGBDiYsjackQlWdE=; b=YX8RH0wX9Gk8j030OvEmc2xje3sArpLtYvqF/StahF/SjwwVmJSFISTy+qox1JVoi+ sOrUv/IqBTF9Ey4qVqKgrDyDb0mjVC38ZgvyqICCLQQPdg3AG9n4vM/s1wjpGfOyVReM b5gygI4aOMFGPIrFVC0Bhai5MUSgkGx0IxJMLxHcqrXupHmw0jMnlRwKxnuLNRwRO4by cjzlrwJotg8tWoeVcfWCp9othEwD5GnD0Eclp3ujxzqQdizG3tTI1NTyuAdVVs9xwzgj pVNqm7ip+BbsqYwLhIu2t5GeA8MIZEv1kqR39i0qBFxTkRfyDq7kmf8mCYnT7uCS/Oi3 NRww== X-Gm-Message-State: AOJu0YwD/DFFxQrayKSf6IybxVFOashlw9chcptRAppiwGntcqNSjrrk 3E0Lsp2jC1RULudFNi1R4u4tXkUlqm9RdtbXQV9MqcWpjrzFA4FV5T0jikRy2NH+2wG8lIvTmCX BnkG7Ms0/kd/4VRCYB5ykswEP7UIRAqBulCweWw== X-Google-Smtp-Source: AGHT+IEDfVR+Sv26y7M4DTC0lSEfqBm2itAAWrx8KOrFg8QTiDOe+sjHk3stsHmcwajq3NHSfaVjgsc2SRal7tsK7dk= X-Received: by 2002:a05:6512:2399:b0:536:54db:ddd0 with SMTP id 2adb3069b0e04-539d472b288mr459550e87.0.1728589419455; Thu, 10 Oct 2024 12:43:39 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 References: <17f4077d-647d-4848-9d6f-97f9886ef636@freebsd.org> <8b249b64-d041-4f12-b6cb-fdb528837f22@freebsd.org> In-Reply-To: From: Vasily Postnicov Date: Thu, 10 Oct 2024 19:43:28 +0000 Message-ID: Subject: Re: Running Mezzano in bhyve To: Peter Grehan Cc: freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="0000000000004f2cfa06242496a6" X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.996]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; ARC_NA(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; RCVD_TLS_LAST(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROMTLD(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; TAGGED_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MISSING_XM_UA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::136:from] X-Rspamd-Queue-Id: 4XPgDF5cBRz4J5c X-Spamd-Bar: --- --0000000000004f2cfa06242496a6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I suspect PCI interrupts are not functioning correctly. Look at this code: ;; Attach interrupt handler. (sup:debug-print-line "Handler: " (ahci-irq-handler ahci)) (sup:irq-attach (sup:platform-irq (pci:pci-intr-line location)) (ahci-irq-handler-function ahci) ahci) and this (defun pci-intr-line (device) (pci-config/8 device +pci-config-intr-line+)) ;; comment by me: the constant is #x3c I found that "PCI 0x3c" means PCI interrupt pin. AFAIK, interrupt pins are not supported by bhyve, is that correct? If it's true, I need either to teach bhyve how to deal with legacy interrupts or to teach Mezzano to understand MSI. What would be easier in your opinion? =D1=87=D1=82, 10 =D0=BE=D0=BA=D1=82. 2024=E2=80=AF=D0=B3. =D0=B2 17:12, Vas= ily Postnicov : > I was able to fix panics in both virtio and AHCI. This is what I found: > > 1) Virtio had a stupid bug, namely Mezzano tried to find an accessor to > some IO port in the runtime doing something like (funcall (intern (format > nil "~a-~a" bus-name slot-name)) ...). Surely, the creator made an error = in > the name of one of the accessors, so FUNCALL tried to call an unbound > symbol, hence the page fault. > 2) AHCI had the following code: > > ;; Magic hacks for Intel devices? > ;; Set port enable bits in Port Control and Status on Intel controllers. > (when (eql (pci:pci-config/16 location pci:+pci-config-vendorid+) #x8086) > (let* ((n-ports (1+ (ldb (byte +ahci-CAP-NP-size+ +ahci-CAP-NP-position= +) > (ahci-global-register ahci > +ahci-register-CAP+)))) > (pcs (pci:pci-config/16 location #x92))) > (setf (pci:pci-config/16 location #x92) (logior pcs > (ash #xFF (- (- 8 > n-ports))))))) > > I checked the value of N-PORTS, it's 20, so (ash #xff (- (- 8 n-ports))) > is 1044480 which is bigger than 2^16-1. I recompiled bhyve with MAX_PORTS= =3D > 6 in bhyve/pci_ahci.c and the panic disappeared. Now I have this output: > > Detected AHCI ABAR at C1002000 > AHCI IRQ is B > Host Capabilities FF30FF25 > Global Host Control 80000000 > Interrupt Status 0 > Ports Implemented 1 > Version 10300 > Command Completion Coalescing Control 0 > Command Completion Coalescing Ports 0 > Enclosure Management Location 0 > Enclosure Management Control 0 > Host Capabilities Extended 4 > BIOS/OS Handoff Control and Status 0 > AHCI HBA version 1.300 > Handler: 0 > Config register: 17 > Port 0 > Waiting for CR/FR to stop. > Allocated port data at 105C33000 > Command List at 105C33000 > Received FIS at 105C33400 > Command Tabl at 105C33500 > Initializing device on port 0 > Command List Base Address 5C33000 > Command List Base Address Upper 32-bits 1 > FIS Base Address 5C33400 > FIS Base Address Upper 32-bits 1 > Interrupt Status 0 > Interrupt Enable 7D80003F > Command and Status 1C017 > Task File Data 50 > Signature 101 > SATA Status (SCR0: SStatus) 133 > SATA Control (SCR2: SControl) 300 > SATA Error (SCR1: SError) 0 > SATA Active (SCR3: SActive) 0 > Command Issue 0 > SATA Notification (SCR4: SNotification) 0 > FIS-based Switching Control 0 > *** AHCI-RUN-COMMAND TIMEOUT EXPIRED! *** > Command completed. > 105C33600: 28A20040 100000 0 3F > 105C33610: 0 59564248 4644452D 2D413239 > 105C33620: 382D4136 39433646 0 30300000 > 105C33630: 20203120 42482020 45205956 54415341 > 105C33640: 49532044 20204B20 20202020 20202020 > 105C33650: 20202020 20202020 20202020 80802020 > 105C33660: B000000 4000 60000 0 > 105C33670: 0 0 A00000 70000 > 105C33680: 780003 780078 40200078 0 > 105C33690: 0 1F0000 40010E 0 > 105C336A0: 2803F0 74004068 40684000 4000B400 > 105C336B0: 7F 0 0 0 > 105C336C0: 0 0 A00000 0 > 105C336D0: 10000 6008 0 0 > 105C336E0: 0 0 0 40080000 > 105C336F0: 4008 0 0 0 > 105C33700: 0 0 0 0 > 105C33710: 0 0 0 0 > 105C33720: 0 0 0 0 > 105C33730: 0 0 0 0 > 105C33740: 0 0 0 0 > 105C33750: 10000 0 0 0 > 105C33760: 0 0 0 0 > 105C33770: 0 0 0 0 > 105C33780: 0 0 0 0 > 105C33790: 0 0 0 0 > 105C337A0: 40000000 0 0 0 > 105C337B0: 0 0 0 1020 > 105C337C0: 0 0 0 0 > 105C337D0: 0 0 0 0 > 105C337E0: 0 0 0 0 > 105C337F0: 0 0 0 78A50000 > Features (83): 7400 > Sector size: 200 > Sector count: A00000 > Serial: BHYVE-FD29-AA68-6F9C > Model: BHYVE SATA DISK > Registered new R/W disk #<149CAC9> sectors:A00000 > Host Capabilities FF30FF25 > Global Host Control 80000002 > Interrupt Status 1 > Ports Implemented 1 > Version 10300 > Command Completion Coalescing Control 0 > Command Completion Coalescing Ports 0 > Enclosure Management Location 0 > Enclosure Management Control 0 > Host Capabilities Extended 4 > BIOS/OS Handoff Control and Status 0 > PCI:0:0:0 1022:7432 NIL - NIL 6:0:0 rid: 0 hdr: 0 intr: FF > 40: Unknown capability 10 > *** AHCI-RUN-COMMAND TIMEOUT EXPIRED! *** > *** AHCI-RUN-COMMAND TIMEOUT EXPIRED! *** > Detected MBR style parition table on disk #<149CAC9> > Detected partition 0 on disk #<149CAC9>. Start: 800 size: 800 > Registered new R/W disk #<149CCD9> sectors:800 > Detected partition 1 on disk #<149CAC9>. Start: 1000 size: 800 > Registered new R/W disk #<149CD89> sectors:800 > Detected partition 2 on disk #<149CAC9>. Start: 2000 size: 9FE000 > Registered new R/W disk #<149CE39> sectors:9FE000 > Looking for paging disk with UUID > 5C:F6:EE:79:2C:DF:45:E1:BA:2B:63:25:C4:1A:5F:10 > *** AHCI-RUN-COMMAND TIMEOUT EXPIRED! *** > Found image with UUID 5C:F6:EE:79:2C:DF:45:E1:BA:2B:63:25:C4:1A:5F:10 on > disk #<149CE39> > Found boot image on disk #<149CE39>! > BML4 at -7FFFFFEFD000 > Store freelist block is 2 > > It seems it is booting, but very very slowly with those "TIMEOUT EXPIRED" > messages. For virtio-blk, it's almost the same with an exception that it > hangs completely. I'll try to investigate further. Meanwhile, can you mak= e > any suggestions why those magic intel AHCI controller hacks are required > and why sc->ports can get bigger than DEF_PORTS in pci_ahci_init in bhyve= ? > --0000000000004f2cfa06242496a6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I suspect PCI interrupts are not function= ing correctly.

Look at this code:
=C2=A0 =C2=A0 ;; Attach interrupt handler.
=C2=A0 =C2=A0 (sup:debu= g-print-line "Handler: " (ahci-irq-handler ahci))
=C2=A0 =C2= =A0 (sup:irq-attach (sup:platform-irq (pci:pci-intr-line location))
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (ahci-ir= q-handler-function ahci)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 ahci)

and this

(defun pci-intr-line (device)
=C2=A0 (pci-config/8 device +pci-c= onfig-intr-line+)) ;; comment by me: the constant is #x3c
I found that "PCI 0x3c" means PCI interrupt pin. AFAI= K, interrupt pins are not supported by bhyve, is that=C2=A0correct? If it&#= 39;s true, I need either to teach bhyve how to deal with legacy interrupts = or to teach Mezzano to understand MSI. What would be easier in your=C2=A0op= inion?

=D1=87=D1=82, 10 =D0=BE=D0=BA=D1=82. 2024=E2=80=AF=D0=B3. =D0=B2= 17:12, Vasily Postnicov <sham= az.mazum@gmail.com>:
I wa= s able to fix panics in both virtio and AHCI. This is what=C2=A0I found:
1) Virtio had a stupid bug, namely Mezzano tried to find a= n accessor to some IO port in the runtime doing something like (funcall (in= tern (format nil "~a-~a" bus-name slot-name)) ...). Surely, the c= reator made an error in the name of one of the accessors, so FUNCALL tried = to call an unbound symbol, hence the page fault.
2) AHCI had the = following code:

;; Magic hacks for Intel devices?<= br>;; Set port enable bits in Port Control and Status on Intel controllers.=
(when (eql (pci:pci-config/16 location pci:+pci-config-vendorid+)= #x8086)
=C2=A0 (let* ((n-ports (1+ (ldb (byte +ahci-CAP-NP-size+ +ahci-= CAP-NP-position+)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(ahci-global-register ahci +ah= ci-register-CAP+))))
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(pcs (pci:pci-con= fig/16 location #x92)))
=C2=A0 =C2=A0 (setf (pci:pci-config/16 location = #x92) (logior pcs
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (ash #xFF (- (-= 8 n-ports)))))))

I checked the value of N-PORTS, = it's 20, so (ash #xff (- (- 8 n-ports))) is=C2=A01044480 which is bigge= r than 2^16-1. I recompiled bhyve with MAX_PORTS =3D 6 in bhyve/pci_ahci.c = and the panic disappeared. Now I have this output:

Detected AHCI ABAR at C1002000
AHCI IRQ is B
Host Capabilities FF30F= F25
Global Host Control 80000000
Interrupt Status 0
Ports Implemen= ted 1
Version 10300
Command Completion Coalescing Control 0
Comman= d Completion Coalescing Ports 0
Enclosure Management Location 0
Enclo= sure Management Control 0
Host Capabilities Extended 4
BIOS/OS Handof= f Control and Status 0
AHCI HBA version 1.300
Handler: 0
Config re= gister: 17
Port 0
Waiting for CR/FR to stop.
Allocated port data a= t 105C33000
Command List at 105C33000
Received FIS at 105C33400
Co= mmand Tabl at 105C33500
Initializing device on port 0
=C2=A0Command L= ist Base Address 5C33000
=C2=A0Command List Base Address Upper 32-bits 1=
=C2=A0FIS Base Address 5C33400
=C2=A0FIS Base Address Upper 32-bits = 1
=C2=A0Interrupt Status 0
=C2=A0Interrupt Enable 7D80003F
=C2=A0C= ommand and Status 1C017
=C2=A0Task File Data 50
=C2=A0Signature 101=C2=A0SATA Status (SCR0: SStatus) 133
=C2=A0SATA Control (SCR2: SContr= ol) 300
=C2=A0SATA Error (SCR1: SError) 0
=C2=A0SATA Active (SCR3: SA= ctive) 0
=C2=A0Command Issue 0
=C2=A0SATA Notification (SCR4: SNotifi= cation) 0
=C2=A0FIS-based Switching Control 0
*** AHCI-RUN-COMMAND TI= MEOUT EXPIRED! ***
Command completed.
105C33600: 28A20040 100000 0 3F=
105C33610: 0 59564248 4644452D 2D413239
105C33620: 382D4136 39433646= 0 30300000
105C33630: 20203120 42482020 45205956 54415341
105C33640:= 49532044 20204B20 20202020 20202020
105C33650: 20202020 20202020 202020= 20 80802020
105C33660: B000000 4000 60000 0
105C33670: 0 0 A00000 700= 00
105C33680: 780003 780078 40200078 0
105C33690: 0 1F0000 40010E 0105C336A0: 2803F0 74004068 40684000 4000B400
105C336B0: 7F 0 0 0
10= 5C336C0: 0 0 A00000 0
105C336D0: 10000 6008 0 0
105C336E0: 0 0 0 4008= 0000
105C336F0: 4008 0 0 0
105C33700: 0 0 0 0
105C33710: 0 0 0 0105C33720: 0 0 0 0
105C33730: 0 0 0 0
105C33740: 0 0 0 0
105C337= 50: 10000 0 0 0
105C33760: 0 0 0 0
105C33770: 0 0 0 0
105C33780: 0= 0 0 0
105C33790: 0 0 0 0
105C337A0: 40000000 0 0 0
105C337B0: 0 0= 0 1020
105C337C0: 0 0 0 0
105C337D0: 0 0 0 0
105C337E0: 0 0 0 0105C337F0: 0 0 0 78A50000
Features (83): 7400
Sector size: 200
S= ector count: A00000
Serial: BHYVE-FD29-AA68-6F9C
Model: BHYVE SATA DI= SK
Registered new R/W disk #<149CAC9> sectors:A00000
Host Capab= ilities FF30FF25
Global Host Control 80000002
Interrupt Status 1
P= orts Implemented 1
Version 10300
Command Completion Coalescing Contro= l 0
Command Completion Coalescing Ports 0
Enclosure Management Locati= on 0
Enclosure Management Control 0
Host Capabilities Extended 4
B= IOS/OS Handoff Control and Status 0
PCI:0:0:0 1022:7432 NIL - NIL 6:0:0 = rid: 0 hdr: 0 intr: FF
=C2=A0 =C2=A0 40: Unknown capability 10
*** AH= CI-RUN-COMMAND TIMEOUT EXPIRED! ***
*** AHCI-RUN-COMMAND TIME= OUT EXPIRED! ***
Detected MBR style parition table on disk #&= lt;149CAC9>
Detected partition 0 on disk #<149CAC9>. Start: 800= size: 800
Registered new R/W disk #<149CCD9> sectors:800
Detec= ted partition 1 on disk #<149CAC9>. Start: 1000 size: 800
Register= ed new R/W disk #<149CD89> sectors:800
Detected partition 2 on dis= k #<149CAC9>. Start: 2000 size: 9FE000
Registered new R/W disk #&l= t;149CE39> sectors:9FE000
Looking for paging disk with UUID 5C:F6:EE:= 79:2C:DF:45:E1:BA:2B:63:25:C4:1A:5F:10
*** AHCI-RUN-COMMAND T= IMEOUT EXPIRED! ***
Found image with UUID 5C:F6:EE:79:2C:DF:45:E1:= BA:2B:63:25:C4:1A:5F:10 on disk #<149CE39>
Found boot image on dis= k #<149CE39>!
BML4 at -7FFFFFEFD000
Store freelist block is 2
It seems it is booting, but very very slowly with those &= quot;TIMEOUT EXPIRED" messages. For virtio-blk, it's almost the sa= me with an exception=C2=A0that=C2=A0it hangs completely. I'll try to in= vestigate further. Meanwhile, can you make any suggestions why those magic = intel AHCI controller hacks are required and why sc->ports can get bigge= r than DEF_PORTS in pci_ahci_init in bhyve?
--0000000000004f2cfa06242496a6-- From nobody Sat Oct 12 06:12:37 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XQY7k3rZvz5YsBl for ; Sat, 12 Oct 2024 06:12:50 +0000 (UTC) (envelope-from SRS0=jCoQ=RI=freebsd.org=grehan@iredmail.onthenet.com.au) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XQY7f17Y0z4FLp for ; Sat, 12 Oct 2024 06:12:42 +0000 (UTC) (envelope-from SRS0=jCoQ=RI=freebsd.org=grehan@iredmail.onthenet.com.au) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=nt.com.au header.s=dkim header.b=SsntkXrI; spf=pass (mx1.freebsd.org: domain of "SRS0=jCoQ=RI=freebsd.org=grehan@iredmail.onthenet.com.au" designates 203.13.68.12 as permitted sender) smtp.mailfrom="SRS0=jCoQ=RI=freebsd.org=grehan@iredmail.onthenet.com.au"; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=freebsd.org (policy=none) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id F32ED20B4999 for ; Sat, 12 Oct 2024 16:12:37 +1000 (AEST) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id DE9052127CB7 for ; Sat, 12 Oct 2024 16:12:37 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=nt.com.au; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:content-language:references:to:subject:subject :user-agent:mime-version:date:date:message-id; s=dkim; t= 1728713557; x=1731305558; bh=B8PAaWnR68Y9zyShvE6XfHMf2xsd03wWj9i GivgORhg=; b=SsntkXrInkNqhPDIF/hmFLv+KcNuUN1n8CO/wkECBQ5puS351TU EMG2UEwaG6Ebu6jubzMQeqj6Pp8LzufmoEk5cnXTjbfhTz5rqCNu4ak9vAcM1vjV VFroB2ZK0Yp0BXgBbdMRgrHlsQKaiAN2hz7KU2wgrP+rIuyr96oCjWJU= Received: from iredmail.onthenet.com.au ([127.0.0.1]) by iredmail.onthenet.com.au (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id EuqlYLHSDSOk for ; Sat, 12 Oct 2024 16:12:37 +1000 (AEST) Received: from [192.168.1.101] (otn-120-29-24-249.broadband.onthenet.net [120.29.24.249]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id BF7772127CB9; Sat, 12 Oct 2024 16:12:37 +1000 (AEST) Message-ID: Date: Sat, 12 Oct 2024 16:12:37 +1000 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Running Mezzano in bhyve To: Vasily Postnicov Cc: freebsd-virtualization@freebsd.org References: <17f4077d-647d-4848-9d6f-97f9886ef636@freebsd.org> <8b249b64-d041-4f12-b6cb-fdb528837f22@freebsd.org> Content-Language: en-US From: Peter Grehan In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-CMAE-Score: 0 X-CMAE-Analysis: v=2.4 cv=Fu4D/Xrq c=1 sm=1 tr=0 ts=670a1356 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=0UtLtLgMz4NvdyIsuxvgLw==:17 a=IkcTkHD0fZMA:10 a=DAUX931o1VcA:10 a=bi0XHdcepdgA:10 a=BYglAFDJemt_E9MsyRoA:9 a=QEXdDO2ut3YA:10 X-Spamd-Result: default: False [-3.29 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FORGED_SENDER(0.30)[grehan@freebsd.org,SRS0=jCoQ=RI=freebsd.org=grehan@iredmail.onthenet.com.au]; R_SPF_ALLOW(-0.20)[+ip4:203.13.68.0/24]; R_DKIM_ALLOW(-0.20)[nt.com.au:s=dkim]; RCVD_IN_DNSWL_LOW(-0.10)[203.13.68.12:from]; RWL_MAILSPIKE_GOOD(-0.10)[203.13.68.12:from]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), DKIM not aligned (relaxed),none]; XM_UA_NO_VERSION(0.01)[]; ARC_NA(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[grehan@freebsd.org,SRS0=jCoQ=RI=freebsd.org=grehan@iredmail.onthenet.com.au]; DKIM_TRACE(0.00)[nt.com.au:+]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; TAGGED_RCPT(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:9313, ipnet:203.13.68.0/24, country:AU]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[] X-Rspamd-Queue-Id: 4XQY7f17Y0z4FLp X-Spamd-Bar: --- > I suspect PCI interrupts are not functioning correctly. >=20 > Look at this code: > =C2=A0 =C2=A0 ;; Attach interrupt handler. > =C2=A0 =C2=A0 (sup:debug-print-line "Handler: " (ahci-irq-handler ahci= )) > =C2=A0 =C2=A0 (sup:irq-attach (sup:platform-irq (pci:pci-intr-line loc= ation)) > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = (ahci-irq-handler-function ahci) > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = ahci) >=20 > and this >=20 > (defun pci-intr-line (device) > =C2=A0 (pci-config/8 device +pci-config-intr-line+)) ;; comment by me:= the=20 > constant is #x3c >=20 > I found that "PCI 0x3c" means PCI interrupt pin. AFAIK, interrupt pins=20 > are not supported by bhyve, is that=C2=A0correct? If it's true, I need = either=20 > to teach bhyve how to deal with legacy interrupts or to teach Mezzano t= o=20 > understand MSI. What would be easier in your=C2=A0opinion? Legacy interrupts should work fine in bhyve for emulated devices. I'd=20 suspect this would be much easier to debug/enhance as opposed to adding=20 MSI (and likely MSI-x). later, Peter. From nobody Sat Oct 12 18:22:59 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XQsLT3kHSz5Ymmf for ; Sat, 12 Oct 2024 18:23:13 +0000 (UTC) (envelope-from shamaz.mazum@gmail.com) Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com [IPv6:2a00:1450:4864:20::12e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XQsLS5ZJQz4Cxk; Sat, 12 Oct 2024 18:23:12 +0000 (UTC) (envelope-from shamaz.mazum@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-x12e.google.com with SMTP id 2adb3069b0e04-539e59dadebso1092280e87.0; Sat, 12 Oct 2024 11:23:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1728757391; x=1729362191; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=stOsPLRCaB+WHjtRfQCe1Wu26pvs+AXcyPVMkEzWBXI=; b=nb2Nzjzni08XIi1mSSLMoy4zDfbi1s9VmPLrm15Yl9MMnnVdtagPdvvDqP7eiaonuq P7JXzrZjmCNqjSmBvVHwPcuqv9AXMx0lk8lYs9GDUzhc3XDfu6tsl0nX/lNBqViR7N/C 45PcSeZRGbmyoimA/Ecte3MeSmyMt9xMAABTxeUzKUL5Q3GCaCa0VH59RqqX+71zlglc 9epc61gw0RhnVuqTYThIzy1oIlEnL91e99AA5OzM690SXSqekRtWebH29Igdf1BJyz+E OonYjFu1m3OfjnxKXMcM7hGljs89NGvSJjhECUfIxuS3FVrZEHl9+CC2nTyVfnHMC4k8 u5yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728757391; x=1729362191; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=stOsPLRCaB+WHjtRfQCe1Wu26pvs+AXcyPVMkEzWBXI=; b=GOAipTnZfMa59/vkLnfw7kqtn74nVAJt7wq7ENHNPxFZGixJqZglZigP50LT7pkl56 kIBBc86FtdYQYwa3jcqpmJ+Y95UGa38CQaM35vTjDweAWI3N/fBkZqA+Vxwp4AjHDbwk 4K1CiAYmTQ9xL1Zo16MwfDGq+cfUmMFBW0MnE07UIMbKFVxjP56FEsDpf/wtFHbrlur5 2x03nRIHEMv+ibnm0DjxEzZoVctWD8zJ+t7Fs2GipX6rfvpvQ8zN5JydOUbqCPSBxvDD pYaNDLN0SEZwxM0ykrlYFDt2sjFprOdZQC035tDSpKCDDc2Aw1jdfCFIOTDOVa6eQncA 8JYQ== X-Gm-Message-State: AOJu0YyLqomZ3JOlKibO+FKgnu96iZXn6eVoXEvcWd3FAjJVs2SvOzzu oaj3qKyTq804x+0i8v05xxNtNFwaVDRsH/oUHut1SLOUGr3vkjo3lBDBG8Z68LyLeKTwpG59yAI APsRbCQhPPM/9AwoGM+Ms5lrZ7vKUNlbcTFbDwg== X-Google-Smtp-Source: AGHT+IECEJOBMr3ZCBsRbcHv9RqViBp+/+krBQsDz2NEfeN6j/stmZU+kcoZXy7f3ZJ7/rNoWbHujuqOEdwIHAStB7M= X-Received: by 2002:a05:6512:1588:b0:539:93b2:1383 with SMTP id 2adb3069b0e04-539da5661c8mr2943149e87.46.1728757390339; Sat, 12 Oct 2024 11:23:10 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 References: <17f4077d-647d-4848-9d6f-97f9886ef636@freebsd.org> <8b249b64-d041-4f12-b6cb-fdb528837f22@freebsd.org> In-Reply-To: From: Vasily Postnicov Date: Sat, 12 Oct 2024 18:22:59 +0000 Message-ID: Subject: Re: Running Mezzano in bhyve To: Peter Grehan Cc: freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="00000000000027764c06244bb20d" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4XQsLS5ZJQz4Cxk X-Spamd-Bar: ---- --00000000000027764c06244bb20d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I was able to boot Mezzano completely (though without a working mouse, just like on my real PC). Both ahci and virtio disks work. This is a list of issues I found: 1) The problem with PIT. Can be solved as you proposed or by patching Mezzano. 2) Mezzano assumes that Intel AHCI controllers report no more than 6 ports. Can be solved by patching Mezzano or defining MAX_PORTS to be 6 in usr.sbin/bhyve/pci_ahci.c 3) According to https://wiki.osdev.org/PCI#Message_Signaled_Interrupts, interrupt line config register must be RW. Bhyve does not support writing to it. I do not know a correct fix, this [1] workaround helps, however. 4) Finally, I had a random deadlock in interrupt handling for the virtio-net device. Likewise, I do not know how to fix it correctly, but this [2] patch helped. Used patches: [1] Workaround for the interrupt line. Forbid 1 byte writes to 0x3c config register (the write is performed by UEFI firmware or bootloader). diff --git a/usr.sbin/bhyve/pci_emul.c b/usr.sbin/bhyve/pci_emul.c index e91b4d0a1e20..6e9d5885a6b1 100644 --- a/usr.sbin/bhyve/pci_emul.c +++ b/usr.sbin/bhyve/pci_emul.c @@ -160,8 +160,11 @@ static __inline void CFGWRITE(struct pci_devinst *pi, int coff, uint32_t val, int bytes) { - if (bytes =3D=3D 1) - pci_set_cfgdata8(pi, coff, val); + if (bytes =3D=3D 1) { + if (coff !=3D PCIR_INTLINE) { + pci_set_cfgdata8(pi, coff, val); + } + } else if (bytes =3D=3D 2) pci_set_cfgdata16(pi, coff, val); else [2] Workaround for a hang up in interrupt handling code: diff --git a/usr.sbin/bhyve/virtio.h b/usr.sbin/bhyve/virtio.h index 4c6c8004b2d1..d1a901ff0069 100644 --- a/usr.sbin/bhyve/virtio.h +++ b/usr.sbin/bhyve/virtio.h @@ -355,13 +355,13 @@ vi_interrupt(struct virtio_softc *vs, uint8_t isr, uint16_t msix_idx) if (pci_msix_enabled(vs->vs_pi)) pci_generate_msix(vs->vs_pi, msix_idx); else { - VS_LOCK(vs); + //VS_LOCK(vs); vs->vs_isr |=3D isr; - pci_generate_msi(vs->vs_pi, 0); + //pci_generate_msi(vs->vs_pi, 0); #ifdef __amd64__ pci_lintr_assert(vs->vs_pi); #endif - VS_UNLOCK(vs); + //VS_UNLOCK(vs); } } Do you have any ideas how to make proper patches for bhyve from these workarounds? P.S. After all 4 issues are addressed to, you can build Demo 5 image, but only after replacing a bootloader. See https://github.com/froggey/Mezzano/issues/173 If you wish, I can send the image to you by some means (Telegram?) Or you can build the bootloader from here: https://github.com/froggey/kboot/tree/mezzano-loader (requires Linux, I guess). I'll try virtio-input to see if the mouse works this way. =D1=81=D0=B1, 12 =D0=BE=D0=BA=D1=82. 2024=E2=80=AF=D0=B3. =D0=B2 06:12, Pet= er Grehan : > > I suspect PCI interrupts are not functioning correctly. > > > > Look at this code: > > ;; Attach interrupt handler. > > (sup:debug-print-line "Handler: " (ahci-irq-handler ahci)) > > (sup:irq-attach (sup:platform-irq (pci:pci-intr-line location)) > > (ahci-irq-handler-function ahci) > > ahci) > > > > and this > > > > (defun pci-intr-line (device) > > (pci-config/8 device +pci-config-intr-line+)) ;; comment by me: the > > constant is #x3c > > > > I found that "PCI 0x3c" means PCI interrupt pin. AFAIK, interrupt pins > > are not supported by bhyve, is that correct? If it's true, I need eithe= r > > to teach bhyve how to deal with legacy interrupts or to teach Mezzano t= o > > understand MSI. What would be easier in your opinion? > > Legacy interrupts should work fine in bhyve for emulated devices. I'd > suspect this would be much easier to debug/enhance as opposed to adding > MSI (and likely MSI-x). > > later, > > Peter. > --00000000000027764c06244bb20d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I was able to boot Mezzano completely (though without a wo= rking mouse, just like on my real PC). Both ahci and virtio disks work. Thi= s is a list of issues I found:

1) The problem with PIT. = Can be solved as you proposed or by patching Mezzano.
2) Mezzano = assumes that Intel AHCI controllers report no more than 6 ports. Can be sol= ved by patching Mezzano or defining MAX_PORTS to be 6 in usr.sbin/bhyve/pci= _ahci.c
3) According=C2=A0to=C2=A0https://wiki.osdev.org/PCI#Message_Sign= aled_Interrupts, interrupt line config register must be RW. Bhyve does = not support writing to it. I do not know a correct fix, this [1] workaround= helps, however.
4) Finally, I had a random deadlock in interrupt= handling for the virtio-net device. Likewise, I do not know how to fix it = correctly, but this [2] patch helped.

Used patches= :
[1] Workaround for the interrupt line. Forbid 1 byte writes to = 0x3c config register (the write is performed by UEFI firmware or bootloader= ).
diff --git a/usr.sbin/bhyve/pci_emul.c b/usr.sbin/bhyve/pci_em= ul.c
index e91b4d0a1e20..6e9d5885a6b1 100644
--- a/usr.sbin/bhyve/pci= _emul.c
+++ b/usr.sbin/bhyve/pci_emul.c
@@ -160,8 +160,11 @@ static _= _inline void
=C2=A0CFGWRITE(struct pci_devinst *pi, int coff, uint32_t v= al, int bytes)
=C2=A0{

- =C2=A0 =C2=A0 =C2=A0 if (bytes =3D=3D 1)=
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pci_set_cfgdata8(pi,= coff, val);
+ =C2=A0 =C2=A0 =C2=A0 if (bytes =3D=3D 1) {
+ =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (coff !=3D PCIR_INTLINE) {
= + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 pci_set_cfgdata8(pi, coff, val);
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 }
+ =C2=A0 =C2=A0 =C2=A0 }
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 else if (bytes =3D=3D 2)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 pci_set_cfgdata16(pi, coff, val);
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 else

[2] Workaround for a hang up in inter= rupt handling code:
diff --git a/usr.sbin/bhyve/virtio.h b/usr.sb= in/bhyve/virtio.h
index 4c6c8004b2d1..d1a901ff0069 100644
--- a/usr.s= bin/bhyve/virtio.h
+++ b/usr.sbin/bhyve/virtio.h
@@ -355,13 +355,13 @= @ vi_interrupt(struct virtio_softc *vs, uint8_t isr, uint16_t msix_idx)
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (pci_msix_enabled(vs->vs_pi))
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pci_generate_msix(vs->v= s_pi, msix_idx);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 else {
- =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 VS_LOCK(vs);
+ =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 //VS_LOCK(vs);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 vs->vs_isr |=3D isr;
- =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pci_generate_msi(vs->vs_pi, 0);
+ =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 //pci_generate_msi(vs->vs_= pi, 0);
=C2=A0#ifdef __amd64__
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 pci_lintr_assert(vs->vs_pi);
=C2=A0#endif
- =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 VS_UNLOCK(vs);
+ =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 //VS_UNLOCK(vs);
=C2=A0 =C2=A0= =C2=A0 =C2=A0 }
=C2=A0}

Do you have any id= eas how to make proper patches for bhyve from these workarounds?
= P.S. After all 4 issues are addressed to, you can build Demo 5 image, but o= nly after replacing a bootloader. See=C2=A0https://github.com/froggey/Mezzano/issues/173= If you wish, I can send the image to you by some means (Telegram?) Or you = can build the bootloader from here:=C2=A0https://github.com/froggey/kboot/tree/mezzan= o-loader (requires Linux, I guess).

I'll t= ry virtio-input to see if the mouse works this way.

=D1=81=D0=B1, 12 = =D0=BE=D0=BA=D1=82. 2024=E2=80=AF=D0=B3. =D0=B2 06:12, Peter Grehan <grehan@freebsd.org>:
> I suspect PCI interrupts are not functioning correctly.<= br> >
> Look at this code:
>=C2=A0 =C2=A0 =C2=A0 ;; Attach interrupt handler.
>=C2=A0 =C2=A0 =C2=A0 (sup:debug-print-line "Handler: " (ahci-= irq-handler ahci))
>=C2=A0 =C2=A0 =C2=A0 (sup:irq-attach (sup:platform-irq (pci:pci-intr-li= ne location))
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 (ahci-irq-handler-function ahci)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 ahci)
>
> and this
>
> (defun pci-intr-line (device)
>=C2=A0 =C2=A0 (pci-config/8 device +pci-config-intr-line+)) ;; comment = by me: the
> constant is #x3c
>
> I found that "PCI 0x3c" means PCI interrupt pin. AFAIK, inte= rrupt pins
> are not supported by bhyve, is that=C2=A0correct? If it's true, I = need either
> to teach bhyve how to deal with legacy interrupts or to teach Mezzano = to
> understand MSI. What would be easier in your=C2=A0opinion?

=C2=A0 Legacy interrupts should work fine in bhyve for emulated devices. I&= #39;d
suspect this would be much easier to debug/enhance as opposed to adding MSI (and likely MSI-x).

later,

Peter.
--00000000000027764c06244bb20d-- From nobody Sun Oct 13 21:00:26 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XRXnR17xSz5YXXB for ; Sun, 13 Oct 2024 21:00:27 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XRXnQ4JGsz4bW0 for ; Sun, 13 Oct 2024 21:00:26 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1728853226; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=7UHggt94DhkFe3xd4GpjosaLeD6l2pZ9rJ6eYkAh0+Y=; b=J/pjWkaw1mjJNjAf/MGaqfvq96HB1JRrPMIWc/IVXgLzeXKxOQCyjfFPQxk9/BdDsXFWLG hUQOM3pfL9Dco5JyUxjnVVAAeIVyqRM9McTEPZujQOGU05q106AKsveUxzx31hZz/85KIW FvKrD777opXPLwgc79oakvAsvMCvVeDmgGVKMzdOOanhUAL2GCwXlRgZmWL5CvEEVcvZCo jW7PIQCGvwK/LXsjy6FFKlkDVkjHG4fn112XwEhdhdGgnnx8Xu/Yw/E9TjbmfwT0Up+Dj+ 0/zWokSPhPiCSNpafsBnwIc72uMXqsQwjgBCjfDszJnzmHlYp7Ga0GCpTJUvgQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1728853226; a=rsa-sha256; cv=none; b=sbMAXQ00jOtqT20QAjlr678npiktuGLGRXfZOuE3rawW+DJ1WXGKSeDIM0II27c6qsfBt1 bvPaDwmwnzpYyCp4hKB1rXQ/SGH2ZFf98oC/pvHyhpUow9rXg5DJqD/RD5n5qAi9m+0lbn 8YfA7Nw/tDmN5x2JCtHnXfrTn3IFR/1QiwpPKv4GimHmv5/NVFnW2F/L7j4b2Ahz9jeEqH vraZlziAhUeXwsHqyqRyNS82krEQzhnWO5wrplRH0KMljY17hzS+sXPqqljPMhhQAJcQr8 Lj8V9hcT0/wd4OEOPI98dC4cOU5FtHGE6jraaxVvwHtAM03OQIQXqa14BQ5QAA== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4XRXnQ3w1kz18Gk for ; Sun, 13 Oct 2024 21:00:26 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 49DL0Qfd079676 for ; Sun, 13 Oct 2024 21:00:26 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 49DL0Q3q079675 for virtualization@FreeBSD.org; Sun, 13 Oct 2024 21:00:26 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202410132100.49DL0Q3q079675@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: virtualization@FreeBSD.org Subject: Problem reports for virtualization@FreeBSD.org that need special attention Date: Sun, 13 Oct 2024 21:00:26 +0000 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="17288532263.DFc3ecdC7.76910" Content-Transfer-Encoding: 7bit --17288532263.DFc3ecdC7.76910 Date: Sun, 13 Oct 2024 21:00:26 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 264267 | UEFI Booting on Azure Generation 2 VMs crashes 1 problems total for which you should take action. --17288532263.DFc3ecdC7.76910 Date: Sun, 13 Oct 2024 21:00:26 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" 
The following is a listing of current problems submitted by FreeBSD users,
which need special attention. These represent problem reports covering
all versions including experimental development code and obsolete releases.

Status      |    Bug Id | Description
------------+-----------+---------------------------------------------------
Open        |    264267 | UEFI Booting on Azure Generation 2 VMs crashes

1 problems total for which you should take action.
--17288532263.DFc3ecdC7.76910--