From nobody Sun Apr 23 00:16:05 2023 X-Original-To: freebsd-emulation@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q3prB5FRFz46Xfk for ; Sun, 23 Apr 2023 00:22:50 +0000 (UTC) (envelope-from tomek@cedro.info) Received: from mail-ua1-x92e.google.com (mail-ua1-x92e.google.com [IPv6:2607:f8b0:4864:20::92e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q3prB2b5sz3RL7 for ; Sun, 23 Apr 2023 00:22:50 +0000 (UTC) (envelope-from tomek@cedro.info) Authentication-Results: mx1.freebsd.org; none Received: by mail-ua1-x92e.google.com with SMTP id a1e0cc1a2514c-7724096f768so2159523241.1 for ; Sat, 22 Apr 2023 17:22:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cedro.info; s=google; t=1682209369; x=1684801369; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=UbXK7Y6w/Z1IWOw6B4gSc/uIw+WaR0CA3ZuYOLK5hN8=; b=R+VQI3L5Ly/9lWudUpIAdTqvoWYij4ZHWjFZ9kIrKqAuyhRI9SA3g9n2hbIcgkc4kr jzfkz6PV0KkeptT9j09oXvCBengWbDwYgFvfWvNZR3+ibnpQnPtkgK5ARHboW8CrKU2o 6fhqKl6Et5nMvLR0iweuYH72u9MNIvsfyocIqCIBR1jzqQoDuDUfczGKf+R10f6BFtaS Sh9gtpjmUKRAakRxSE9l24G+7uwZenN2NPgtLJc97o9RXDqMwNhp/QH4NMOoSS/St6k8 Sq6+ZfX87LXoqSoozTuHoRq0AKP0EiqQn8RopysM5Yq3VWc0lN4+jcslIzP29sEEL1iW zdDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682209369; x=1684801369; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UbXK7Y6w/Z1IWOw6B4gSc/uIw+WaR0CA3ZuYOLK5hN8=; b=b089IbP52OhdDFqp3lELM7uNrv8ZnqvbPfeCzX0STKHH/fQCxrQnZD6VrsZ/O3yV1R yyfydebRKIqCChXkoKwn8nkVJMoRUoToQXX3GVcT5bYjESvaJzA2kQ31+ZXGV5Fg3f/Z ylu8Q2muK8dSoanQ7SWW71X6jYu6VmhVlOQ7WFLBccKdWW5STqWhlbWI0Krm3uHJDOM5 auMBzLaysWRwgTmaSgEva0oy6Tg43LUXc/+L4FjVnod+G7fup1jfAwPYpsjbQrjvklSz 3KrJR9Xh1azsWChOGrcP5dYWBnvNZEXh5zL8/umY71MHGYc5JwqDQKudV8i7SeICWidF GOJQ== X-Gm-Message-State: AAQBX9fbfav+oXCKdTCN/Pta2rI5puGuyXwAoItnk80aW9SEhL1ozgSg ZbHhCDBCLvQSzcmyqINg7BO2G7m/J4ZR/suzn7g= X-Google-Smtp-Source: AKy350ZVl8Y6dZ8od/BWVda2bNejgGLUZ50NsJ5ftNEICIecgNgbzaDelrG/54jPUfJZ6+Z/bQ+TKw== X-Received: by 2002:a25:500c:0:b0:b92:497e:38fb with SMTP id e12-20020a25500c000000b00b92497e38fbmr5538572ybb.65.1682208977435; Sat, 22 Apr 2023 17:16:17 -0700 (PDT) Received: from mail-yb1-f173.google.com (mail-yb1-f173.google.com. [209.85.219.173]) by smtp.gmail.com with ESMTPSA id d134-20020a25e68c000000b00b9949799ce3sm898253ybh.32.2023.04.22.17.16.16 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 22 Apr 2023 17:16:17 -0700 (PDT) Received: by mail-yb1-f173.google.com with SMTP id 3f1490d57ef6-b8ed0d0a33dso4457802276.1; Sat, 22 Apr 2023 17:16:16 -0700 (PDT) X-Received: by 2002:a25:cb41:0:b0:b95:be99:858c with SMTP id b62-20020a25cb41000000b00b95be99858cmr6330953ybg.50.1682208976727; Sat, 22 Apr 2023 17:16:16 -0700 (PDT) List-Id: Development of Emulators of other operating systems List-Archive: https://lists.freebsd.org/archives/freebsd-emulation List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-emulation@freebsd.org MIME-Version: 1.0 References: <20230422211625.fbdd0c384b6d144dd9502348@sohara.org> In-Reply-To: <20230422211625.fbdd0c384b6d144dd9502348@sohara.org> From: Tomek CEDRO Date: Sun, 23 Apr 2023 02:16:05 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: embedded emulation: bhyve + vt-x on amd fx-8320 To: "Steve O'Hara-Smith" Cc: FreeBSD Questions Mailing List , freebsd-emulation@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4Q3prB2b5sz3RL7 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Sat, Apr 22, 2023 at 10:16=E2=80=AFPM Steve O'Hara-Smith wrote: > Check BIOS settings, I've found virtualisation support disabled b= y > default in the BIOS when the CPU supports it just fine. Thanks Steve, all is enabled, I would not be able to start VM without that.. the problem that I have is with "nested virtualization" :-) --=20 CeDeROM, SQ7MHZ, http://www.tomek.cedro.info From nobody Sun Apr 23 00:25:44 2023 X-Original-To: freebsd-emulation@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q3pvn5q4Hz46Xlf for ; Sun, 23 Apr 2023 00:25:57 +0000 (UTC) (envelope-from tomek@cedro.info) Received: from mail-yb1-xb33.google.com (mail-yb1-xb33.google.com [IPv6:2607:f8b0:4864:20::b33]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q3pvn2yMcz3m8H for ; Sun, 23 Apr 2023 00:25:57 +0000 (UTC) (envelope-from tomek@cedro.info) Authentication-Results: mx1.freebsd.org; none Received: by mail-yb1-xb33.google.com with SMTP id 3f1490d57ef6-b8f72e5bd9bso4520527276.0 for ; Sat, 22 Apr 2023 17:25:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cedro.info; s=google; t=1682209556; x=1684801556; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=FFyoAZdGRvPghM6tcRP7qV0IZI3S/ltv4MemlGiUrz4=; b=AxBcjhyzQBTy//PQG/57nhIwOxHksEqpyfJOxBaI7Ws8UmyfGBOpxX4R0ZdvLA6mii 30/Y2nsmx7DFw/XJwVtFHhLtrfIesYxMk95PxMehRxE7S9yi1lEpC1stHDL407jCprGd 8KjYts2SWkSoT8F8QJ1vUMfy79fE7fvcOqXuRMFOAGkQgOYkDGdREzXmbTWpsRt/ioQC 01QlxQ/W/7BDD8DOYg2rSlvRvBJKUToIE91s4XHbS+CRy7iRKIEwoSR/eccyBH6Wog7R o2xgkeCXgjIG7Ojh94Cfi54RcgsOK0qx0C3SZ06YBfM/zadIieElEL7I1Er63J+7qKgE hTEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682209556; x=1684801556; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FFyoAZdGRvPghM6tcRP7qV0IZI3S/ltv4MemlGiUrz4=; b=aibsHlHAUvcTGdACazlp4pNHlgQXo6iz0475x4IrzuhTGM+OPKuvLd75sqn45rP1sq a1E8e5e1aU0uITw2LncVJIFDdoZEdXmkPVreFywjd51WTnLBhWXePTkIULprUweiqbTO J4m4OaAzLg+y7hIlxYPXYfnTTWc2FVaTpCDo+E4nH0dZ1pslFtBDD0Y5tK0+THWVU4TI MJJJBhG2/3J+Ve5tBW4aoZS8ic+GU36iM/2u45ks6Q9DHlIKUE+JqP0iL1HPBBIaFtmq o7EAmDxqAG1oBbQgSTLPXHqywA6EaEvcunbOLLW7HTvy8pKI3C+pRXW4+2qdc6GhoEej sQLA== X-Gm-Message-State: AAQBX9eIe9Pecay2lQUvfdi4ghuB7lbC/0wAfcA27oETEWODJsHISZqT YUWJ6vtlM8NuJjGDJNzSFhEOrw== X-Google-Smtp-Source: AKy350bipO+ZkknOGEl3CXDWzOcZOgnqY7TzZuB07oQqKmkUuwgteLNzF7zH3rF7aIzBKv5XuVILpg== X-Received: by 2002:a25:d301:0:b0:b92:3f8b:d1aa with SMTP id e1-20020a25d301000000b00b923f8bd1aamr5611801ybf.61.1682209556513; Sat, 22 Apr 2023 17:25:56 -0700 (PDT) Received: from mail-yw1-f179.google.com (mail-yw1-f179.google.com. [209.85.128.179]) by smtp.gmail.com with ESMTPSA id k65-20020a819344000000b00545a08184efsm1963396ywg.127.2023.04.22.17.25.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 22 Apr 2023 17:25:56 -0700 (PDT) Received: by mail-yw1-f179.google.com with SMTP id 00721157ae682-54f9b37c634so36443807b3.2; Sat, 22 Apr 2023 17:25:55 -0700 (PDT) X-Received: by 2002:a0d:d50a:0:b0:54f:c68f:dc40 with SMTP id x10-20020a0dd50a000000b0054fc68fdc40mr5665586ywd.39.1682209555721; Sat, 22 Apr 2023 17:25:55 -0700 (PDT) List-Id: Development of Emulators of other operating systems List-Archive: https://lists.freebsd.org/archives/freebsd-emulation List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-emulation@freebsd.org MIME-Version: 1.0 References: <1033603027.10889563.1682198162191.JavaMail.zimbra@shaw.ca> In-Reply-To: <1033603027.10889563.1682198162191.JavaMail.zimbra@shaw.ca> From: Tomek CEDRO Date: Sun, 23 Apr 2023 02:25:44 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: embedded emulation: bhyve + vt-x on amd fx-8320 To: Dale Scott Cc: FreeBSD Questions Mailing List , freebsd-emulation@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4Q3pvn2yMcz3m8H X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Sat, Apr 22, 2023 at 11:16=E2=80=AFPM Dale Scott wrote: > From the handbook https://docs.freebsd.org/en/books/handbook/virtualizati= on/#virtualization-host-bhyve > > "The easiest way to tell if a processor supports bhyve is to run dmesg or= look in /var/run/dmesg.boot for the POPCNT processor feature flag on the F= eatures2 line for AMD=C2=AE processors or EPT and UG on the VT-x line for I= ntel=C2=AE processors." > > Unfortunately it seems you might be out of luck. Well, on FreeBSD I am able to start VM with both vbox and bhyve.. but the problem is with that "NESTED VIRTUALIZATION" (this seems to be the proper name) that does not seem to work. So I did setup a, yuck, windoze, just to see in practice if "nested virtualization" works on other OS on the same hardware. I did install windoze, then VBox, then created Linux Debian 11 virtual machine, then installed Android Studio on that vm, then launched Virtual Android Device and IT WORKED! o_O So my AMD FX-8320 CPU seems to support "nested virtualization" (its called VT-D for Intel and AMD-V for AMD). Question is why Nested Virtualization does not work on FreeBSD? Should I set it up somehow? Is it supported here at all? Any hints appreciated :-) --=20 CeDeROM, SQ7MHZ, http://www.tomek.cedro.info From nobody Sun Apr 23 17:02:05 2023 X-Original-To: freebsd-emulation@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q4F1b45yBz469xb for ; Sun, 23 Apr 2023 17:02:27 +0000 (UTC) (envelope-from tedm@mittelstaedt.us) Received: from mail.freebsd-corp-net-guide.com (gw.portlandia-servers.com [50.198.160.189]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.freebsd-corp-net-guide.com", Issuer "Cool Dudes Co" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q4F1Y6YgRz3FGT for ; Sun, 23 Apr 2023 17:02:25 +0000 (UTC) (envelope-from tedm@mittelstaedt.us) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=mittelstaedt.us header.s=default header.b=Z0gxBDJI; spf=pass (mx1.freebsd.org: domain of tedm@mittelstaedt.us designates 50.198.160.189 as permitted sender) smtp.mailfrom=tedm@mittelstaedt.us; dmarc=pass (policy=none) header.from=mittelstaedt.us Received: from [192.168.1.148] ([192.168.1.148]) (authenticated bits=0) by mail.freebsd-corp-net-guide.com (8.15.2/8.15.2) with ESMTPA id 33NH2BQQ059154 for ; Sun, 23 Apr 2023 10:02:17 -0700 (PDT) (envelope-from tedm@mittelstaedt.us) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=mittelstaedt.us; s=default; t=1682269337; bh=sdEI2zJRfIAFh3z4HA5k7fyuj8lfSXZRrWZAVfknkL4=; h=Date:Subject:To:References:From:In-Reply-To; b=Z0gxBDJIZD6lcK/undLcwJRS+/rTMq6v+ezvICz9xQ+vlkmXJGzHvskcYW9G7J/9y kXFGPE0/oY2Rh7CK1D8TjpDWQNpvCywxZXOisUMJMh05u31GSQ0H/V9tXNZoee8NXR y+wlWK2C/QIkak5GBZz5vTvtX5bTi70nZOObkzrk= Message-ID: <3774d906-e8f6-1330-7605-515f34084ff4@mittelstaedt.us> Date: Sun, 23 Apr 2023 10:02:05 -0700 List-Id: Development of Emulators of other operating systems List-Archive: https://lists.freebsd.org/archives/freebsd-emulation List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-emulation@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: embedded emulation: bhyve + vt-x on amd fx-8320 To: freebsd-emulation@freebsd.org References: <20230422211625.fbdd0c384b6d144dd9502348@sohara.org> Content-Language: en-US From: Ted Mittelstaedt In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=4.1 tests=ALL_TRUSTED autolearn=disabled version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mail.freebsd-corp-net-guide.com X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[mittelstaedt.us,none]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[mittelstaedt.us:s=default]; MIME_GOOD(-0.10)[text/plain]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; DKIM_TRACE(0.00)[mittelstaedt.us:+]; ASN(0.00)[asn:7922, ipnet:50.128.0.0/9, country:US]; MLMMJ_DEST(0.00)[freebsd-emulation@freebsd.org]; RCVD_TLS_LAST(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FREEFALL_USER(0.00)[tedm]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-emulation@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4Q4F1Y6YgRz3FGT X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N That requires more than just CPU hardware support.  Unlike what you think at the hardware level there is no such thing as "nested virtualization" instead each virtual session is separate the OS and CPU just keep track of what session is tied to what other session.   Sort of like trained monkeys juggling colored balls. Amazing to look at but nothing really special going on there. Since it works for you with Windows or Linux, use those OSes.  I sort of think it's the ultimate "up yours, Microsoft" to use a Windows OS as nothing more than a platform to virtualize Linux sessions that you are doing the actual work in LOL.  I mean, everyone and their dog has an OS now that can run virtual sessions it's nothing special anymore nor does it show any real cleverness in OS development to write an OS that can virtualize other OSes. The fact you are virtualizing at ALL is a knock against the host OS as it's basically an admission that the host OS does not have what it takes to get the work done so you have to go to the trouble of creating a virtual guest with a REAL os in it. Ted On 4/22/2023 5:16 PM, Tomek CEDRO wrote: > On Sat, Apr 22, 2023 at 10:16 PM Steve O'Hara-Smith wrote: >> Check BIOS settings, I've found virtualisation support disabled by >> default in the BIOS when the CPU supports it just fine. > Thanks Steve, all is enabled, I would not be able to start VM without > that.. the problem that I have is with "nested virtualization" :-) > From nobody Sun Apr 23 21:00:45 2023 X-Original-To: emulation@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q4LJY4jDSz46T4S for ; Sun, 23 Apr 2023 21:00:45 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q4LJY1qYQz3K68 for ; Sun, 23 Apr 2023 21:00:45 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1682283645; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=uYvzrZV7yPKAqON/Ghmr7XsbIsKBz2FHEIF8iKt3a4Y=; b=jQfs+V12Yi0rtKQWSxFmmrTSCD59B+npES9n1hdb+fAUFi6wWabOtCWpV/JcS+ftnK1Tns YBbl+g0DTz2hYzPf8OW5FsT4VoeMLM3x85ehbp0c4HnRWpv4vblN2F4YBkh1wbnWoAD7qa OFBlinymjg27RLfg3VA7TtbWiDKPsuprFs3t97eiKP0Ek/HmbqukcKKYCoA9l2zUmFIYy8 5+IG4CyT2DB/G7nxDUhGfBizDQXIUHMC11qWjHcy2NJBZe3vm6rYlA9EUsyex14pS/lsED HyGQceh1GXlx5R85rxJkw0Md/rTbVcU90qO/KxBtNF0inNVI8eI58S3RZqpnwg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1682283645; a=rsa-sha256; cv=none; b=nkR2Or+RWrmkpbBazPSAmhJEh6BreOIzb4oPGnVgI+nVVUYI+6q6EtWxHcuiOUfk1ulZKV FGDP6pNNAf+aWIhcbueJ1DHDIZd76GMLMHcrv45HyU0iD8E9HXDOyRm5AYgTq27zejICIk SCKms/QXeQd2L97ABHKes2NpfTkoVo5bMNmlcw0hyrgw4uxjOAUxiHphhaZmFZOLWLKzn/ 61xnDoz2vBpEI2Z5H4NYUcXhpA19K9zZPM8U+FEdaeOFuld+diuav46G39SLgD26Hexv+d tCg1IQBgrZgYw6Ch44ESAtqYIkGfsMDF172WYxLWYuyrLbkweDR2AJV8aDI47g== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Q4LJY0tHZzyJs for ; Sun, 23 Apr 2023 21:00:45 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 33NL0jvZ046591 for ; Sun, 23 Apr 2023 21:00:45 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 33NL0jst046588 for emulation@FreeBSD.org; Sun, 23 Apr 2023 21:00:45 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202304232100.33NL0jst046588@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: emulation@FreeBSD.org Subject: Problem reports for emulation@FreeBSD.org that need special attention Date: Sun, 23 Apr 2023 21:00:45 +0000 List-Id: Development of Emulators of other operating systems List-Archive: https://lists.freebsd.org/archives/freebsd-emulation List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-emulation@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="16822836450.a91D161bC.40354" Content-Transfer-Encoding: 7bit X-ThisMailContainsUnwantedMimeParts: N --16822836450.a91D161bC.40354 Date: Sun, 23 Apr 2023 21:00:44 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 269934 | emulators/qemu-user-static does not support capab 1 problems total for which you should take action. --16822836450.a91D161bC.40354 Date: Sun, 23 Apr 2023 21:00:44 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" 
The following is a listing of current problems submitted by FreeBSD users,
which need special attention. These represent problem reports covering
all versions including experimental development code and obsolete releases.

Status      |    Bug Id | Description
------------+-----------+---------------------------------------------------
New         |    269934 | emulators/qemu-user-static does not support capab

1 problems total for which you should take action.
--16822836450.a91D161bC.40354-- From nobody Mon Apr 24 09:53:02 2023 X-Original-To: freebsd-emulation@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q4gRw2V1Rz46Zts for ; Mon, 24 Apr 2023 09:53:16 +0000 (UTC) (envelope-from tomek@cedro.info) Received: from mail-yw1-x1132.google.com (mail-yw1-x1132.google.com [IPv6:2607:f8b0:4864:20::1132]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q4gRw0nylz3mRY for ; Mon, 24 Apr 2023 09:53:15 +0000 (UTC) (envelope-from tomek@cedro.info) Authentication-Results: mx1.freebsd.org; none Received: by mail-yw1-x1132.google.com with SMTP id 00721157ae682-54f8b46d031so46835207b3.3 for ; Mon, 24 Apr 2023 02:53:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cedro.info; s=google; t=1682329995; x=1684921995; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=LFh17tFYuXzT7aEVzlesgG6YsyCatpyHVIXYAyWxFQI=; b=aP/NhwiUibdRsXjMT3UKPqIvU3qDoCwH6tlnR/yYVN4VwPWc3FprAOZfdIUdDFyjMp wY18hiYL9yJ9IoVWUm/HutwHDwv5I0NptOFUDeO2gc6Y2IzRSHYSnJcjtw5f+WILGP/M NRHL7Swm+uA6jvgs99558cv0KgMFPGDoUsulBV1DIUsI5NIgraKK6sWj57UbTl1D8tub fOmaBMx4PXVxdCYV3G3iwSdWLywhxI/Z7Ik7sNyPpU3kGBn8qC8K3hFeSis/Q/fk0ofO /l6n8Lh5WyPi1ZTLl7+13CLkp4fCI1z/dVsYnwhVYCE7cB6wSz84OuA5qMRfLBYIAXpp IULA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682329995; x=1684921995; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LFh17tFYuXzT7aEVzlesgG6YsyCatpyHVIXYAyWxFQI=; b=eihc3oRVhaddfSTMa2o6FBc5J/znLftKsEVcQ0ZLiWugCMnU+gSVZ20sUQL/BG5sJk eYPM8R8rWZT6/CU/YbbNyy8/c2uMuIbRZhV7kDqT2BOqAzyRfpgK6Z2xfF8DFakgjkrz fXBHHppCjY9nUHr0so28MQ/ZihgLH8/xDBoP5nn5plWwSZTAMllXXEjMIO66CwsKGvOP NGQnsepHxxEnlABYoVUz/1CivJVnJSdJ/pPnDd7DekTN2zHRqWvHXywoOggSbnPC3F3d dyLF8meabz7+TpvRdklkHBq26ZaOyvv5dVu6SeNDg+GnxvjNA2v+SPhJwwUqjs8hUjkz hVHQ== X-Gm-Message-State: AAQBX9eFSiSM3vjVpPitSRJ0001MtU7movABVFkMEs2k6EQRJ1CukH2M eZOjmXkQAHTlwEq+Al1z0722Yg== X-Google-Smtp-Source: AKy350aTnjjZUd3oDdPH7bPIfbX1DaVmtDqRci570zrQUhP4khAf2CXA0YtlTlNTqkhC1nMGN7jO/Q== X-Received: by 2002:a0d:da85:0:b0:550:e009:9951 with SMTP id c127-20020a0dda85000000b00550e0099951mr8447892ywe.33.1682329994831; Mon, 24 Apr 2023 02:53:14 -0700 (PDT) Received: from mail-yw1-f180.google.com (mail-yw1-f180.google.com. [209.85.128.180]) by smtp.gmail.com with ESMTPSA id z128-20020a818986000000b0054fba0d6a88sm2813412ywf.100.2023.04.24.02.53.14 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 24 Apr 2023 02:53:14 -0700 (PDT) Received: by mail-yw1-f180.google.com with SMTP id 00721157ae682-555d2b43a23so46962627b3.2; Mon, 24 Apr 2023 02:53:14 -0700 (PDT) X-Received: by 2002:a81:6ed5:0:b0:556:2d9c:3491 with SMTP id j204-20020a816ed5000000b005562d9c3491mr4642499ywc.46.1682329994083; Mon, 24 Apr 2023 02:53:14 -0700 (PDT) List-Id: Development of Emulators of other operating systems List-Archive: https://lists.freebsd.org/archives/freebsd-emulation List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-emulation@freebsd.org MIME-Version: 1.0 References: <1033603027.10889563.1682198162191.JavaMail.zimbra@shaw.ca> In-Reply-To: From: Tomek CEDRO Date: Mon, 24 Apr 2023 11:53:02 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: embedded emulation: bhyve + vt-x on amd fx-8320 To: Pete Wright Cc: Dale Scott , FreeBSD Questions Mailing List , freebsd-emulation@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4Q4gRw0nylz3mRY X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Mon, Apr 24, 2023 at 5:24=E2=80=AFAM Pete Wright wrote: > > On Sun, Apr 23, 2023 at 02:25:44AM +0200, Tomek CEDRO wrote: > > installed Android Studio on that vm, then launched Virtual Android > > Device and IT WORKED! o_O > > > > So my AMD FX-8320 CPU seems to support "nested virtualization" (its > > called VT-D for Intel and AMD-V for AMD). > > > > Question is why Nested Virtualization does not work on FreeBSD? > > > > Should I set it up somehow? Is it supported here at all? > > I don't think nested hardware virt is supported by bhyve at the moment. > a reading of the man pages seems to confirm this as well. i've done para= -virt > nesting before but thats all in software and very slow. > > you might want to see if you can use a new'ish QEMU though to emulate the= hardware > needed to get android studio running. Thanks Pete, this is my main question if bhyve / vbox / kernel supports it at all :-) I got Android Studio running some time ago but there was problem with Gradle that did not want to detect FreeBSD as supported platform. Maybe I could make Android Emulator running on FreeBSD.. but also I need NDK and SDK for cross compilation.. and Google rejected my patches that reported FreeBSD as Linux in order to use Linux binaries, they say "this platform is unsupported" and by that they mean they won't even accept source code patches to make things work here. Thus my approach with virtualization.. but there Android Emulator does not start when no hardware acceleration is available. Some time ago I thought Microsoft sucked but Google sux even more.. not to mention Apple that does not even allow to virtualize macOS where XCode is the only tool to build iOS apps.. this whole brave new world of mobile applications is a good state of our current civilization ;-) I hoped someone on the list would know the current state of "nested virtualization" in FreeBSD.. if its currently supported and/or maybe a timeline when it may be available :-) --=20 CeDeROM, SQ7MHZ, http://www.tomek.cedro.info From nobody Mon Apr 24 10:04:03 2023 X-Original-To: freebsd-emulation@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q4ghf0cV7z46bvt for ; Mon, 24 Apr 2023 10:04:18 +0000 (UTC) (envelope-from tomek@cedro.info) Received: from mail-yw1-x1134.google.com (mail-yw1-x1134.google.com [IPv6:2607:f8b0:4864:20::1134]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q4ghd5Wdkz43P8 for ; Mon, 24 Apr 2023 10:04:17 +0000 (UTC) (envelope-from tomek@cedro.info) Authentication-Results: mx1.freebsd.org; none Received: by mail-yw1-x1134.google.com with SMTP id 00721157ae682-54fc1824f0bso46947207b3.0 for ; Mon, 24 Apr 2023 03:04:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cedro.info; s=google; t=1682330657; x=1684922657; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=XT9QiHIYZcFooJQuNJDMTGafMkIZM37sZ0e3N/yUPcI=; b=R3L6UhFjMq54ko8XdeaGePwz/3qlwxncfBTUVmbsPh18dhBSCUwp0PPTrtrmRvWJRg G+QpCnlEsloHzlJTa/Gb3+d6wNMdug4aldpUOQXuw8W+CkSp2mzVsY+HRChoMUpmrCa6 BwAO24AyhB2yw+JfQagASkHN9ePePm8BXvL12XRrK/7I5mr7zyyOtBMfd/fcMfT6ebMq mrOG3OjNzsimTslyNhNftufNCNNtIQbmwffbwAcHE962yViIp19MyHydmou4xUcpimne mUxwL44zMlbyM0rhh9WpT/6rlceN8e217nmw9vnAY2HnPNDJeaqhPIDPzCcq7jL/THF7 iQPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682330657; x=1684922657; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XT9QiHIYZcFooJQuNJDMTGafMkIZM37sZ0e3N/yUPcI=; b=S74G5rthZ+RHv31ZP4GMyhCtYzs+HBkG2ntr/WBCTaOLCJyiAeKhlsfULB/9/KAITp 3gBu07UdyEcGYgOTl2tCvEqfvWvO8Oa6jK8z/Hk6dxYr3E+PHucsUp5sEs/jlmVWy2Ya DwsCSlC+rKRJ8bfAWP/pV7skLZ8JbO+OdGLyD7c29JwtgXak0LMlzeUT1GoQ78qbBZGg bdoRaCRKBHTmfU6Q/yYYQ2iEwOAYS2T5TFzB//vAGW+PEbmVdb6UeNVZ+Y5Gt8sM7fuJ nYmqQ2EmuSU5Zf9VqF0D7SRXkL2oqo4a6psqtbM4gFxHDQF0HBApUNwt84m8bOh9+/H9 gSFw== X-Gm-Message-State: AAQBX9cKN3RC8tVQ6tWjDS5NELknrYvaBKqgGmPsaeLFBdUHdrSr9CfE /53dnMEfRLVJyT0hmcpiKN5QzGbPdcTh2Plw6po= X-Google-Smtp-Source: AKy350bcz9RMYxvdHf5cSeDBI3fX2B3/4kATYsRerPhWeELZiGGJV/Od/vAM0290iO75V8alrMEquA== X-Received: by 2002:a81:6c0d:0:b0:556:1070:7c60 with SMTP id h13-20020a816c0d000000b0055610707c60mr8164089ywc.18.1682330656867; Mon, 24 Apr 2023 03:04:16 -0700 (PDT) Received: from mail-yw1-f171.google.com (mail-yw1-f171.google.com. [209.85.128.171]) by smtp.gmail.com with ESMTPSA id o124-20020a0de582000000b00545a08184c6sm2840045ywe.86.2023.04.24.03.04.16 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 24 Apr 2023 03:04:16 -0700 (PDT) Received: by mail-yw1-f171.google.com with SMTP id 00721157ae682-54fc6949475so46807607b3.3 for ; Mon, 24 Apr 2023 03:04:16 -0700 (PDT) X-Received: by 2002:a0d:cc83:0:b0:54f:bec1:c118 with SMTP id o125-20020a0dcc83000000b0054fbec1c118mr8160872ywd.38.1682330655839; Mon, 24 Apr 2023 03:04:15 -0700 (PDT) List-Id: Development of Emulators of other operating systems List-Archive: https://lists.freebsd.org/archives/freebsd-emulation List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-emulation@freebsd.org MIME-Version: 1.0 References: <20230422211625.fbdd0c384b6d144dd9502348@sohara.org> <3774d906-e8f6-1330-7605-515f34084ff4@mittelstaedt.us> In-Reply-To: <3774d906-e8f6-1330-7605-515f34084ff4@mittelstaedt.us> From: Tomek CEDRO Date: Mon, 24 Apr 2023 12:04:03 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: embedded emulation: bhyve + vt-x on amd fx-8320 To: Ted Mittelstaedt Cc: freebsd-emulation@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4Q4ghd5Wdkz43P8 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Sun, Apr 23, 2023 at 7:02=E2=80=AFPM Ted Mittelstaedt wrote: > That requires more than just CPU hardware support. Unlike what you > think at the hardware level there is no such thing as "nested > virtualization" instead each virtual session is separate the OS and CPU > just keep track of what session is tied to what other session. Sort of > like trained monkeys juggling colored balls. Amazing to look at but > nothing really special going on there. Probably yes but the kernel / hypervisor code must be here and this is my initial question :-) > Since it works for you with Windows or Linux, use those OSes. I sort of > think it's the ultimate "up yours, Microsoft" to use a Windows OS as > nothing more than a platform to virtualize Linux sessions that you are > doing the actual work in LOL. I mean, everyone and their dog has an OS > now that can run virtual sessions it's nothing special anymore nor does > it show any real cleverness in OS development to write an OS that can > virtualize other OSes. Well in theory the Android is Open-Source so it should work everywhere, even if patches are required. I just avoid Windows and this is my "up yours". The same with Linux. FreeBSD is my choice and I feel best here, I do not really care about other OS and have nothing to prove to anyone, apples and oranges :-) > The fact you are virtualizing at ALL is a knock against the host OS as > it's basically an admission that the host OS does not have what it takes > to get the work done so you have to go to the trouble of creating a > virtual guest with a REAL os in it. Except the "Open-Source" part that is tied by some company to a binary releases and does not accept patches to make things work on other OS ;-) The best solution for mobile apps development would be a modern macOS machine, but decent configuration cost around $7.5k here (MacStudio 20xARM CPU 128GB RAM) and it has predefined lifetime (my MacBookPro 2015 stopped at macOS 12.6.5 while macOS 13.3 is out and that impacts XCode development tools too). This seems a necessary cost. Well, goodbye free applications. --=20 CeDeROM, SQ7MHZ, http://www.tomek.cedro.info From nobody Wed Apr 26 14:56:23 2023 X-Original-To: freebsd-emulation@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q624x1Dpbz46d5F; Wed, 26 Apr 2023 14:56:33 +0000 (UTC) (envelope-from byshenknet@byshenk.net) Received: from v1.leiden.byshenk.net (v1.leiden.byshenk.net [IPv6:2a01:7c8:aac3:6f1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q624v4T3Vz3Gjp; Wed, 26 Apr 2023 14:56:31 +0000 (UTC) (envelope-from byshenknet@byshenk.net) Authentication-Results: mx1.freebsd.org; dkim=none; spf=softfail (mx1.freebsd.org: 2a01:7c8:aac3:6f1::1 is neither permitted nor denied by domain of byshenknet@byshenk.net) smtp.mailfrom=byshenknet@byshenk.net; dmarc=none Received: by v1.leiden.byshenk.net (Postfix, from userid 1002) id 2D5B9759AA; Wed, 26 Apr 2023 16:56:23 +0200 (CEST) Date: Wed, 26 Apr 2023 16:56:23 +0200 From: Gregory Byshenk To: Tomek CEDRO Cc: FreeBSD Questions Mailing List , freebsd-emulation@freebsd.org Subject: Re: embedded emulation: bhyve + vt-x on amd fx-8320 Message-ID: References: <1033603027.10889563.1682198162191.JavaMail.zimbra@shaw.ca> List-Id: Development of Emulators of other operating systems List-Archive: https://lists.freebsd.org/archives/freebsd-emulation List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-emulation@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spamd-Result: default: False [0.11 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.998]; NEURAL_SPAM_LONG(0.93)[0.926]; FORGED_SENDER(0.30)[freebsd@byshenk.net,byshenknet@byshenk.net]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_SHORT(-0.02)[-0.022]; MLMMJ_DEST(0.00)[freebsd-emulation@freebsd.org,freebsd-questions@freebsd.org]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:20857, ipnet:2a01:7c8::/32, country:NL]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[freebsd@byshenk.net,byshenknet@byshenk.net]; RCPT_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[byshenk.net]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4Q624v4T3Vz3Gjp X-Spamd-Bar: / X-ThisMailContainsUnwantedMimeParts: N On Mon, Apr 24, 2023 at 11:53:02AM +0200, Tomek CEDRO wrote: > Thanks Pete, this is my main question if bhyve / vbox / kernel > supports it at all :-) >From my experience it does. Some time back I upgraded the CPUs on an HP Z800 because the original ones did not support VT-D. Then I had to enable that in the BIOS. But afterwards I was able to run a rancher cluster under bhyve to play with. it was only a toy, but it worked. -- gregory byshenk - gbyshenk@byshenk.net - Leiden, NL From nobody Wed Apr 26 17:51:53 2023 X-Original-To: freebsd-emulation@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q65zV1g2Xz474P5 for ; Wed, 26 Apr 2023 17:52:06 +0000 (UTC) (envelope-from tomek@cedro.info) Received: from mail-yb1-xb29.google.com (mail-yb1-xb29.google.com [IPv6:2607:f8b0:4864:20::b29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q65zV1CY1z3tPX for ; Wed, 26 Apr 2023 17:52:06 +0000 (UTC) (envelope-from tomek@cedro.info) Authentication-Results: mx1.freebsd.org; none Received: by mail-yb1-xb29.google.com with SMTP id 3f1490d57ef6-b9554ad3721so11395820276.0 for ; Wed, 26 Apr 2023 10:52:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cedro.info; s=google; t=1682531525; x=1685123525; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=yOyLvsegG4LiPfQcH08XadMMcXihI1Km8HTMdbBizqg=; b=jMVfqo1uDdD4OCUUQhkwUVG2lvsCnL0rw6ETFUc5ezJMHrjQvKNOHicHCHxOsAN9Wm 72xNn5Dv6CR8D7YtsaK4QaovTN+36rYfG5m8Vns9X6jrhzrwdgIMo2Ue4J2VmCc7MATf Ma7uR58urDA9m95vTVudSDBHWwsVsGi02Dq+T7s1q9df4ezT5IwBjFEAppXD3NQ2n/a/ RNLERrEvvQi0OYsO4NUyFC/HEqrh3zitGFEpJdyOEbFc2H+G23lEYrHYCWILA/kyIeRr cH4GxWTSfPsy6xfED5vQQyB1y2A5J/+jw92uxjrRCBJcUiEqqcJeQiEXDQPr49Q6S8lJ 5LVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682531525; x=1685123525; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yOyLvsegG4LiPfQcH08XadMMcXihI1Km8HTMdbBizqg=; b=fXmfU+egCk8NW2v+QUmNH2anY6PTvdPPuQvxr4MkCcqOlC09W8+L0qUBKaRYaKbZqA p4HmIKNRzcKvAQXNOGQROBbdxikWHhNJMn66lfewNNLAjHHRzvE2vgjmThGfpqXlzi/T YyIchjKhkpMH5dkRGxfxxNEibZ84O73S5ZKXNYURqX3A7/uru9XZnza266oOQpXtcvDY DPvTw6+a4ciksdHYf6GLW28xCzCcFgFdKaxjEekhlPqlohNHMJiZokDtUwuhjXEVt6Rj H8Ro3GLOhooQOU0u30KigPMReHVa/jDdnDE28lr1GATHjqq354e2lg/iIlNS48uwdUR2 N5IQ== X-Gm-Message-State: AAQBX9fp/eYglQlGpM+p9dcd6spK2tjY2r50BQMykopzd+IvEM4GJCTU CGbETcZvKoDtgyef85Qn+HJHqA== X-Google-Smtp-Source: AKy350YMht97LaZA3r0FTsVD5mwW9iK1z/JQUl86pg8md8U9gmCRiaqY8ym9VRaLkaee0V61PRuC3A== X-Received: by 2002:a25:d683:0:b0:b8f:57bd:45fe with SMTP id n125-20020a25d683000000b00b8f57bd45femr13770083ybg.10.1682531525428; Wed, 26 Apr 2023 10:52:05 -0700 (PDT) Received: from mail-yw1-f174.google.com (mail-yw1-f174.google.com. [209.85.128.174]) by smtp.gmail.com with ESMTPSA id i3-20020a258b03000000b00b8c08669033sm4102720ybl.40.2023.04.26.10.52.04 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Apr 2023 10:52:05 -0700 (PDT) Received: by mail-yw1-f174.google.com with SMTP id 00721157ae682-54f99770f86so88046167b3.1; Wed, 26 Apr 2023 10:52:04 -0700 (PDT) X-Received: by 2002:a0d:db82:0:b0:555:cce2:8a16 with SMTP id d124-20020a0ddb82000000b00555cce28a16mr12237185ywe.22.1682531524632; Wed, 26 Apr 2023 10:52:04 -0700 (PDT) List-Id: Development of Emulators of other operating systems List-Archive: https://lists.freebsd.org/archives/freebsd-emulation List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-emulation@freebsd.org MIME-Version: 1.0 References: <1033603027.10889563.1682198162191.JavaMail.zimbra@shaw.ca> In-Reply-To: From: Tomek CEDRO Date: Wed, 26 Apr 2023 19:51:53 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: embedded emulation: bhyve + vt-x on amd fx-8320 To: Gregory Byshenk Cc: Tomek CEDRO , FreeBSD Questions Mailing List , freebsd-emulation@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4Q65zV1CY1z3tPX X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Wed, Apr 26, 2023 at 4:56=E2=80=AFPM Gregory Byshenk wrote: > On Mon, Apr 24, 2023 at 11:53:02AM +0200, Tomek CEDRO wrote: > > Thanks Pete, this is my main question if bhyve / vbox / kernel > > supports it at all :-) > > From my experience it does. > Some time back I upgraded the CPUs on an HP Z800 because the > original ones did not support VT-D. Then I had to enable that > in the BIOS. > But afterwards I was able to run a rancher cluster under bhyve > to play with. it was only a toy, but it worked. Thank you Greg. I see no Virtualization settings in my BIOS. Probably the hardware is too old for that and need an upgrade after 10 years :-) Thank you for the confirmation! :-) --=20 CeDeROM, SQ7MHZ, http://www.tomek.cedro.info From nobody Wed Apr 26 20:50:01 2023 X-Original-To: freebsd-emulation@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q69x411Kfz47Hct for ; Wed, 26 Apr 2023 20:50:16 +0000 (UTC) (envelope-from tomek@cedro.info) Received: from mail-yw1-x112a.google.com (mail-yw1-x112a.google.com [IPv6:2607:f8b0:4864:20::112a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q69x33d8Nz4Q0m for ; Wed, 26 Apr 2023 20:50:15 +0000 (UTC) (envelope-from tomek@cedro.info) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=cedro.info header.s=google header.b=PEZEh+ug; spf=none (mx1.freebsd.org: domain of tomek@cedro.info has no SPF policy when checking 2607:f8b0:4864:20::112a) smtp.mailfrom=tomek@cedro.info; dmarc=none Received: by mail-yw1-x112a.google.com with SMTP id 00721157ae682-54fb89e1666so59500147b3.3 for ; Wed, 26 Apr 2023 13:50:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cedro.info; s=google; t=1682542214; x=1685134214; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=P2zaddbV48G06oWm77s/OC7u9RBcdJMYPXRLkdvYqkI=; b=PEZEh+ug4HQY7/nErrmQ1uOmfczazHHwxtcaGTMF3CHORhD+LmxhnR9RTwFRGocZl4 U86Ff0oZLKkKNwugMVB+AyJEvCceBukdwOZ/TBbb6izgttefmxCRj6SE9uQqZwmzvc5H 6vQMoQ8upllzN/kfm7Y126v0o/WmxAvq9FQh8UKAPAcfJYPJ6TMKTO7fABJJ3LsliaOZ 9EUC+ZJj5UlLSs7l0bgzWSoAk+1aGCcnxbQVdgd72N5Y7ePlSuSpOodKNw8NaBYLW63P mudTeW97OR7Xqwnw8vWy2J63fUt+Nu9fzcZ0VHA8M8IxC362ymLXt2q70BvbMMM9LSzF Izsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682542214; x=1685134214; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=P2zaddbV48G06oWm77s/OC7u9RBcdJMYPXRLkdvYqkI=; b=KBdcPAa+L+EUwzDyECGWRqcqYZ5JQWZ6x3Wdp6rijU8PiYKeLZKeBE8pAXNeeyPqUa ctSQo568EU4VZonjCAEQBtzM/sWjyfJaI4ysSFkhWjfkVMKr1U7ID59zqyyMSopSWfxR jqUFjEeQ0bkRNn+t+9sPWR+UrhJGfcUm3cglbs8J4MCBAa7HhGJEmjcimGA91y7HMtY/ 2mgA34gR71vTx79Jtt9oMURBVDCtyWWpDhUUcSuXXYvyYFPPwg5FKU7HOehw3KqaAoTX usuIQHYvbJi2zWvjjl3fhWXtm0C8W++aBrCKCd5uaIOqk9hbtMNPrdpoya+qU4rLxHsZ l5sQ== X-Gm-Message-State: AAQBX9cRdc0E7+NdJOYlB7HBSVUWXgE/+zpNlRDBbRakJAZRSH1uNn/I ufRG+cX3iGgBpZseku5XbR5d9w== X-Google-Smtp-Source: AKy350bNRF1vkdYnzSKNAGQiZ3frBGVgkp6oPPJXqtGlImO0iiBUj9CmYMrWanWNWpFl7GTrkat08w== X-Received: by 2002:a0d:ca92:0:b0:54e:d70a:fa5f with SMTP id m140-20020a0dca92000000b0054ed70afa5fmr13957130ywd.49.1682542214292; Wed, 26 Apr 2023 13:50:14 -0700 (PDT) Received: from mail-yw1-f175.google.com (mail-yw1-f175.google.com. [209.85.128.175]) by smtp.gmail.com with ESMTPSA id y195-20020a0dd6cc000000b0054f9a3b60a1sm4442428ywd.33.2023.04.26.13.50.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Apr 2023 13:50:13 -0700 (PDT) Received: by mail-yw1-f175.google.com with SMTP id 00721157ae682-5529f3b8623so59485637b3.2; Wed, 26 Apr 2023 13:50:13 -0700 (PDT) X-Received: by 2002:a0d:f047:0:b0:54f:d245:a79d with SMTP id z68-20020a0df047000000b0054fd245a79dmr13738060ywe.34.1682542213350; Wed, 26 Apr 2023 13:50:13 -0700 (PDT) List-Id: Development of Emulators of other operating systems List-Archive: https://lists.freebsd.org/archives/freebsd-emulation List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-emulation@freebsd.org MIME-Version: 1.0 References: <1033603027.10889563.1682198162191.JavaMail.zimbra@shaw.ca> In-Reply-To: From: Tomek CEDRO Date: Wed, 26 Apr 2023 22:50:01 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: embedded emulation: bhyve + vt-x on amd fx-8320 To: =?UTF-8?Q?Jan_Jur=C3=A1k?= Cc: Gregory Byshenk , FreeBSD Questions Mailing List , freebsd-emulation@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-1.80 / 15.00]; SUSPICIOUS_RECIPS(1.50)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; R_DKIM_ALLOW(-0.20)[cedro.info:s=google]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-emulation@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::112a:from,209.85.128.175:received]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; DKIM_TRACE(0.00)[cedro.info:+]; RCVD_COUNT_THREE(0.00)[4]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-emulation@freebsd.org]; TAGGED_RCPT(0.00)[]; DMARC_NA(0.00)[cedro.info]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4Q69x33d8Nz4Q0m X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N On Wed, Apr 26, 2023 at 10:15=E2=80=AFPM Jan Jur=C3=A1k wrote: > answer as usually late, but you can check procesor virtualisation flags w= ith command: > (..) > On Freebsd: > > egrep -woi 'vmx|ept|vpid|npt|tpr_shadow|flexpriority|vnmi|lm|aes' /var/ru= n/dmesg.boot | sort | uniq; egrep -woi 'vmx|lm|aes' /var/run/dmesg.boot | s= ort | uniq > > AES > EPT > LM > VMX > VPID > AES > LM > VMX > > this is for intel, amd has it's own implementation for bot. Basically VMX= is minimum, the extended page support - EPT is second most important not m= inimum but most modern hypervisors like bhyve also need this one. > i believe for amd you are looking for AMD-v SVM but rather verify if corr= ect acronyms. Thanks Jan.. I also found dmidecode very useful to find lots of information about the host hardware.. flags are not there.. just curious why this virtual machine in virtual machine worked on windoze o_O Handle 0x0004, DMI type 4, 42 bytes Processor Information Socket Designation: Socket 942 Type: Central Processor Family: FX Manufacturer: AMD ID: 20 0F 60 00 FF FB 8B 17 Signature: Family 21, Model 2, Stepping 0 Flags: FPU (Floating-point unit on-chip) VME (Virtual mode extension) DE (Debugging extension) PSE (Page size extension) TSC (Time stamp counter) MSR (Model specific registers) PAE (Physical address extension) MCE (Machine check exception) CX8 (CMPXCHG8 instruction supported) APIC (On-chip APIC hardware supported) SEP (Fast system call) MTRR (Memory type range registers) PGE (Page global enable) MCA (Machine check architecture) CMOV (Conditional move instruction supported) PAT (Page attribute table) PSE-36 (36-bit page size extension) CLFSH (CLFLUSH instruction supported) MMX (MMX technology supported) FXSR (FXSAVE and FXSTOR instructions supported) SSE (Streaming SIMD extensions) SSE2 (Streaming SIMD extensions 2) HTT (Multi-threading) Version: AMD FX(tm)-8320 Eight-Core Processor Voltage: 1.4 V External Clock: 200 MHz Max Speed: 3600 MHz Current Speed: 3600 MHz Status: Populated, Enabled Upgrade: Socket AM3 L1 Cache Handle: 0x0005 L2 Cache Handle: 0x0006 L3 Cache Handle: 0x0007 Serial Number: To Be Filled By O.E.M. Asset Tag: To Be Filled By O.E.M. Part Number: To Be Filled By O.E.M. Core Count: 8 Core Enabled: 8 Thread Count: 8 Characteristics: 64-bit capable --=20 CeDeROM, SQ7MHZ, http://www.tomek.cedro.info From nobody Fri Apr 28 09:21:45 2023 X-Original-To: emulation@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q76Yk33ypz48DRn for ; Fri, 28 Apr 2023 09:21:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q76Yk0Z3lz4ZYR for ; Fri, 28 Apr 2023 09:21:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1682673706; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=x5FBo6wfwyhyb0OwOnL4QqiVetIQKHDhnhntEMESuzo=; b=K3vLMWWDQSoe7hzNnQ27IcDOfHkdGhllcSpnrtPkEMkr5EurWqPaxMCoPxsNvk2PlDVyW9 q7JgipCtvV6fIebIdxv17RCZxtl0xaPe7Dxk1eAZqeFH6E+Gbjkb+LUe/YeL7q60Yvxtmk WMc3DW16sjjZiuEqLg2ZG90BLZBsb2ATuLbgbQDgdLAW/fd8xhjDcCUp/UUUULfxV/y1hr +KLR18vrktaQxFJwQsHGb29Ah6zgCbWP2xq/W7KKGkpM4kLtnsWMrQa+vroK6VO0xjhr41 X3BpM31lqkLFvHiZJ5QNbVYRB48N6J0JLP/cx7aWaVZFxRF0vk6wlOwv1/ItkQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1682673706; a=rsa-sha256; cv=none; b=HwUTEEXbgQq/WmY1ADlYhjHgPVId1hXJmtMdmeyaqGbgmDFlnAd6OhNRf+DX7jvV1Qq53G y0NKmwLNVVgbOsjCbeDNPlMiZ8jxDWC0eLThm2Fva+Fm/WXdPFB+acHTfUnrj9GXDkCFUt PY2OGNN+Lw2ETv4GcwKhX3N41cJcrJJMToLYgCV8ety74lhoBZhGBjFd3FRS/Gqgl3rgnT mAzjmjFuTq6ol4gg44PYPmxYW23bPn0b3alBTdaFb2jqxzfw3sp3dgAjAYLiwPxPPjrGMq Y1oI9SaLn3GFcP2lShMWNYc3agE/JJaF3Gi0aD6FoE0zvQ6ljslo4kWVbMlu2w== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Q76Yj6mK6ztpj for ; Fri, 28 Apr 2023 09:21:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 33S9Lj32025451 for ; Fri, 28 Apr 2023 09:21:45 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 33S9Ljr6025450 for emulation@FreeBSD.org; Fri, 28 Apr 2023 09:21:45 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: emulation@FreeBSD.org Subject: [Bug 267686] emulators/qemu, emulators/qemu-devel, emulators/qemu-user-static: review WWW Date: Fri, 28 Apr 2023 09:21:45 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: fuz@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bofh@freebsd.org X-Bugzilla-Flags: maintainer-feedback? maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Development of Emulators of other operating systems List-Archive: https://lists.freebsd.org/archives/freebsd-emulation List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-emulation@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D267686 --- Comment #4 from Robert Clausecker --- *** Bug 267685 has been marked as a duplicate of this bug. *** --=20 You are receiving this mail because: You are on the CC list for the bug.=