From nobody Mon Jan 29 16:35:39 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TNv6z3GYYz590rH for ; Mon, 29 Jan 2024 16:35:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TNv6z2BhPz4p54 for ; Mon, 29 Jan 2024 16:35:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706546139; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DOBG1OyqRMvnM0vd0MJKrXrFJ+2m9uFGfxvU3k0fr7U=; b=hAFNyc42etf739/MMNDL+z3AGIZl4w0GPPZ8O6wZ6X5caOc/erSl9NRpS7R5lZFaJ3FbmM Aw3zFlDtZaKQzPrWK4MkN0dkHdhpAJZnTYuP2pPcugM2851+rd4tpa5Afc//RIS7v+4OvG oqNbUYxAckuJino8cnFAPnxbL+eikyZeYO2l9yQ43HF24NnTIW9UxKfQWadCJMhqnFrMHO bndxnlyKdbLexePXxgxo0kJr6LWII7zRWiBDJNCzOegknAHMJlqmZdaqc6AMgL4mY+omHE JFdkvuQiwM/XBuQiv3weG7z/VjOpYfDuH1UNshBo1Txrx5lXy2tFrkzi8y+NHA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706546139; a=rsa-sha256; cv=none; b=cSCQIJb/TZbyN64gWDxUFblU967MNQ3craY0hQCCHIm3A0eH/KlHwisI/rD2A7MVJpA0eC EBQGbMdBVZz5uzFQCF9uOuQ80No1ZRw+HGpR0R+gMtJIcxsLAO+8tAlWJ5JdH2GdtfV67R eplA81EuyAywVRGNmOa61VcvXffWPZPqAvOsxurQ6wleo/FCgrUvJhaYWOPQ2ft/ySQQB/ ktbSkgCDUSW6RHQRkWNZzuNulkiyaFfqQGU3NT7CHsOV85DuufYFBz7MqZoDusVc97bV1v Ko8OZGtbMm15sPB01RormmpSOHJ0NEf+OJXZYR5jPzH3yznvzaoiRmfgGyVN4w== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TNv6z1GN0zyn8 for ; Mon, 29 Jan 2024 16:35:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 40TGZdtJ067069 for ; Mon, 29 Jan 2024 16:35:39 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 40TGZdDn067068 for virtualization@FreeBSD.org; Mon, 29 Jan 2024 16:35:39 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 276575] Host can cause a crash in bhyve nvme emulation Date: Mon, 29 Jan 2024 16:35:39 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 14.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: chuck@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: chuck@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276575 Chuck Tuffli changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|virtualization@FreeBSD.org |chuck@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Mon Jan 29 20:38:56 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TP0Wj0H88z59N6b for ; Mon, 29 Jan 2024 20:38:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TP0Wh4K9Fz4Mtl for ; Mon, 29 Jan 2024 20:38:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706560736; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3w0SkddtcmjTh8H7dZXnd/+VY0ECp3CmpqpP3d/kHPo=; b=UgbzPiGvFsPNVzR8TTQlDv2G9uB+R4PGyGZNiQRVc4hheuiGSUnNd1v/8t4ZmpMCOLjSqV rWBHcCGwFwUxz+BKjodqhdNpii7RtUtSVMG83HsfDEguo+JOI0neepyiteEt17S4LxxPZU ME8aYsr9/bmwIdDq/C9IpqyVsbVTjPDPa2GzGrKcEO4HNcRhmw0YIv7ibr2iqOMjWU4IhJ OSBkSb3uIZMiRZ31dMFo60xb/GqsmW96Hy38fSg/F8KEe7v66V8MjXFaMaXtMm9ybsl9sb noohCp2LrLic/VhNZe5Lg+g0O1xK5ECwfZXhzRQOMMWggkBu1/4XDPckGdpemg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706560736; a=rsa-sha256; cv=none; b=DsiNu4kTdL65W8uu95+Kllzyxo+sfIXWn3dCmUhwRjExFp8smJp4wYZmyWZ3tLVPJ+HZ0/ +U/EjEJND0BRHC36jI3S+KLSAYD5CwfQ8HaJ7oOG2af4htd0BtHVraV2xz2iFiuaRc2ICi JBh79i/x1ko7BG2c5rjw3Gmm7M5oXuL1RyIuFz/oGvOiLWAxQAKftP85K6PP3THQXTT5lb RM0GkJU/xO9sqTN8krVEzEdWrQvADM/valA/tp0FcnBrp0KIOXHEC5JkcIZemJRil93V2b HKFJUgCxqtYJvGesU9gFASSwVkFdZycDscHi0MU1TB8H73UZLwhvwqU15HNRCA== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TP0Wh3NG4z1625 for ; Mon, 29 Jan 2024 20:38:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 40TKcuKt003158 for ; Mon, 29 Jan 2024 20:38:56 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 40TKcu4l003157 for virtualization@FreeBSD.org; Mon, 29 Jan 2024 20:38:56 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 276720] AMIs fail to boot in AWS EC2 Date: Mon, 29 Jan 2024 20:38:56 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276720 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |virtualization@FreeBSD.org CC| |cperciva@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Mon Jan 29 20:49:18 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TP0lg1p0Nz59Nx0 for ; Mon, 29 Jan 2024 20:49:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TP0lf59jzz4NwY for ; Mon, 29 Jan 2024 20:49:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706561358; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=swNEqiEIlbI02k9fg5RlqRF3DgElv12Z7btiA0Wu93Y=; b=Be8/+gfB2NX7eiR6ZozKTed6J9MfimEmUBQVa6ntzCiGY/TS+pAkoZZ7LIMxApafWWIoxw /9be3Q66qHYJZE26uMsYxMwV9SQwEV7c8uu+2jZ0zBqTRkoZZujOn0OxlLi50cIIxYawVc FqeFOGuUDypYNm0JMi1/doPIZ4Io9BsNiK0u8bcCJ4ag+4i8chH4RPnQCGGv8DAbH2tHgY 9CxJjOfm7KPuzZtmhELm6gSUnDNq7ndj935Za2t2yyWireVNJOr4eJl/pfgvLYgyfUZ87q ZEpKzaFTg2yxNbJed9crwZ5+tTszYhtctzhhf9xU4C1Hw16mIQ8HGnHEMEwLtw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706561358; a=rsa-sha256; cv=none; b=EiDOx9rzUBLuRLRd/W4S6x7KRNKUNUs5bxH5oQOGf13FzKdTSDf0XjdxPlTzXa7gO9LSEG KD5kD8VffT8CG/BxBOstCRAyVBLt24wS8dDTV7qioZpthLtezQV+HHZ++78IE9frN9/mxf DIyhJH48ue76oEZCmD0A63r5mCpusYXFubR7lng4J17EzL0sj9RRr4kUpbDMFSV8tErPTf WbQ6Ic3sVZxZwdRJMM4J+ss/BcySm2HY6vPnv6mjhOy/uvegcos6cbfz17Pi4MvPx5/MjG ekYjAO2dgC1ZSpK0HPY2Ec0Blflv0r7bgwVFbSwGlP1JCBeiv1iEa93GsCT/hg== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TP0lf4DV0z16Q3 for ; Mon, 29 Jan 2024 20:49:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 40TKnI0k053015 for ; Mon, 29 Jan 2024 20:49:18 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 40TKnImL053011 for virtualization@FreeBSD.org; Mon, 29 Jan 2024 20:49:18 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 276720] AMIs fail to boot in AWS EC2 Date: Mon, 29 Jan 2024 20:49:18 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: cperciva@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276720 --- Comment #1 from Colin Percival --- Can you tell me the exact AMI IDs you tried? --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Mon Jan 29 22:41:45 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TP3Fc6Mhkz57bqf for ; Mon, 29 Jan 2024 22:41:56 +0000 (UTC) (envelope-from jo@durchholz.org) Received: from www382.your-server.de (www382.your-server.de [78.46.146.228]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TP3Fb2VDZz4mrq for ; Mon, 29 Jan 2024 22:41:55 +0000 (UTC) (envelope-from jo@durchholz.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=durchholz.org header.s=default2202 header.b=fVmuTMT1; dmarc=none; spf=pass (mx1.freebsd.org: domain of jo@durchholz.org designates 78.46.146.228 as permitted sender) smtp.mailfrom=jo@durchholz.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=durchholz.org; s=default2202; h=Content-Transfer-Encoding:Content-Type: Subject:From:To:MIME-Version:Date:Message-ID:Sender:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References; bh=psfxLCj5fxuJFDy1eN0V/pPZElVtr+Qb+LD0WiqH9RQ=; b=fVmuTMT1srqoxU3AtxYpBdKICI grJ6vONWk4bmInatfGfGF7goe5AnP2A0c/tngSHuBG6zjqLEW31A7WJr73DZNF24Y2oFZHL3+ggS8 exLC3eOW0Dr5yF8qb4ZbR5RqX9+NRnEn3dDgXbdxh9uUBl4phy1+FxmHoESqF8+qzi1+wktwbyeLl 4RwfgVA+VzF/VtD8Tf1mpt6y//9OVWtkAdqKN4ISBN0jWwlmPbgJirq2H0U/PSgvQ5bEXJNKhH6bf YT+C5+Zg4dx79LoIICO1dwmFZ5meZgTBRADzMWV/oRb4bzgRDT+445VB52bhfVYt6HtfHSE2j6Hvk gpyorivw==; Received: from sslproxy05.your-server.de ([78.46.172.2]) by www382.your-server.de with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rUaK2-000Kkn-EF for freebsd-virtualization@freebsd.org; Mon, 29 Jan 2024 23:41:46 +0100 Received: from [2003:ed:d72b:9c00:1101:617a:f2c9:379b] by sslproxy05.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rUaK2-0000cp-9s for freebsd-virtualization@freebsd.org; Mon, 29 Jan 2024 23:41:46 +0100 Message-ID: Date: Mon, 29 Jan 2024 23:41:45 +0100 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: "freebsd-virtualization@FreeBSD.org" From: Jo Durchholz Subject: Contributing the build of a KVM image Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Authenticated-Sender: jo@durchholz.org X-Virus-Scanned: Clear (ClamAV 0.103.10/27169/Mon Jan 29 10:39:53 2024) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.47 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.984]; R_SPF_ALLOW(-0.20)[+a]; R_DKIM_ALLOW(-0.20)[durchholz.org:s=default2202]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:24940, ipnet:78.46.0.0/15, country:DE]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[durchholz.org]; HAS_X_AS(0.00)[]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_EQ_ADDR_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; DKIM_TRACE(0.00)[durchholz.org:+] X-Rspamd-Queue-Id: 4TP3Fb2VDZz4mrq Hi all, where do I turn to if I want to contribute to the FreeBSD build/release process so that a KVM image will be built? This is not actually about virtualization per se but about the build&release process, so I guess I'd be wrong here. Regards, Jo From nobody Tue Jan 30 00:28:02 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TP5cJ3433z57nGJ for ; Tue, 30 Jan 2024 00:28:16 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from mail.nomadlogic.org (mail.nomadlogic.org [66.165.241.226]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TP5cH4Nggz40FH for ; Tue, 30 Jan 2024 00:28:15 +0000 (UTC) (envelope-from pete@nomadlogic.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=nomadlogic.org header.s=04242021 header.b=M1keZwxt; dmarc=pass (policy=quarantine) header.from=nomadlogic.org; spf=pass (mx1.freebsd.org: domain of pete@nomadlogic.org designates 66.165.241.226 as permitted sender) smtp.mailfrom=pete@nomadlogic.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nomadlogic.org; s=04242021; t=1706574480; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KLb+a4sYa9CBEu0YkVAt2TKbNVf1/qk+JEqKiBqNjxU=; b=M1keZwxtKZ7hVpmX1doQxlJslvUTveZWmh7MjI+o9HOqBFoaZpwVyaGysoTR9SRncFCr9k wEvz6EYJDIjwSDm6p2aVRGGWQ/JC4pBUObR61bWqW/4HsRW5XLiaEhxseY4bJMTARLSBXm prd5g/YVubO15kL766d2sXxUDX+JAp0= Received: from [192.168.1.160] ( [47.150.83.63]) by mail.nomadlogic.org (OpenSMTPD) with ESMTPSA id 2d30b585 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Tue, 30 Jan 2024 00:27:59 +0000 (UTC) Message-ID: <8d266dd2-5ebc-40db-995f-4a4bfb9707eb@nomadlogic.org> Date: Mon, 29 Jan 2024 16:28:02 -0800 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Contributing the build of a KVM image Content-Language: en-US To: virtualization@freebsd.org References: From: Pete Wright In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.986]; DMARC_POLICY_ALLOW(-0.50)[nomadlogic.org,quarantine]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[nomadlogic.org:s=04242021]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:29802, ipnet:66.165.240.0/22, country:US]; RCVD_COUNT_ONE(0.00)[1]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_ALL(0.00)[]; MLMMJ_DEST(0.00)[virtualization@freebsd.org]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[virtualization@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[nomadlogic.org:+] X-Rspamd-Queue-Id: 4TP5cH4Nggz40FH On 1/29/24 14:41, Jo Durchholz wrote: > Hi all, > > where do I turn to if I want to contribute to the FreeBSD > build/release process so that a KVM image will be built? > > This is not actually about virtualization per se but about the > build&release process, so I guess I'd be wrong here. I'd suggest filing a bug first, along with a patch if you have one: https://bugs.freebsd.org/bugzilla/ depending on the complexity of changes developers may ask for you to start a review in phabricator: https://wiki.freebsd.org/Phabricator if you'd like to get more eyes on this it may be worth sending a message to freebsd-hackers@ or freebsd-current@ -pete -- Pete Wright pete@nomadlogic.org From nobody Tue Jan 30 14:34:27 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TPSNh0H6xz58xQq for ; Tue, 30 Jan 2024 14:34:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TPSNg5jTFz4nQS for ; Tue, 30 Jan 2024 14:34:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706625267; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=fyYZjby2BT9egTtw5T92QtYr8HesgTSja3QyUHhm3Mo=; b=UTcRhdkUiamEbt7pjs37/XmIJtBsHE6y0Bdg5bXJ+5Dmor4PV8rSTFdVSP+/73uFMZeaAi 0O8FV0XRiG9iShbjLLh1wuAb1GzwdytcYVdXmbx5vJSPy1cqF4xYUtjTTWlT3WjffMhusd q1slEyc4zwPyTGYyvFIDSkIAwe5lApJp1nOMs69hLPcD1w3Z37t5yVTIJWhcyfFE1cONos 29o3kaEH3xE9QGJsyRz/so8p4UnH4LUAAV4XS/59J8xN1DlTKGfMm17pnddTc3IdHRAriC UeTTRInsqR7+DyRgOKSqcOvC3Ls80X+EytqWdigDsbGkYIGcAVdpwJagXPLuYA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706625267; a=rsa-sha256; cv=none; b=tCI6VANXcHoYRhBA1Nz1OetdM4fOnoS4by5fuj728AoMsPWsiREwrCAObUVEq59c3cu2Ky QXoVvtWEei9UzF5FlPSmeg8pm+Cf/8/nJhXHyjfbbiJJT0l2UwsiBWU04h0VZftDojoMwb HYE1C6sSks2f9TxYrsHF6Qbi6aiTo7syPb3Mpjwe774aZgGqCHThq9CfQwcxxNKYFYmhG4 FQGH1mAg4I59FXdTuUiFoTDxlkIkmYx1sIlbRJ0DHQ1lS16SPCZPpVfP7d4lNQSS/bKH3m l33a7gjrVHwIkimiUMGmXP7ge7tFSevmUME6IYJD38REFtBHHHL3MPQjzY3u4A== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TPSNg4l1fzfls for ; Tue, 30 Jan 2024 14:34:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 40UEYRPE009893 for ; Tue, 30 Jan 2024 14:34:27 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 40UEYR58009892 for virtualization@FreeBSD.org; Tue, 30 Jan 2024 14:34:27 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 276734] bhyve gets stuck during VM boot Date: Tue, 30 Jan 2024 14:34:27 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 14.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: antranigv@freebsd.am X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276734 Bug ID: 276734 Summary: bhyve gets stuck during VM boot Product: Base System Version: 14.0-RELEASE Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: bhyve Assignee: virtualization@FreeBSD.org Reporter: antranigv@freebsd.am Every once in a while (around 1/10) bhyve doesn't boot the VM. It's not cle= ar why. Here's all the information I can provide, and we're open to rebooting our V= M to replicate when needed in order to send more information. # pgrep -lf bhyve 67937 bhyve -c 200,sockets=3D2,cores=3D50,threads=3D2 -m 1536G -AHP -p 0:28= -p 100:156 -p 1:29 -p 101:157 -p 2:30 -p 102:158 -p 3:31 -p 103:159 -p 4:32 -p 104:160 -p 5:33 -p 105:161 -p 6:34 -p 106:162 -p 7:35 -p 107:163 -p 8:36 -p 108:164 -p 9:37 -p 109:165 -p 10:38 -p 110:166 -p 11:39 -p 111:167 -p 12:40= -p 112:168 -p 13:41 -p 113:169 -p 14:42 -p 114:170 -p 15:43 -p 115:171 -p 16:4= 4 -p 116:172 -p 17:45 -p 117:173 -p 18:46 -p 118:174 -p 19:47 -p 119:175 -p 20:4= 8 -p 120:176 -p 21:49 -p 121:177 -p 22:50 -p 122:178 -p 23:51 -p 123:179 -p 24:5= 2 -p 124:180 -p 25:53 -p 125:181 -p 26:54 -p 126:182 -p 27:55 -p 127:183 -p 28:5= 6 -p 128:184 -p 29:57 -p 129:185 -p 30:58 -p 130:186 -p 31:59 -p 131:187 -p 32:6= 0 -p 132:188 -p 33:61 -p 133:189 -p 34:62 -p 134:190 -p 35:63 -p 135:191 -p 36:6= 4 -p 136:192 -p 37:65 -p 137:193 -p 38:66 -p 138:194 -p 39:67 -p 139:195 -p 40:6= 8 -p 140:196 -p 41:69 -p 141:197 -p 42:70 -p 142:198 -p 43:71 -p 143:199 -p 44:7= 2 -p 144:200 -p 45:73 -p 145:201 -p 46:74 -p 146:202 -p 47:75 -p 147:203 -p 48:7= 6 -p 148:204 -p 49:77 -p 149:205 -p 50:78 -p 150:206 -p 51:79 -p 151:207 -p 52:8= 0 -p 152:208 -p 53:81 -p 153:209 -p 54:82 -p 154:210 -p 55:83 -p 155:211 -p 56:8= 4 -p 156:212 -p 57:85 -p 157:213 -p 58:86 -p 158:214 -p 59:87 -p 159:215 -p 60:8= 8 -p 160:216 -p 61:89 -p 161:217 -p 62:90 -p 162:218 -p 63:91 -p 163:219 -p 64:9= 2 -p 164:220 -p 65:93 -p 165:221 -p 66:94 -p 166:222 -p 67:95 -p 167:223 -p 68:9= 6 -p 168:224 -p 69:97 -p 169:225 -p 70:98 -p 170:226 -p 71:99 -p 171:227 -p 72:1= 00 -p 172:228 -p 73:101 -p 173:229 -p 74:102 -p 174:230 -p 75:103 -p 175:231 -p 76:104 -p 176:232 -p 77:105 -p 177:233 -p 78:106 -p 178:234 -p 79:107 -p 179:235 -p 80:108 -p 180:236 -p 81:109 -p 181:237 -p 82:110 -p 182:238 -p 83:111 -p 183:239 -p 84:112 -p 184:240 -p 85:113 -p 185:241 -p 86:114 -p 186:242 -p 87:115 -p 187:243 -p 88:116 -p 188:244 -p 89:117 -p 189:245 -p 90:118 -p 190:246 -p 91:119 -p 191:247 -p 92:120 -p 192:248 -p 93:121 -p 193:249 -p 94:122 -p 194:250 -p 95:123 -p 195:251 -p 96:124 -p 196:252 -p 97:125 -p 197:253 -p 98:126 -p 198:254 -p 99:127 -p 199:255 -U 2893e12c-637d-11ee-af94-7cc255269cb8 -u -S -s 0,amd_hostbridge -s 31,lpc -s 4:0,virtio-blk,/usr/local/vm/comp0/disk0.img -s 4:1,virtio-9p,proj=3D/mnt/p= roj -s 4:2,virtio-9p,user=3D/mnt/user -s 4:3,nvme,/dev/zvol/zscratch/scratch -s 5:0,virtio-net,tap0,mac=3D58:9c:fc:0e:4b:00 -s 6:0,passthru,68/0/1 -l com1,/dev/nmdm-comp0.1A comp0 69027 bhyve: system.grp 69850 bhyve: system.pwd 68574 bhyve: system.pwd 70386 bhyve: system.grp # procstat -k 67937 PID TID COMM TDNAME KSTACK=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 67937 117360 bhyve - mi_switch _sleep vm_wait_doms vm_wait_domain vm_page_alloc_noobj_domain uma_small_alloc keg_alloc_slab zone_import zone_alloc_item malloc amdvi_update_mapping iommu_create_mapping vm_iommu_modify vm_assign_pptdev vmmdev_ioctl devfs_io= ctl vn_ioctl devfs_ioctl_f=20 67937 117420 bhyve blk-4:0-0 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117421 bhyve blk-4:0-1 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117422 bhyve blk-4:0-2 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117423 bhyve blk-4:0-3 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117424 bhyve blk-4:0-4 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117425 bhyve blk-4:0-5 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117426 bhyve blk-4:0-6 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117427 bhyve blk-4:0-7 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117430 bhyve 9p-responder mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117431 bhyve 9p-worker:0 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117432 bhyve 9p-worker:1 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117433 bhyve 9p-worker:2 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117434 bhyve 9p-worker:3 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117435 bhyve 9p-worker:4 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117436 bhyve 9p-worker:5 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117437 bhyve 9p-worker:6 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117438 bhyve 9p-worker:7 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117442 bhyve 9p-responder mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117443 bhyve 9p-worker:0 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117444 bhyve 9p-worker:1 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117445 bhyve 9p-worker:2 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117446 bhyve 9p-worker:3 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117447 bhyve 9p-worker:4 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117448 bhyve 9p-worker:5 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117449 bhyve 9p-worker:6 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117450 bhyve 9p-worker:7 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117451 bhyve blk-4:3-0 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117452 bhyve blk-4:3-1 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117453 bhyve blk-4:3-2 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117454 bhyve blk-4:3-3 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117455 bhyve blk-4:3-4 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117456 bhyve blk-4:3-5 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117457 bhyve blk-4:3-6 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117458 bhyve blk-4:3-7 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117459 bhyve nvme-aen-4:3 mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 67937 117460 bhyve vtnet-5:0 tx mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private sys__umtx_op amd64_syscall fast_syscall_common= =20 # procstat fds -C 67937 PID COMM FD T FLAGS CAPABILITIES=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 PRO NAME=20=20=20=20=20=20=20=20 67937 bhyve text v r------- -=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 - /usr/sbin/bhyve=20=20=20 67937 bhyve ctty v rw------ -=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 - /dev/pts/4=20=20=20=20=20=20=20=20 67937 bhyve cwd v r------- -=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 - /=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20 67937 bhyve root v r------- -=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 - /=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20 67937 bhyve 0 v r------- rd,wr,se,mm,cr,fe,fy,ft,cd,cf,cm,cn,fc,fl,fp,fk,fs,sf,fu,ls,lt,md,mf,mn,rs,= rt,sl,un,lo,eg,es,ed,el,ag,as,ad,ac,at,bd,co,pn,sn,gs,ln,pf,ss,sh,mg,ms,sg,= sp,sw,ev,ke,kc,io,ty,pg,pw,pk,ba,ca,prd,pwr,mmr,mmw,mmx,mrw,mrx,mwx,mma,re,= sd,scl,ssr - /dev/null=20=20=20=20=20=20=20=20=20 67937 bhyve 1 v -w------ rd,wr,se,mm,cr,fe,fy,ft,cd,cf,cm,cn,fc,fl,fp,fk,fs,sf,fu,ls,lt,md,mf,mn,rs,= rt,sl,un,lo,eg,es,ed,el,ag,as,ad,ac,at,bd,co,pn,sn,gs,ln,pf,ss,sh,mg,ms,sg,= sp,sw,ev,ke,kc,io,ty,pg,pw,pk,ba,ca,prd,pwr,mmr,mmw,mmx,mrw,mrx,mwx,mma,re,= sd,scl,ssr - /dev/null=20=20=20=20=20=20=20=20=20 67937 bhyve 2 v -w------ rd,wr,se,mm,cr,fe,fy,ft,cd,cf,cm,cn,fc,fl,fp,fk,fs,sf,fu,ls,lt,md,mf,mn,rs,= rt,sl,un,lo,eg,es,ed,el,ag,as,ad,ac,at,bd,co,pn,sn,gs,ln,pf,ss,sh,mg,ms,sg,= sp,sw,ev,ke,kc,io,ty,pg,pw,pk,ba,ca,prd,pwr,mmr,mmw,mmx,mrw,mrx,mwx,mma,re,= sd,scl,ssr - /usr/local/vm/comp0/bhyve.log 67937 bhyve 3 v rw------=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 rd,wr,se,mm,io,prd,pwr,mmr,mmw,mrw,re,sd - /dev/vmm/comp0=20=20=20=20 67937 bhyve 4 k rw------=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 ke,kc - -=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20 67937 bhyve 5 v rw------=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20 rd,wr,se,fy,fp,fs,ev,io,prd,pwr,re,sd - /usr/local/vm/comp0/disk0.img 67937 bhyve 6 v r-------=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20 rd,wr,se,cr,fy,ft,cm,cn,fp,fs,sf,fu,ls,lt,md,mn,rs,rt,sl,un,lo,eg,es,ed,el,= ag,as,ad,ac,prd,pwr,re,sd - /mnt/proj=20=20=20=20=20=20=20=20=20 67937 bhyve 7 v r-------=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20 rd,wr,se,cr,fy,ft,cm,cn,fp,fs,sf,fu,ls,lt,md,mn,rs,rt,sl,un,lo,eg,es,ed,el,= ag,as,ad,ac,prd,pwr,re,sd - /mnt/user=20=20=20=20=20=20=20=20=20 67937 bhyve 8 s rw---n-- rd,wr,se,mm,cr,fe,fy,ft,cd,cf,cm,cn,fc,fl,fp,fk,fs,sf,fu,ls,lt,md,mf,mn,rs,= rt,sl,un,lo,eg,es,ed,el,ag,as,ad,ac,at,bd,co,pn,sn,gs,ln,pf,ss,sh,mg,ms,sg,= sp,sw,ev,ke,kc,io,ty,pg,pw,pk,ba,ca,prd,pwr,mmr,mmw,mmx,mrw,mrx,mwx,mma,re,= sd,scl,ssr UDS 0 0 - 67937 bhyve 9 v rw------=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20 rd,wr,se,fy,fp,fs,ev,io,prd,pwr,re,sd - /dev/zvol/zscratch/scratch 67937 bhyve 10 s rw---n-- rd,wr,se,mm,cr,fe,fy,ft,cd,cf,cm,cn,fc,fl,fp,fk,fs,sf,fu,ls,lt,md,mf,mn,rs,= rt,sl,un,lo,eg,es,ed,el,ag,as,ad,ac,at,bd,co,pn,sn,gs,ln,pf,ss,sh,mg,ms,sg,= sp,sw,ev,ke,kc,io,ty,pg,pw,pk,ba,ca,prd,pwr,mmr,mmw,mmx,mrw,mrx,mwx,mma,re,= sd,scl,ssr UDS 0 0 - 67937 bhyve 11 s rw---n-- rd,wr,se,mm,cr,fe,fy,ft,cd,cf,cm,cn,fc,fl,fp,fk,fs,sf,fu,ls,lt,md,mf,mn,rs,= rt,sl,un,lo,eg,es,ed,el,ag,as,ad,ac,at,bd,co,pn,sn,gs,ln,pf,ss,sh,mg,ms,sg,= sp,sw,ev,ke,kc,io,ty,pg,pw,pk,ba,ca,prd,pwr,mmr,mmw,mmx,mrw,mrx,mwx,mma,re,= sd,scl,ssr UDS 0 0 - 67937 bhyve 12 v rw---n--=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 rd,wr,ev,re,sd - /dev/tap0=20=20=20=20=20=20=20=20=20 67937 bhyve 13 s rw---n-- rd,wr,se,mm,cr,fe,fy,ft,cd,cf,cm,cn,fc,fl,fp,fk,fs,sf,fu,ls,lt,md,mf,mn,rs,= rt,sl,un,lo,eg,es,ed,el,ag,as,ad,ac,at,bd,co,pn,sn,gs,ln,pf,ss,sh,mg,ms,sg,= sp,sw,ev,ke,kc,io,ty,pg,pw,pk,ba,ca,prd,pwr,mmr,mmw,mmx,mrw,mrx,mwx,mma,re,= sd,scl,ssr UDS 0 0 - 67937 bhyve 14 v rw------=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20 rd,wr,io,re,sd - /dev/pci=20=20=20=20=20=20=20=20=20= =20 # ps aux | grep bhyve root 68574 0.0 0.0 1610652128 399900 - Is 13:16 0:00.00 bhyve: system.pwd (bhyve) root 69027 0.0 0.0 1610652128 399900 - Is 13:16 0:00.00 bhyve: system.grp (bhyve) root 69850 0.0 0.0 1610653424 400128 - Is 13:16 0:00.00 bhyve: system.pwd (bhyve) root 70386 0.0 0.0 1610653424 400124 - Is 13:16 0:00.00 bhyve: system.grp (bhyve) root 67937 0.0 0.0 1610693008 401856 4 D 13:16 0:16.77 bhyve -c 200,sockets=3D2,cores=3D50,threads=3D2 -m 1536G -AHP -p 0:28 -p 10= 0:156 -p 1:29 -p 101:157 -p 2:30 -p 102:158 -p 3:31 -p 103:159 -p 4:32 -p=20 Any tips to debug would be appreciated. Cheers! --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Tue Jan 30 14:38:28 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TPSTK0JcXz58xSl for ; Tue, 30 Jan 2024 14:38:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TPSTJ49M9z4nqk for ; Tue, 30 Jan 2024 14:38:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706625508; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1E52m3L11YzKJtV+2BipPSG/NvAxwFd00+4mcf4d0ls=; b=LV6+VUxumOryi7JKPJDJ7UF3sLK92igmpfC4FIbxMqh8q2NKXPBnloFaQUW9oXV4LRsZk7 QvRT2yKayHZbjLtfd4MJh8FIQx1xXlTfYCZOVvISl8xJ9yLaxAs46bhZRTm/dimjYUy1jO M0j/JaIgXyZDmYysbxB8FC/avQEMdG2B04kt9Z7obvzuWLeLEIiAz+3VLhq09QmAbp3naI RnBJSe8/1CKwmopaL68wkxUPDw8c+bqjyvs54clwJ0v90DIr6GHNGNuJ5b0vgbJcim64AZ B6fTEh0wwJAdEWHPTclm9oE26/Mg0bu8V25IQ4IbsIctyOccbDP92e5ofbUMXA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706625508; a=rsa-sha256; cv=none; b=wNdSwivl4cEo875212m+086/N4LkoGhnOJ1FxfRrZPhC6wgjYlqwJ5FtmdFu7S6ZX1S+zE INdndwgqg/mW+VSXwX2qIus4UWE6tZkTwXbniic89Ig68pEaJ/jvlB6WOROAorrwU/CA46 hlTIcNL7DfieOKCIo7apyilEkXaKpzS9exgFNdKf0caIZrAjJ55SedAprzEK2k8KYd6f5s zKtFcXNecTuCaitTctLf/DMkR7ajIMiJF5YW6pf3uYpzE5BCfBDK3FOLr/G60hWt9ypWlR bqoYmsbLxM+ihNsvwYhNo9p1nh0uGXiNvqVopBXrMmx1yFIZ0oXAmGFBqHJBLw== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TPSTJ3DKxzfyV for ; Tue, 30 Jan 2024 14:38:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 40UEcStP023431 for ; Tue, 30 Jan 2024 14:38:28 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 40UEcSJS023430 for virtualization@FreeBSD.org; Tue, 30 Jan 2024 14:38:28 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 276734] bhyve gets stuck during VM boot Date: Tue, 30 Jan 2024 14:38:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 14.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: antranigv@freebsd.am X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276734 --- Comment #1 from Antranig Vartanian --- Just noticed. I can't even kill the VM instance nor I can do bhyvectl destr= oy. # bhyvectl --destroy --vm=3Dcomp0 load: 0.56 cmd: bhyvectl 48937 [devdrn] 5.61r 0.00u 0.00s 0% 2184k It has been stuck like this for a while and memory usage hasn't change. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Tue Jan 30 15:08:49 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TPT8K4sSqz5918F for ; Tue, 30 Jan 2024 15:08:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TPT8K2sJyz4tpm for ; Tue, 30 Jan 2024 15:08:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706627329; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0jfDr0yiISpOjU1zi4Ye6QbdJmQuw0hxiRj0/XeqN4A=; b=l/HJ9oxrzeX6lwSMVAoBrA8ziiYwC0jx12rmkx2xX/al5fn1CHF0MR6B6z8pHmSjIB7ghW 7XnGrCIfnasbshDIDKghZ1xIyHGKxaJ4nhNyCT7dnQ/gXb36rk/jV9Xcsq3vL2nzuIpLoE pFww551v8O61IUg5mZhBVBZkxDYkzGhoY6cyYuejVprWLEdQerX3tjIDP0YnBYHo5Pac/+ 6FsuW8MwQltW+hyCrzl9eVmEnmiRwf2b+58Edfgn2Du4maxHAH8Jh26+jNsr3QMg7zMjys oJ2RX28BNPpZ57NmRxkBaEdUYY3+U+0dmCnkLTNAepnCoINW/Ro8KHOZpQ3ykg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706627329; a=rsa-sha256; cv=none; b=Rc6LgBy/W8VZY54j9DcBj8O+SHVJvHgCUgwz8rx8mArK/9Xt25WIfqxy7Uvgbsnep7lBEJ fFUDiAfqp1xfFrw2rTZlbmTG021FGNei1IbzZN3RDm48xGuAFMYPU7DTuyoA9z5swWWzJW U7druLMNRb9mo2iblSf9iT97oWRlVbflHwE1qKg8da8oNGqbz8YKRTWQGb4+1vPuIT32ED GXn56Z9LoSpvCUcWoOByswS/jqzv8DzPfKz9n8nsluvUQg/mOFTb8K9rrvJKd//qql4wxW qcrT4V+OqFb60KoJhhuMOweGXZHWpcnOKnPKndgENppR87nB4sAa6ORZ0X/eqg== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TPT8K1pg2zgjD for ; Tue, 30 Jan 2024 15:08:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 40UF8nGU077319 for ; Tue, 30 Jan 2024 15:08:49 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 40UF8nDF077310 for virtualization@FreeBSD.org; Tue, 30 Jan 2024 15:08:49 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 276720] AMIs fail to boot in AWS EC2 Date: Tue, 30 Jan 2024 15:08:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: mgrooms@shrew.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276720 --- Comment #2 from mgrooms@shrew.net --- Hi Colin. Thanks for the response. I'm running a bunch of FreeBSD in AWS, so thanks for all your hard work. I left some of these new instances up for several minutes knowing that the initial boot ( growfs ) can take some time to complete. After opening this = bug report, I left one up and running overnight and the boot completed at some point. Here is the AMI used ... ami-0059170a80e36d30f When I look at the boot log, I see this in the output right after the conso= le messages ... Jan 29 20:56:27 freebsd freebsd-update[870]: src component not installed, skipped Jan 29 20:56:27 freebsd freebsd-update[870]: Looking up aws.update.FreeBSD.= org mirrors... 1 mirrors found. Jan 29 21:00:57 freebsd freebsd-update[870]: Fetching public key from dualstack.aws.update.freebsd.org... failed. Jan 29 21:00:57 freebsd freebsd-update[870]: No mirrors remaining, giving u= p. Jan 29 21:00:57 freebsd freebsd-update[870]:=20 Jan 29 21:00:57 freebsd freebsd-update[870]: This may be because upgrading = from this platform (amd64) Jan 29 21:00:57 freebsd freebsd-update[870]: or release (13.2-RELEASE) is unsupported by freebsd-update. Only Jan 29 21:00:57 freebsd freebsd-update[870]: platforms with Tier 1 support = can be upgraded by freebsd-update. Jan 29 21:00:57 freebsd freebsd-update[870]: See https://www.freebsd.org/platforms/ for more info. Jan 29 21:00:57 freebsd freebsd-update[870]:=20 Jan 29 21:00:57 freebsd freebsd-update[870]: If unsupported, FreeBSD must be upgraded by source. I think this caused a 4.5 minute delay because I booted the AMI with a secu= rity group that didn't allow access to aws.update.FreeBSD.org. That doesn't seem ideal :/ In any case, I'm pretty sure that solves the mystery. Sorry for the noise. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Tue Jan 30 15:43:30 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TPTwL57vrz593Tx for ; Tue, 30 Jan 2024 15:43:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TPTwL1zt1z40jT for ; Tue, 30 Jan 2024 15:43:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706629410; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vq8njNL1t7jAWYstJbpplV89Vwue7QU7NsAWRUlkNpE=; b=ZqJPRg0C4PvYImtc4GuE3Yq0cMWpPrGp1XNhWE5d9G3JqmkJA8U7VPDxnFhBO4O9qWfcOS XKqNKig1WGTEjJsu6UOsgZCxGMBM8ZqTrqciBa2efr1EZKwukLbX6icVgBEbXaHiEZhIgd ZSftwyiIUxEx2E+y75Ii+LkgSbw5e4SeO4/y+QdJLG2ESmPDf05QKPxmSM8vbdxoRflfDz lSlf6uJwfiZmDPccB8lDaviZdAt1vKuzWzpJzdCGu3mGd+7Of28sHy/0laAzAIgZfWv3hG q3Zk08PTSgjB7z0WAi4NmW4j1TYQyTAbklf51elr9r4T+SyO1yl8wL3IY33m5g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706629410; a=rsa-sha256; cv=none; b=IYYBKIyZsW3F2Sre5k8jdcW8MsidxQZHICZXQIoOhAOjOLel/RcdqY98iDSmiH9LO8YbZg Ss8MjMrUy1I0HDaG8mIitvpRz7yo2EmSQgAzBxbhW3FmDdkraFLsjeV8HshEToUQ6j8lt8 XbRcOJO94fuoaPyJaviUlH4j1FwPY1fT/Yvhdw5eOhj3RkJt/zLzoY5hZbrCE4/6gNPAIP EWJ8nhUOIbKRJ41zRmoNIOOzl7yRJdDuZzEGFXKSjsKjHXG2KwTU1XLvJXGDYtxQwtSVuN R5pAPYAnQ2up7WPvxztPevb5Fe11wwfo/io/eM0tYgiXai7mvXkgaZ7G9OeNYQ== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TPTwL0SqLzhgg for ; Tue, 30 Jan 2024 15:43:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 40UFhU9D058767 for ; Tue, 30 Jan 2024 15:43:30 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 40UFhUHG058766 for virtualization@FreeBSD.org; Tue, 30 Jan 2024 15:43:30 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 276720] AMIs fail to boot in AWS EC2 Date: Tue, 30 Jan 2024 15:43:30 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: michaelo@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276720 Michael Osipov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |michaelo@FreeBSD.org --- Comment #3 from Michael Osipov --- (In reply to mgrooms from comment #2) It still shouldn't block for 4.5 min? No? --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Tue Jan 30 16:56:37 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TPWXk2mb4z599kF for ; Tue, 30 Jan 2024 16:56:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TPWXk1WlCz49wp for ; Tue, 30 Jan 2024 16:56:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706633798; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lniSN7Fc3lWvm0sOTV/nvpBU8LAGRIEUM+5HPQgjwAU=; b=Ypml+s7Na4bpGz98vXLMnUV2Nf6O6A4lYh2vCFisRDbCsRHQ8r++temg7/VExIoX7ITTOG rTPNxUylW34CdZvS7gyqbt9e269wrXzgz3V7QSBrMZpPx6ig4GidZ2vLzw0X7z1fKnf9U5 8bMUcV76HZcZdvXflw4bjyVdvWqI12LTRvLEmx5/vKm3lk+NTy4NQ3d+wMmPyXnLZ00HqH FDelwhgPcYN+NHyUCfIk10+LQR1jg0ga/BHoBUdUVU58gWEng2G2lGwjKwyyf33MPaShHH qfl1yYL4SuHp6adU/mOb5KyMKfAABf3spVusobLElc0CvZtYso66fVL+h22A/w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706633798; a=rsa-sha256; cv=none; b=YpHRkkWka54ShXeM9x9N1R6+RwlmSf61ORCNRDrHJi5cLVan/LENBxFFPMxrEYaKLajHjx m/aJWwnym9Iuwp8cPAEg5JzYOLbbgwz5kYPOGU6ZxlC6mEQ4geLTf5Ld++qHd/UuSxuOo9 GJti7gYBz7GcFS/26MN40tcueEDK9y0twMVDBg5uH6YjaWu9COrrtl9rzi47I50Gc58Ipv Dma2GFK32xcwrZhnYF4twGfiSzG0D80+5AzwvoMkRHxUkmTRti3tHlhQJuM4CluwUQrkum VY/Jtb1DA82Ds1LzuvpxJ1nXGO6/S91VwAG3ZgHToeG9p5bcTq56nHwPh0dAuw== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TPWXk0W1wzkSZ for ; Tue, 30 Jan 2024 16:56:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 40UGucSM012975 for ; Tue, 30 Jan 2024 16:56:38 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 40UGucVa012971 for virtualization@FreeBSD.org; Tue, 30 Jan 2024 16:56:38 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 276720] AMIs fail to boot in AWS EC2 Date: Tue, 30 Jan 2024 16:56:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: mgrooms@shrew.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276720 --- Comment #4 from mgrooms@shrew.net --- This is likely just freebsd-update waiting to establish a connection. I bel= ieve that is 60s by default which would add up with multiple attempts. Maybe the= re is a way to shorten the TCP connect timeout in this case to prevent the boot delay? It looks like update uses fetch which appears to honor the following ... -T seconds, --timeout=3Dseconds Set timeout value to seconds. Overrides the environment variables FTP_TIMEOUT for FTP transfers or HTTP_TIMEOUT for HTTP transfers if set. Maybe that means that the init script could export an HTTP_TIMEOUT value < = 60? --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Tue Jan 30 17:02:27 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TPWgR44Ywz59BCt for ; Tue, 30 Jan 2024 17:02:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TPWgR26y2z4D05 for ; Tue, 30 Jan 2024 17:02:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706634147; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GaEbS1VYxj+3eVxyzqhOxfzjypWDSTkehCptOVsEKr0=; b=rpKj6GdmEuYijGx+iwCgyUGAL0D6LIhM6nWg+Zqx2hqqRlEW792xwOOhMe4nmjzr8B+FUx j/ij5BQ9/LecnCoYKS4kZkQmOv2eUU+fwBqKxAfaoqlm8Fc0CBMFc1riQyC15NK6yDt5dU mhIbiRuBVOW8U2upXALZlXzLKwRy2uLV6RUt3Dh1cN6Eny89YWYH1rHX2zUEAda056VTSW ssEP5e3EQpAyNwO7IxalrpLH2GAM12g+ccH69SD/67R6A3+1euFC0Pyleme7q0Ry8oLegd JCzBgA/+mvjZtR2JI1eHO29lWBCBATYj8uIw0TgOAm/jM5CrdAiRVQDsphI4uw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706634147; a=rsa-sha256; cv=none; b=HtX8BQcZlQFYOY2VgpIKBC2JeVjJi5M4oKj2fYNjRppUyMDBENDqIqx28auaSxI5eJdroT B6E+nv0CBl1twZY9GdpoTFUPsv7i4hsZFo8Rhm5RzT3yIUQlif02+nOE344+ff4OjANmdG x28O1lND0iZKuFOGRiZizUyjYwAGWO6OKYDzGsXoTPbGhPfZ9MyTXHYvaeN0QlpMRj5Hbk hiYEJVjZZm5gtS90R2vJSy5aIfaz0HuBRW+f4GPZG1TK5bNdJ6Bmz+kbvb+6SlT81HWTzT yoNzgFNdxhdkfXoqT6du8fT21aeXO2hdDNNHHqvENI+OEy2iHvgNqZcrlWqmgg== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TPWgR1CP3zkFf for ; Tue, 30 Jan 2024 17:02:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 40UH2Rml050566 for ; Tue, 30 Jan 2024 17:02:27 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 40UH2RT0050565 for virtualization@FreeBSD.org; Tue, 30 Jan 2024 17:02:27 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 276720] AMIs fail to boot in AWS EC2 Date: Tue, 30 Jan 2024 17:02:27 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: michaelo@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276720 --- Comment #5 from Michael Osipov --- (In reply to mgrooms from comment #4) Here it is: ./fetch.c:#define TIMEOUT 120 Two minutes per default is insane. I'd say that 30 s are more than enough... --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Tue Jan 30 21:27:12 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TPdXw6ZBTz57d7K for ; Tue, 30 Jan 2024 21:27:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TPdXw2ztSz4tHH for ; Tue, 30 Jan 2024 21:27:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706650032; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TX/mqWmcnUOQeF+KDlJiTvD1t4klEohLJXnnSr9XhHk=; b=fN6nr0pDRGgKSS2/gwf73cRh9tXQqs6S9y0I/YaZuLLiBl3cNZu4AxL+/Y1oHvqI5gQ917 ojfhM9H6w7uvsGV1OgZ3YPtTIJr3D2e7Awn1sBulUr6SRhbd4wxvmYr+pofdmXivHTFgzx caVWDE+snlnNd0CsIMEZbkhqk5HrWTFXhjFpwLplJdr321md7b+8VXCrM2ZbY4p3bSoEEC aGx8QdWkBKhjJqynm3WHZp1RtqQxxGDXk9izUGS0C4EBSlyTixBgTjFzcRRhYynk7SRjC8 daVk2HKwuBYjKXFQC9tVU1lkvV4fWJEi2dVf5pOXWX28tH/PVntipYQqIDhGlg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706650032; a=rsa-sha256; cv=none; b=essLs6LLlbL6lvBAg8CXoqz7pb4U8BZQhurSoz1zqPlbDRdgM7Sr2w88vkOByR+70gnl0/ jlxNUhED5JiaHICOzC06SXg8kx0zWp0+Y5axyZxT5DCcGwEC+UfKvSs0DurouR9xADmWUk GSdGwcI+BgD70nOCPRm0XWbYOAbjZ9h9fuSCsQBSUSHgZXNX9KoAo3WMo/tHa28D4e+kja 4Ahdak52Wcf43gJL0X4vfBahQxIDjndrobjbku9gEmZzZ+CUduejwe1BPESTiQuN4lGYt2 mr3FNp8ASMJJMahjCAhgWnVS+rRKvUoN+8UKx2a0W1HjkmmWTUB7+m8yTRiWyg== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TPdXw24VMzs6s for ; Tue, 30 Jan 2024 21:27:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 40ULRCfY080787 for ; Tue, 30 Jan 2024 21:27:12 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 40ULRCQ7080785 for virtualization@FreeBSD.org; Tue, 30 Jan 2024 21:27:12 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 276720] AMIs fail to boot in AWS EC2 Date: Tue, 30 Jan 2024 21:27:12 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276720 --- Comment #6 from commit-hook@FreeBSD.org --- A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=3D39126a2d9768e46e0fe8795c815721d= 122adadad commit 39126a2d9768e46e0fe8795c815721d122adadad Author: Colin Percival AuthorDate: 2024-01-30 21:20:11 +0000 Commit: Colin Percival CommitDate: 2024-01-30 21:26:23 +0000 sysutils/firstboot-freebsd-update: HTTP_TIMEOUT=3D5 If a system with firstboot_freebsd_update_enable=3D"YES" boots without access to the FreeBSD Update mirrors (e.g. an EC2 instance which has an EC2 security group settings which block outbound HTTP) the boot will hang until it times out. The default timeout of 120 seconds is suboptimal. Run freebsd-update with a timeout of 5 seconds, and bump the package version to 1.4 to reflect this change. Reported by: mgrooms@shrew.net PR: 276720 Sponsored by: https://www.patreon.com/cperciva sysutils/firstboot-freebsd-update/Makefile | 2 +- sysutils/firstboot-freebsd-update/files/firstboot_freebsd_update.in | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Tue Jan 30 21:28:10 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TPdZ306VGz57d07 for ; Tue, 30 Jan 2024 21:28:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TPdZ24LfYz4tKB for ; Tue, 30 Jan 2024 21:28:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706650090; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=q2b0+t/MFHcOoMtD8lxu1N1TjW39MAYlrJ5yIsG2zao=; b=qC7luQbaJDRLca3yXpATjeN5GoJpxnHqkiF5k1AzyT8FzPbu8y0nYMgHFda9W+ANuLIjpR pyuegrHp+kLMGyZXqc0g22OrbBjnpShlGVlsjKbYhlUBi4ApzNrIQeTN/hsnRJvHwst5G7 DeiK0wKYEYzYHlMieJQd4fFM4dUzS6zefJdyUWGiBTAen3N1Ghitxeze75ApAgniduRf7J xTAqENdPCNnnJ1J4Z73SSlp+HBfSDH24l00j8Vtuv5uypEjXsPPBzjl4fobDtzCzf1gYwj Pm/pcb9zcAs/kwCM9SBA0kYDewldI93483CBmLV8yhepRmrNEqEh5wMkQ6yesw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706650090; a=rsa-sha256; cv=none; b=g3WTPhqDsSLba8FeAF0wEXWw6wXkRfeGTEgXULyd6MALHzVqd+EHAVr5igpSx1Yh7CtCLr SVAS6HeRInbxzPeb72jMVVganX4Tt1uZhdWdcFZDbh8iaf9PuFx04Oh/FFy/4RQleLA/8D gMMhV53pvuJQKAAq9YpMS78b5sVmXMDgtf64KMRhwIdoKQ3V9+3jUB+I8V7NQMfJ+Bufhm 9Nmkhb6SB99dAkapAhNpOmv1E+RE1P7Jxnv6UB4QwxDnafh5E6WvqmmMdSQ3AtbpyG/0U7 z3CtIV6NwJjfyHtXo4LWrrLQz6G+yKgVslxA4gloMz51Iml3+S57XFYNpB5WXA== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TPdZ23QpLzrCl for ; Tue, 30 Jan 2024 21:28:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 40ULSAAG083823 for ; Tue, 30 Jan 2024 21:28:10 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 40ULSASs083822 for virtualization@FreeBSD.org; Tue, 30 Jan 2024 21:28:10 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 276720] AMIs fail to boot in AWS EC2 Date: Tue, 30 Jan 2024 21:28:10 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: cperciva@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276720 Colin Percival changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|New |Closed --- Comment #7 from Colin Percival --- Should be much faster now. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Tue Jan 30 21:29:45 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TPdbs2JDKz57dBD for ; Tue, 30 Jan 2024 21:29:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TPdbs0pLzz4tnC for ; Tue, 30 Jan 2024 21:29:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706650185; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cxi9fnC++C9xFHyDjEoB6WikMByrJcLsUoAczyEHAPs=; b=W76COiKhXp9mYRhxVigURVWlf43FrcXcb6sZrQhNviyU5p4XBw+b1EUH5+LApUJNIFqgUB 5lkUeDAqCV7xfT99iSVSBKhQ8CSlXcgEuVHMq9I65gwXa7OrdlBdeblrEVVAq2+75NBcFM c+V3vqdkwYoNAOJVkxvzLBtL9WYXjHikn6Sn4UhVeQ0rVbdhzjV4HFlYNreu+P1z8mM1/y dz6qjydgk2fEqxiJeyiBVwaM1kTGzX35yqGuAcLx1IOSwhmCBz+Y/Bb7xhgP+gCCrWI4j5 im28djdKJNJlURbs7HqGKvS+8qA7JRgvY8ch7INQZ1u4hbYJVYEB5BCRaMQfGQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706650185; a=rsa-sha256; cv=none; b=xqks4qbomuzpEqbj3i4x2/SirfBlLzSXK/wk5CCEV25YvipVN3rvsnbRcdbPOsgh4iXYny OGb+3MEbBSy/vipY1nIpFY5Qp8aKGrfiwONkcntElAk580WOvYT2uKHC8C6r2uVQzRJVMo 6Qb/sR1zG33VdD7WgT2SjdWB3w3MoSd+WpddncDlaH5VsQ+g9Z2Lt4kYofLncdp4cs6Zzc qV3kH90lGRrHcXCzDaMSQ3rSVmQh6cyVZJhJf5KJ0ROtKnFgEq8cgrlc0bI4H4TC+B3tZc PQ/hJJn22CBs0N7dngwUXvGuj4dLuWpqH0yDy/GcYYNz0P3g2pcrgixtM9fYTg== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TPdbr71HWzs93 for ; Tue, 30 Jan 2024 21:29:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 40ULTiPp088841 for ; Tue, 30 Jan 2024 21:29:44 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 40ULTi0V088840 for virtualization@FreeBSD.org; Tue, 30 Jan 2024 21:29:44 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 276720] AMIs fail to boot in AWS EC2 Date: Tue, 30 Jan 2024 21:29:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: mgrooms@shrew.net X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276720 --- Comment #8 from mgrooms@shrew.net --- Thank you! --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Wed Jan 31 01:53:37 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TPlSd6S4wz586sj for ; Wed, 31 Jan 2024 01:53:53 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (pdx.rh.CN85.dnsmgr.net [65.75.216.6]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TPlSc57w0z4HNr; Wed, 31 Jan 2024 01:53:52 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd-rwg@gndrsh.dnsmgr.net has no SPF policy when checking 65.75.216.6) smtp.mailfrom=freebsd-rwg@gndrsh.dnsmgr.net Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id 40V1rbgd024805; Tue, 30 Jan 2024 17:53:37 -0800 (PST) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id 40V1rbOr024804; Tue, 30 Jan 2024 17:53:37 -0800 (PST) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <202401310153.40V1rbOr024804@gndrsh.dnsmgr.net> Subject: Re: [Bug 276463] UEFI-booted VM limited to 503GB RAM rather than requested 1.5TB In-Reply-To: To: bcran@freebsd.org Date: Tue, 30 Jan 2024 17:53:37 -0800 (PST) CC: virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Spamd-Bar: - X-Spamd-Result: default: False [-1.14 / 15.00]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-0.99)[-0.986]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_SHORT(-0.06)[-0.059]; RCPT_COUNT_TWO(0.00)[2]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; ASN(0.00)[asn:10494, ipnet:65.75.216.0/23, country:US]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_NONE(0.00)[]; MLMMJ_DEST(0.00)[virtualization@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[dnsmgr.net]; RCVD_TLS_LAST(0.00)[]; R_SPF_NA(0.00)[no SPF record] X-Rspamd-Queue-Id: 4TPlSc57w0z4HNr [ Charset UTF-8 unsupported, converting... ] > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=276463 > > Rebecca Cran changed: > > What |Removed |Added > ---------------------------------------------------------------------------- > CC| |bcran@FreeBSD.org > Assignee|virtualization@FreeBSD.org |bcran@FreeBSD.org Please put virtualization@FreeBSD.org into the CC list not sure why it double added you, I thought this was fixed long ago in bugzilla so that we stopped the black hole from the email list when people take a bug. > --- Comment #5 from Rebecca Cran --- > I'll work on it. I think this is something going amuck with the ACPI table building. The number 503G also smells of 40bit PAE. -- Rod Grimes rgrimes@freebsd.org From nobody Wed Jan 31 02:31:53 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TPmJT3Gm8z58BD6 for ; Wed, 31 Jan 2024 02:31:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TPmJT10Qcz4Kw2 for ; Wed, 31 Jan 2024 02:31:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706668313; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m/Em9h8vZyMf8pMjsWc3gefMaVflaHlV3/1JUV7KayE=; b=MyJsjF2jujBGe1h0Et8iHBR9B1zcO0vtuC+St5lTgE7B45GPiMzJURTHO85KfzzNI8Oxva 0CChk+PDrmxZSx1yCn+Pj4jYCZgDcLnCrlIQ7NrE/OB0cdf55SQPHVQxuQ2CQkLUclX8uF WXAFWPPYITIvAj+vbsv97HXHnsrVnR7ZgzSEHsyVHmzofl/RnFdBf1t0viT3JIZsxVy6Mc aonRAB9wslq92JAFn17uXxc8Zx3S7oY1nYvgGLai7Z3ZOuN1pcRK0F5brTJ6SnIIK/U2ON R1+qwZ0EmOiHSaS2PJwciiszfjx9sgLRJnUlZilYRBFI48+gUvLWRwUNTdkUuQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706668313; a=rsa-sha256; cv=none; b=SMLK/Uy7lf2yntcT0AdKxKiNESKqVnCiCt0yZDCCETwLw2m672i690JVnhcTSVqt3D+uFt ilbHl4sHWXk5zO7og++6I/9w17XV76/xQQPcuIisEXiIF6VCSnOpkXQNJxpzkK3v4mEbol dDzuwa/L/BtRYkKibmMhOkJPW1a21gyggpZQcs8Z1NBpxLSY5IyqXqXcuCUY6YQCUgdgxh 3thXIZt2gPpzDbTXCCznMcU7C3x6WXD7h4K19NmCRr7VMwbYFOrrx3eh3uCGS+2gpsOpIa 5PzCyKWrIyywVK6BH1HOAjuoEQvblCVeWmsbqtIU6bcEgxMG7ABhsWtQYx+yDQ== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TPmJT04vZz11My for ; Wed, 31 Jan 2024 02:31:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 40V2VqIq011718 for ; Wed, 31 Jan 2024 02:31:52 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 40V2Vqq6011709 for virtualization@FreeBSD.org; Wed, 31 Jan 2024 02:31:52 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 276463] UEFI-booted VM limited to 503GB RAM rather than requested 1.5TB Date: Wed, 31 Jan 2024 02:31:53 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 13.2-RELEASE X-Bugzilla-Keywords: bhyve X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: bcran@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bcran@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276463 Rebecca Cran changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |virtualization@FreeBSD.org --=20 You are receiving this mail because: You are on the CC list for the bug.= From nobody Thu Feb 1 12:12:43 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TQd8S6yN5z58Wc4 for ; Thu, 1 Feb 2024 12:12:56 +0000 (UTC) (envelope-from aimass@yabarana.com) Received: from mail-il1-x12f.google.com (mail-il1-x12f.google.com [IPv6:2607:f8b0:4864:20::12f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TQd8R6sPvz4Mlq for ; Thu, 1 Feb 2024 12:12:55 +0000 (UTC) (envelope-from aimass@yabarana.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yabarana-com.20230601.gappssmtp.com header.s=20230601 header.b=S+WaPZEJ; dmarc=none; spf=pass (mx1.freebsd.org: domain of aimass@yabarana.com designates 2607:f8b0:4864:20::12f as permitted sender) smtp.mailfrom=aimass@yabarana.com Received: by mail-il1-x12f.google.com with SMTP id e9e14a558f8ab-363844ae819so3200825ab.1 for ; Thu, 01 Feb 2024 04:12:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yabarana-com.20230601.gappssmtp.com; s=20230601; t=1706789574; x=1707394374; darn=freebsd.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=kPDVBFtTMUg+hNwtkEIpaBPOKPLW+zOs11wjoxZtUuE=; b=S+WaPZEJ3Jh7D1lIMckLxBqX6o3bdvCYiIpc6c/JPebH39Htf78ZdIy0NPMm4NUzx1 i8UIDxmh9Zlr6S8qsVE+DKjjoTA2tqRZGikgBgkxVcTXkIqEC4uLNoDkR1vhHzO1bLSz 1kyfZH74B2As3DcAhccaP+6OqaMlO599D4ZP3wmmhyQ3H0obpi0z1pGduNXGfhdeROyH 0hKCeBHbBhYKf4gNUPxoWIAqS9TUldd/IUD2eUORvN18IWfdwVFikLxjiqY1/i1pZLl4 TdwkQmBgci1wUKVbfTehDgrvy2CoSIQ/ftZ+jUUnUu6z2NocESw8pF3ImMqqLcL9byT0 7/IA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706789574; x=1707394374; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=kPDVBFtTMUg+hNwtkEIpaBPOKPLW+zOs11wjoxZtUuE=; b=McoLA1qN1hQYzZ++0ODjT+EG8AQSsFyXg/FAD1w100mOSVO4NGlF6icO/CkyrZAnYP nJrv7losKbR26K6iEm4HSaotK4oZlar9DHtfU6C7fzuMpAuQFWB5jDs4O8MMFCug9V4u 9B9GKp8Ihl3OQtugqP3ke5pfK4ws/K2rQ/+L8BocrmHH80EKj2wQpKWtPRBpmpLopnfh UU/W1dswrjbPKE2PanYNFpfY94sq9EGtMO3U4oPCpMH4btvXC9mGpr/gZbdaKjE5ESiX afcNSX9Lxr/qJy2HP1l35kEwztbu/HPhtyEcscehvotMoBkzIDF8A7nEE44+IKkosDQu 7Tdw== X-Gm-Message-State: AOJu0YyKYLPsbBK4IL+MnJQoIgZIpjFONmh2sUlClkFsyGJOf5HtclWF Udd7TkkZz6vXuJobk24bSmENjLy7OGJL12EQb4Y3VLXgMNVyqM6QwT168ywswP0IKYWsDaTWgQy F6eNvRzcRA2x4uAADyS8k5NFTbkBE+RlnCZOmR6H3Al4Fd4tB3do= X-Google-Smtp-Source: AGHT+IFj0ryA4odA/2Zo0C81dImsTmUHHz5KgtAQoc3O3Mdtugvsh2U82dimQ0Baly/UpUgCDYvxOfFJzSODhQdjW8A= X-Received: by 2002:a05:6e02:963:b0:363:8fc2:90fa with SMTP id q3-20020a056e02096300b003638fc290famr4681790ilt.19.1706789574592; Thu, 01 Feb 2024 04:12:54 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 From: Alejandro Imass Date: Thu, 1 Feb 2024 13:12:43 +0100 Message-ID: Subject: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? To: FreeBSD virtualization Content-Type: multipart/alternative; boundary="0000000000004ce624061050eabf" X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; R_DKIM_ALLOW(-0.20)[yabarana-com.20230601.gappssmtp.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MISSING_XM_UA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[yabarana.com]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::12f:from]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yabarana-com.20230601.gappssmtp.com:+] X-Rspamd-Queue-Id: 4TQd8R6sPvz4Mlq --0000000000004ce624061050eabf Content-Type: text/plain; charset="UTF-8" I made the title provocative for a flame but in good faith, so hear me out ;-) Being a FreeBSD user for many years I perfectly understand why supporting Docker is useless outside of virtualization or just for fun. I only put it in the subject to get your attention. But can the same be said of Kubernetes? In the understanding that k8s relies on some Linux specific things like namespaces and control groups etc. but we have similar utilities that could be used, so not sure why no serious effort to port k8s to BSD so far... Moreover, why is there not a serious alternative to Kubernetes-like, or a FreeBSd native Kubernetes port, at this point in 2024 ? I've read comments about Nomad but does it actually work with Bastille for example ? I mean we have all the building blocks, and we invented "container" so wtf is there no similar orchestration and auto scaling jail technology at our disposal on FBSD so far ? What am I missing? --0000000000004ce624061050eabf Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I made the title provocative for a flame but in good faith= , so hear me out ;-)

Being a FreeBSD user for many years= I perfectly understand why supporting Docker is useless outside of virtual= ization=C2=A0or just for fun. I only put it in the subject to get your atte= ntion.

But can the same be said of Kubernetes? In = the understanding that k8s relies on some Linux specific things like namesp= aces and control groups etc. but we have similar utilities that could be us= ed, so not sure why no serious effort to port k8s to BSD so far...

Moreover, why is there not a serious alternative to Kubern= etes-like, or a FreeBSd native Kubernetes port, at this point in 2024 ? I&#= 39;ve read comments about Nomad but does it actually work with Bastille for= example ? I mean we have all the building blocks, and we invented "co= ntainer" so wtf is there no similar orchestration and auto scaling jai= l technology=C2=A0at our disposal on FBSD so far ?

What am I missing?


--0000000000004ce624061050eabf-- From nobody Thu Feb 1 12:40:11 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TQdmd4mJ4z58Z89 for ; Thu, 1 Feb 2024 12:40:49 +0000 (UTC) (envelope-from odhiambo@gmail.com) Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com [IPv6:2a00:1450:4864:20::234]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TQdmd2sH9z4RCY for ; Thu, 1 Feb 2024 12:40:49 +0000 (UTC) (envelope-from odhiambo@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lj1-x234.google.com with SMTP id 38308e7fff4ca-2d043160cd1so10742051fa.1 for ; Thu, 01 Feb 2024 04:40:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706791247; x=1707396047; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=H8Rsr6BJt4+edya9wfXBk1W75DRcape5LBqG4CdBFok=; b=BFG6ly3MBKLCBBAO63v9KSjqg7X/Zwnm1UMnY5uHSSFk7edN/60fC4bz1NxF9SQFSE 0EBpWuslk7Wm/wbi01iTRNC5xoKNyY4Dpl9ogr9qg5bRFybDJQXndetehXCWvVeDsS9K ad7yCNqxfKTXcGfEqg91UE4v68WSOt2l7sjJfWbDr1odhdh64SpAZXIIaMKigWgqS09x TV8pF2a53As6fRWA+lJ/JNDhpFSzvGDh++XlEnLo9NKUth9sEEMigcJWJAMtsFGyXvH+ g1S4P3rOR7T8YWqF2PBb0PnEtJzi2Gg4qIPMAseI6L0g6VzYX5Ji8WD3VbKqAEFTQbMv o1Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706791247; x=1707396047; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=H8Rsr6BJt4+edya9wfXBk1W75DRcape5LBqG4CdBFok=; b=ROdEjTizubIOSAlf76LACb0T4LRCj4jQEkUddv2hcXWr1Idbw4rwRQQjCQQsuAUI4Q /9udqk+29OUdua6CanXL9r28Nm80Mk9Qg0qFupL5OVxszUOLHPGE6M/5vPKLu3IU/MQ0 d2ZWtAapT1GzZ+IA/De4P8qM8vgA3N304NbY8noZNiev0V9wgxkontavBHvIgFJ3Qfpu yG9A6+q0AxCSRjpVZbi/wZLr6K0V51K7nMHp2QcH6dfZeWGAySY5F1K7KnLF1OLkc7zz FZPS5WAv6S9XAinxOMgSnEzihDrvWeLaaIyoSmS83yy/t6h2R3Byhf+Haf7zMkfR0i3g KJ5Q== X-Gm-Message-State: AOJu0YyHj3YnF4rg4MoFz2aDhEjWhwouQf83IoO+KD15i701C3oIuiSV EnKo0p/ARhKDJ5L/QVXwEZ/QxZKpTNKDLe8BHQ16W8pQ9gOAe/soRpcYw9e/6fUdCjqhG0LThcK GXZoJ4F6bk6ZaUaKnq0WcjnHvhI8= X-Google-Smtp-Source: AGHT+IFwCD92FGrQnvv0AKciHK2bgvW8Vvf+6LYTD9hZI2t0ItLANvxHFlZZCgYMyjsbit9Qa5JcxDAOUY3dNREwvSE= X-Received: by 2002:a2e:8244:0:b0:2cd:9503:f91 with SMTP id j4-20020a2e8244000000b002cd95030f91mr3325919ljh.15.1706791246790; Thu, 01 Feb 2024 04:40:46 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Odhiambo Washington Date: Thu, 1 Feb 2024 15:40:11 +0300 Message-ID: Subject: Re: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? To: Alejandro Imass Cc: FreeBSD virtualization Content-Type: multipart/alternative; boundary="000000000000f888240610514d78" X-Rspamd-Queue-Id: 4TQdmd2sH9z4RCY X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] --000000000000f888240610514d78 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Feb 1, 2024 at 3:13=E2=80=AFPM Alejandro Imass wrote: > I made the title provocative for a flame but in good faith, so hear me ou= t > ;-) > > Being a FreeBSD user for many years I perfectly understand why supporting > Docker is useless outside of virtualization or just for fun. I only put i= t > in the subject to get your attention. > > But can the same be said of Kubernetes? In the understanding that k8s > relies on some Linux specific things like namespaces and control groups > etc. but we have similar utilities that could be used, so not sure why no > serious effort to port k8s to BSD so far... > > Moreover, why is there not a serious alternative to Kubernetes-like, or a > FreeBSd native Kubernetes port, at this point in 2024 ? I've read comment= s > about Nomad but does it actually work with Bastille for example ? I mean = we > have all the building blocks, and we invented "container" so wtf is there > no similar orchestration and auto scaling jail technology at our disposal > on FBSD so far ? > > What am I missing? > You're missing the fact that you could virtualize a Linux host and run all those Docker/k8s inside it and just proxy access to them. For example, I am running https://ushahidi.kictanet.or.ke/views/map inside an Ubuntu VM under bhyve. Apache on the FreeBSD host is doing the proxy. PS: I am lucky to be the 1st one to pour more fuel into your flame :-) --=20 Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' =C2=AF\_(=E3=83=84)_/=C2=AF :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html] --000000000000f888240610514d78 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Feb 1, 2024 at 3:13=E2=80=AFP= M Alejandro Imass <aimass@yabaran= a.com> wrote:
I made the title provocative for a flame but in good = faith, so hear me out ;-)

Being a FreeBSD user for many = years I perfectly understand why supporting Docker is useless outside of vi= rtualization=C2=A0or just for fun. I only put it in the subject to get your= attention.

But can the same be said of Kubernetes= ? In the understanding that k8s relies on some Linux specific things like n= amespaces and control groups etc. but we have similar utilities that could = be used, so not sure why no serious effort to port k8s to BSD so far...

Moreover, why is there not a serious alternative to K= ubernetes-like, or a FreeBSd native Kubernetes port, at this point in 2024 = ? I've read comments about Nomad but does it actually work with Bastill= e for example ? I mean we have all the building blocks, and we invented &qu= ot;container" so wtf is there no similar orchestration and auto scalin= g jail technology=C2=A0at our disposal on FBSD so far ?

What am I missing?

You= 9;re missing the fact that you could virtualize a Linux host and run all th= ose Docker/k8s inside it and just proxy access to them.
For examp= le, I am running=C2=A0https://ushahidi.kictanet.or.ke/views/map inside an Ubuntu VM under bh= yve. Apache on the FreeBSD host is doing the proxy.

PS: I am lucky to be the 1st one to pour more fuel into your flame :-)


--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE=
+254 7 3200 0004/+254 7 2274 3223
=C2=A0In=C2=A0an Intern= et failure case, the #1 suspect is a constant: DNS.
"Oh, the cruft.",=C2=A0egrep -v '^$|= ^.*#'=C2=A0=C2=AF\_(=E3=83=84)_/=C2=AF= =C2=A0:-)
[How to ask smart questions:=C2=A0http://www.catb.org/~esr/faqs/smart-questions.html]
--000000000000f888240610514d78-- From nobody Thu Feb 1 12:43:31 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TQdrV2dFhz58Zdj for ; Thu, 1 Feb 2024 12:44:10 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TQdrT6CWvz4RsC for ; Thu, 1 Feb 2024 12:44:09 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x52c.google.com with SMTP id 4fb4d7f45d1cf-55cca88b6a5so1063115a12.1 for ; Thu, 01 Feb 2024 04:44:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706791448; x=1707396248; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=CmEJu5P9/GEsWjoc3jh/8HXMd1U6aZhuvCdv2fZZ0+g=; b=bI1PT7rXwFT3Eq9elmV4bU/qq+YV1B50u8RAJfmuv/zINu0TMs9nruMyHFvsJePv5m +F9Ovs8/M++8Z7w4GvjwzksA5tDSfbRlGOWb8kzSqNpvMgrxmNQnD4NV6jhhF6CmtC7x HJ8WXlTUO5HI5tk/G06kVuLYE51w5TohweoVutuP2sug6uuJ2AotORJ+HB2nB9qq1Qzo bmnPYuw/0tYgO/Ho2pkFUbMAFyr3sBSZe8dcYFr3UBIVB4dKpCZt8zac/deKSX2q2rvs rvA3jI2SERpNdEQeWKZPAMu4eQycRtLaaUykE30ijOB0bbbTzev51weLsAUbZQi63WCp yh9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706791448; x=1707396248; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=CmEJu5P9/GEsWjoc3jh/8HXMd1U6aZhuvCdv2fZZ0+g=; b=wEoJ+d7W/6Wvi2CR32unRQQ8j3r+1/tBji+gvvLWyFeC9i9OLo/od4ESu6cdYDj90n ZxkRuUNG4Vvp/VjQUASCkYhYCVp+rcwdvoVUGLxFiMqOEgENlwwlyrETt56GP0WEJ8GV ph25Sz8ut+UTEXz72u+GhNpPOtz+/kT26E94urf8Mr6MUpqHIbX2Lie7dFiZygOs0W71 HglOq6P+p0ePK0wEie0lNYjNnbHY5eF50w5IPPxNO5oyJ4irVBrDBQ1QhOD028t74SZb VmCbQCPrpyT73RxsXBpA+53VIDBUEIlK4+ZpPQE6OkeXIvti0SjsCbXLWMYC9vfBpYIZ l4CQ== X-Gm-Message-State: AOJu0YyfMvUggVNYCt42fuM//GLv/CofhVNu8iwMqc3ex6oMX8Hs2mpg OEVu+fnDJzqVE6439Dls5bIQF7+y7zUKwEvh1fli7TS5oSqyTEatW9opLc3/zgCJfTM7aAsM4NS 6L5nLVxq7rlZT5FQgS0Lf7+3qvdt+QLAnXRE= X-Google-Smtp-Source: AGHT+IEAeeVbRxR58Uw6cI/cbffZUu3ATT1M1+HGMG6AamXlohEz9uk4wwZ/CVnB0ZnOkbwltbS+5UADlUnmNhjkqyY= X-Received: by 2002:a17:906:3450:b0:a30:3303:e079 with SMTP id d16-20020a170906345000b00a303303e079mr3105905ejb.63.1706791448285; Thu, 01 Feb 2024 04:44:08 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Mario Marietto Date: Thu, 1 Feb 2024 13:43:31 +0100 Message-ID: Subject: Re: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? To: Odhiambo Washington Cc: Alejandro Imass , FreeBSD virtualization Content-Type: multipart/alternative; boundary="000000000000fb18a206105159d3" X-Rspamd-Queue-Id: 4TQdrT6CWvz4RsC X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] --000000000000fb18a206105159d3 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Would be cool to investigate if the Linuxulator supports Kubernetes.... On Thu, Feb 1, 2024 at 1:40=E2=80=AFPM Odhiambo Washington wrote: > > > On Thu, Feb 1, 2024 at 3:13=E2=80=AFPM Alejandro Imass > wrote: > >> I made the title provocative for a flame but in good faith, so hear me >> out ;-) >> >> Being a FreeBSD user for many years I perfectly understand why supportin= g >> Docker is useless outside of virtualization or just for fun. I only put = it >> in the subject to get your attention. >> >> But can the same be said of Kubernetes? In the understanding that k8s >> relies on some Linux specific things like namespaces and control groups >> etc. but we have similar utilities that could be used, so not sure why n= o >> serious effort to port k8s to BSD so far... >> >> Moreover, why is there not a serious alternative to Kubernetes-like, or = a >> FreeBSd native Kubernetes port, at this point in 2024 ? I've read commen= ts >> about Nomad but does it actually work with Bastille for example ? I mean= we >> have all the building blocks, and we invented "container" so wtf is ther= e >> no similar orchestration and auto scaling jail technology at our disposa= l >> on FBSD so far ? >> >> What am I missing? >> > > You're missing the fact that you could virtualize a Linux host and run al= l > those Docker/k8s inside it and just proxy access to them. > For example, I am running https://ushahidi.kictanet.or.ke/views/map > inside an Ubuntu VM under bhyve. Apache on the FreeBSD host is doing the > proxy. > > PS: I am lucky to be the 1st one to pour more fuel into your flame :-) > > > -- > Best regards, > Odhiambo WASHINGTON, > Nairobi,KE > +254 7 3200 0004/+254 7 2274 3223 > In an Internet failure case, the #1 suspect is a constant: DNS. > "Oh, the cruft.", egrep -v '^$|^.*#' =C2=AF\_(=E3=83=84)_/=C2=AF :-) > [How to ask smart questions: > http://www.catb.org/~esr/faqs/smart-questions.html] > --=20 Mario. --000000000000fb18a206105159d3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Would be cool to investigate if the Linuxulator supports K= ubernetes....



On Thu, Feb 1, 2024 at 3:13=E2=80=AFPM Alejandro I= mass <aimass@ya= barana.com> wrote:
I made the title provocative for a flame but in= good faith, so hear me out ;-)

Being a FreeBSD user for= many years I perfectly understand why supporting Docker is useless outside= of virtualization=C2=A0or just for fun. I only put it in the subject to ge= t your attention.

But can the same be said of Kube= rnetes? In the understanding that k8s relies on some Linux specific things = like namespaces and control groups etc. but we have similar utilities that = could be used, so not sure why no serious effort to port k8s to BSD so far.= ..

Moreover, why is there not a serious alternativ= e to Kubernetes-like, or a FreeBSd native Kubernetes port, at this point in= 2024 ? I've read comments about Nomad but does it actually work with B= astille for example ? I mean we have all the building blocks, and we invent= ed "container" so wtf is there no similar orchestration and auto = scaling jail technology=C2=A0at our disposal on FBSD so far ?
What am I missing?

= You're missing the fact that you could virtualize a Linux host and run = all those Docker/k8s inside it and just proxy access to them.
For= example, I am running=C2=A0https://ushahidi.kictanet.or.ke/views/map insi= de an Ubuntu VM under bhyve. Apache on the FreeBSD host is doing the proxy.=

PS: I am lucky to be the 1st one to pour more fue= l into your flame :-)


--
Best regards,
Odhiambo = WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
=C2=A0In=C2=A0an Internet failure case, the #1 suspect is a constant: DNS= .
"Oh, the cruft.",=C2=A0egrep -v '^$|^.*#'=C2=A0=C2=AF\_(= =E3=83=84)_/=C2=AF=C2=A0:-)<= /div>
[How to ask smart questions:=C2= =A0http://www.catb.org/~esr/f= aqs/smart-questions.html]


--
Mario.
--000000000000fb18a206105159d3-- From nobody Thu Feb 1 14:43:24 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TQhVL0TWsz58mVy for ; Thu, 1 Feb 2024 14:43:38 +0000 (UTC) (envelope-from aimass@yabarana.com) Received: from mail-il1-x136.google.com (mail-il1-x136.google.com [IPv6:2607:f8b0:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TQhVK2r7zz4hnS for ; Thu, 1 Feb 2024 14:43:37 +0000 (UTC) (envelope-from aimass@yabarana.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-il1-x136.google.com with SMTP id e9e14a558f8ab-363844ae819so3818745ab.1 for ; Thu, 01 Feb 2024 06:43:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yabarana-com.20230601.gappssmtp.com; s=20230601; t=1706798616; x=1707403416; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=O22RKWLHpCdeq2Ly8l2q/8ZRxTk0HFqxCQnJJHakMck=; b=nBC6CTWqgdp6Pv3zBbNHZXNdstWVp52Oe+vAe/Y2lHjAwFHcObEoSMla5qIcvO15vK nvK9doqBVM807kwcoQCAwVr8ElnA+fdstnRS6g1HVkURwMUCrlsgJYGRwDb22DCqAdwm JZAwVonYIlmGStuedFbMXQdGpsZkrldxc0cJ6J8TCMvaIfYBJsbRjScr8LM/it0n9CvO bwfD037cHa9pPmQErTDqj290IsbKsTJCubNUMT4bJn0NaLQ9B/K8D+M5NZb0SsRs5r+J PBpTUMXu8uMUPVnEAK3255r5J3R/9CIwZGDgdSNDVeAnpIU1sjupRhGpKeIHT2zbaHrO pSIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706798616; x=1707403416; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=O22RKWLHpCdeq2Ly8l2q/8ZRxTk0HFqxCQnJJHakMck=; b=XV2Cj3uEd3f+KbqdelenuvxiwZEXwp8j2GsuQ89f6VWt1PHEc4ZZ49fY957Bu+kn3I tOqN+wVpYfdYAJOSp9L1sSkssMhNj3OGxUisSHv2Mutwd+rIVBfwUUdycp7ERFmbF5RZ aUECz5jGwLofjZvWSLT7NPAP3GhWr09xVVZs5ge9UPMywiTBo1JYkuy8WneNRS0dLzxm XDYhXYveZyo5+oh69OOPLJps1n65DtRJGjR+Tzo4LIgXtyi1RmaADiMVjgipUCpkg03c uoZKVlJ0h0qoCXEcTz3AU9uYrj6yAvKc/rmPCgizETTdu/Ny9qHEpMSnCv4D+DG2BCBu yMXg== X-Gm-Message-State: AOJu0YwG3f95lXsWKN2jFA9QKuryIeUTl4oOpBkLoQd0i/kU+RrDYgcv mYRJZ0q8T0OJdqXcDiqZPUaAROmy96Cg9GiIzMA6QyTHxqE2+la1X7gdmZz0Q88PFOpXgfL5sfv WLg5naTB9HBswjbQa0P9XPTRj5vdc8OUEQ4FaaEFfgezcLsPOTiY= X-Google-Smtp-Source: AGHT+IHq7iLknH1kugU1iyFKcJu5RhFucHGYVHGJjULkBKmBw7MDQDO8QyIgGlIMrZxtIq2ellDjXRYw0D1kd1OP5Ys= X-Received: by 2002:a92:cf48:0:b0:363:9280:8912 with SMTP id c8-20020a92cf48000000b0036392808912mr4994002ilr.27.1706798615943; Thu, 01 Feb 2024 06:43:35 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alejandro Imass Date: Thu, 1 Feb 2024 15:43:24 +0100 Message-ID: Subject: Re: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? To: Odhiambo Washington Cc: FreeBSD virtualization Content-Type: multipart/alternative; boundary="00000000000034f1f00610530524" X-Rspamd-Queue-Id: 4TQhVK2r7zz4hnS X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] --00000000000034f1f00610530524 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Feb 1, 2024 at 1:40=E2=80=AFPM Odhiambo Washington wrote: ... > You're missing the fact that you could virtualize a Linux host and run al= l > those Docker/k8s inside it and just proxy access to them. > For example, I am running https://ushahidi.kictanet.or.ke/views/map > inside an Ubuntu VM under bhyve. Apache on the FreeBSD host is doing the > proxy. > > PS: I am lucky to be the 1st one to pour more fuel into your flame :-) > > Hi Odhiambo, thanks for opening the discussion! I am aware that hypervisor and full virtualization is an option, no doubt. But IMHO ideally it should be native jails with Bastille, CBSD or whatever. Best, --=20 Alex > --00000000000034f1f00610530524 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Feb 1, 2024 at 1:40=E2=80=AFP= M Odhiambo Washington <odhiambo@gm= ail.com> wrote:

...
=C2=A0
You're missing the fact that you = could virtualize a Linux host and run all those Docker/k8s inside it and ju= st proxy access to them.
For example, I am running=C2=A0https://us= hahidi.kictanet.or.ke/views/map inside an Ubuntu VM under bhyve. Apache= on the FreeBSD host is doing the proxy.

PS: I am = lucky to be the 1st one to pour more fuel into your flame :-)


Hi=C2=A0= Odhiambo, thanks for opening the discussion!

I am = aware that hypervisor and full virtualization is an option, no doubt.
=

But IMHO ideally it should be native jails with Bastill= e, CBSD or whatever.


Best,

--=C2=A0
Alex
--00000000000034f1f00610530524-- From nobody Thu Feb 1 14:47:38 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TQhbC5hMDz58myl for ; Thu, 1 Feb 2024 14:47:51 +0000 (UTC) (envelope-from aimass@yabarana.com) Received: from mail-il1-x133.google.com (mail-il1-x133.google.com [IPv6:2607:f8b0:4864:20::133]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TQhbC41hrz4jw8 for ; Thu, 1 Feb 2024 14:47:51 +0000 (UTC) (envelope-from aimass@yabarana.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-il1-x133.google.com with SMTP id e9e14a558f8ab-3637fb0bd6cso3832765ab.0 for ; Thu, 01 Feb 2024 06:47:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yabarana-com.20230601.gappssmtp.com; s=20230601; t=1706798869; x=1707403669; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ctpHBjdWmMSRYv0SHdSQYqincASZkH07YDYbiaGJ+fI=; b=dgq8rDUo9Ez6u9gg29Yqq0tJKvoHLgeZaZOJSXfcclS7doFg1zFccNRvpp4dfAzm8G NT1ITc6z+7eS/kOGjAPVf0GdFXOjNMzBhudsR7ZYfmsjuQ2PEVmUepFtV2Sfm1FqeSWo wEcOXy5EIAcMu9WGqqmuu9bxgFyDqNInqye9X7yqN+g6WIFhCc6xPjz/dGJypITh8GR5 xh7Por6x3Q/BjwyolkrczstjJQNqMn5NuZipGXlcG0YnFvpttxzLWA/k0nHQJPDmWHiR ThJ1t6NaeDnRnzXvhvTeNirxHDhOdT5jqCbyJ5aPsdbszw8vpiJt/M34QcNJcpopgZSc 6S7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706798869; x=1707403669; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ctpHBjdWmMSRYv0SHdSQYqincASZkH07YDYbiaGJ+fI=; b=aP3eDkT63T2SBCBLWzadUtNgglflZjaDgzV10rIVRmnSQ06UlUDmnIdEW8flX9YbOw bWZid4pqpJkgWKkRqCYhTp3lb4B+mwMn4aI5hgW4OPeDBCHgE7IyeA2nJvWYMIo5+acP GWhAywtY6DlnGimFz6hA5ctcPvuFjDMZBZHfwmDtw+oAo0nJDGIoviJttx9SEkQ29B5T w5ohDogMjQr6dmzN7K5RnvwCTC1Lasw/xlyTN8HPqSxEcef0pSsG8gLOKNd0C1Hjd0wc muNaCJ3GZtA/0iUWz/cX9nE4yOPd0WTocsWwI9kRi/eGCK9sWXMIR40HeeEnszEAyDcZ P7zg== X-Gm-Message-State: AOJu0YxdLDN5/N3Uw3Rwyc+ZFl+Ye6Md6U/7OAevAhvC3vmO25DRwCjE SI+ykx2OKVtXBoPrBJU0vm4VOQt1kjbSdN6yL4SRyIposClm/ImG3tlPuM+b3cvSpjBrbIzVw+O RW+SARMjmxMUNbtJjVhYOzzb3A5Ac1bWEUQiNZw== X-Google-Smtp-Source: AGHT+IHGIegOitJcTU07xqtCPgxfk27hX0ttJGceCpZc+MyLOllFnStwgHS3B7FAuyoSg9DMnhSWVVkzb3hRW+WjXsw= X-Received: by 2002:a92:cb4b:0:b0:363:792c:ac4e with SMTP id f11-20020a92cb4b000000b00363792cac4emr5110253ilq.19.1706798869457; Thu, 01 Feb 2024 06:47:49 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alejandro Imass Date: Thu, 1 Feb 2024 15:47:38 +0100 Message-ID: Subject: Re: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? To: Mario Marietto Cc: Odhiambo Washington , FreeBSD virtualization Content-Type: multipart/alternative; boundary="0000000000005140ce06105314f6" X-Rspamd-Queue-Id: 4TQhbC41hrz4jw8 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] --0000000000005140ce06105314f6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Feb 1, 2024 at 1:44=E2=80=AFPM Mario Marietto wrote: > Would be cool to investigate if the Linuxulator supports Kubernetes.... > > I understand that Linux native container can already run on BSD with Linuxulator layer. But I would rather traduce Dockerfile to Bastillefile of CBSDFile and go with native code. IMO, the issue is not so much the container tech, we invented that in the first place. The issue is not having a decent orchestrator / autoscaling service. Although I think that runj is the correct starting point: https://github.com/samuelkarp/runj Although I am not a k8s expert not sure if runj counts as an alternative to k8s or more like container runtime for k8s. Would be nice that actual k8s experts could chime in here ... > On Thu, Feb 1, 2024 at 1:40=E2=80=AFPM Odhiambo Washington > wrote: > >> >> >> On Thu, Feb 1, 2024 at 3:13=E2=80=AFPM Alejandro Imass >> wrote: >> >>> >>> ... > You're missing the fact that you could virtualize a Linux host and run al= l >> those Docker/k8s inside it and just proxy access to them. >> For example, I am running https://ushahidi.kictanet.or.ke/views/map >> inside an Ubuntu VM under bhyve. Apache on the FreeBSD host is doing the >> proxy. >> >> PS: I am lucky to be the 1st one to pour more fuel into your flame :-) >> >> >> -- >> Best regards, >> Odhiambo WASHINGTON, >> Nairobi,KE >> +254 7 3200 0004/+254 7 2274 3223 >> In an Internet failure case, the #1 suspect is a constant: DNS. >> "Oh, the cruft.", egrep -v '^$|^.*#' =C2=AF\_(=E3=83=84)_/=C2=AF :-) >> [How to ask smart questions: >> http://www.catb.org/~esr/faqs/smart-questions.html] >> > > > -- > Mario. > --0000000000005140ce06105314f6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Feb 1, 2024 at 1:44=E2=80=AFP= M Mario Marietto <marietto2008= @gmail.com> wrote:
Would= be cool to investigate if the Linuxulator supports Kubernetes....



I understand that Lin= ux native container can already run on BSD with Linuxulator layer. But I wo= uld rather traduce Dockerfile to Bastillefile of CBSDFile and go with nativ= e code.

IMO, the issue is not so much the containe= r tech, we invented that in the first place. The issue is not having a dece= nt orchestrator / autoscaling service.

Although I = think that runj=C2=A0is the correct starting point:

https://github.com/samuelk= arp/runj

Although I am not a k8s expert no= t sure if runj=C2=A0counts as an alternative to k8s or more like container = runtime for k8s. Would be nice that actual k8s experts could chime in here = ...


=C2=A0
=
On Thu, Fe= b 1, 2024 at 1:40=E2=80=AFPM Odhiambo Washington <odhiambo@gmail.com> wrote:


On Thu, Feb 1, 202= 4 at 3:13=E2=80=AFPM Alejandro Imass <aimass@yabarana.com> wrote:

<= /div>

...
=C2=A0
You're missing the fact that you could virtualize a Linux host and ru= n all those Docker/k8s inside it and just proxy access to them.
F= or example, I am running=C2=A0https://ushahidi.kictanet.or.ke/views/map in= side an Ubuntu VM under bhyve. Apache on the FreeBSD host is doing the prox= y.

PS: I am lucky to be the 1st one to pour more f= uel into your flame :-)


--
Best regards,
Odhiambo= WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
<= span style=3D"color:rgb(34,34,34)">=C2=A0In=C2=A0an Internet failure case, the #1 suspect is a constant: DN= S.
"Oh, the cruft.",=C2=A0egrep -v '^$|^.*#'=C2=A0=C2=AF\_= (=E3=83=84)_/=C2=AF=C2=A0:-)=
[How to ask smart questions:=C2= =A0http://www.catb.org/~esr/f= aqs/smart-questions.html]


--
Mario.
--0000000000005140ce06105314f6-- From nobody Thu Feb 1 14:56:16 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TQhn30rzWz58nrl for ; Thu, 1 Feb 2024 14:56:23 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from DEU01-BE0-obe.outbound.protection.outlook.com (mail-be0deu01on20701.outbound.protection.outlook.com [IPv6:2a01:111:f403:2622::701]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TQhn25PWBz4lMw for ; Thu, 1 Feb 2024 14:56:22 +0000 (UTC) (envelope-from hausen@punkt.de) Authentication-Results: mx1.freebsd.org; none ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=D6YefTWg0d9XYhmxE0WMGByKAm9H1KkFI+UQpAW3DdJszIo09qyQhVUGx9cK3DEnz10cYCDGEFt7VPKEaM0ol+zmurGR2/rEZ/hzSjvpFWI5ZV2yMjUdnE4eZphTd8UHqIS6ZO2jRwLP42otcYPPh/c8PY0hh/GSWzU2y9WIhef2ir1pCcHq0QYl0MRRdNIvh2G2b6ybxhHcci2gtpAkAJnhO1aDBL1xceB21/fA0tqpzLAiQ1w52QLRv+Nbcu6xyukXAyVAvTnwXhR5L6fGzT/4ul1DbRbhc4Tl4y9otmn48nynqzduUx3sFGt6KfycgRO6uiPsKJeLtzy4qvz2Zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=j4FJU0hqo39iYq6v4KZvDtuB0fk2eKQolRQx4PJBKKk=; b=QR95V1ciSAjPZDAroX2N6x9B0zAtfMyOoxFOk9Bl5qlUtsASYvoUUxV3fTnqGlmkQqpLM5szq7JrrK0bvOM0+Oh8UVoVkscLm0eAGiNvDRFKfltlW7gkyuID690BiRBVT40s94dFzvX66gxbmfacQjzQQVrivayK14BJh1EsGwlecHN2SrRKjrWE+KftZx5ToBQVFpOmnfyKQAAkt+R0aQqbu0EN/0TLjhIcYlVsukcZ7Ghcsw4vX1e5QoatPaCah+9fEnPzbNRqQNFy1C696TSksXyREfSmJRxqs66RmJNM9058m0+/NNirsAFgVR1WozzsA/zkSf8xva4she/b5w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=punkt.de; dmarc=pass action=none header.from=punkt.de; dkim=pass header.d=punkt.de; arc=none Received: from FRYP281MB0144.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:5::15) by FRYP281MB2558.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:77::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.32; Thu, 1 Feb 2024 14:56:16 +0000 Received: from FRYP281MB0144.DEUP281.PROD.OUTLOOK.COM ([fe80::6d88:c747:9d24:3c0c]) by FRYP281MB0144.DEUP281.PROD.OUTLOOK.COM ([fe80::6d88:c747:9d24:3c0c%5]) with mapi id 15.20.7249.025; Thu, 1 Feb 2024 14:56:16 +0000 From: "Patrick M. Hausen" To: FreeBSD virtualization CC: Mario Marietto , Odhiambo Washington , Alejandro Imass Subject: Re: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? Thread-Topic: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? Thread-Index: AQHaVQgEFqLpAxa/m0CyMJQ3xeuUWrD1bYeAgAAA7oCAACKuAIAAAlyA Date: Thu, 1 Feb 2024 14:56:16 +0000 Message-ID: <39C7327E-5A5A-4074-9EDA-1AC1BBE38A56@punkt.de> References: In-Reply-To: Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: FRYP281MB0144:EE_|FRYP281MB2558:EE_ x-ms-office365-filtering-correlation-id: defc45d6-a771-47ee-db4d-08dc2335f0dc x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: n+xyTykdYAbgridfdaTBTNovNrr+4t3dVEcgS8ItiCBBg8KOS+mfd2RylCkThjZSOkgpLzrQ0HQEl2W7Led0+XdV0CpmbeRz9FmlvkgG+b5NLffpAD2I5sZd7qfvlJ76QyVnQZLvDjaWzHuDxMUMRjfRw4sYekskcmla+9wIhy64duW+5NlpjkZcSXFWPY+iUC7QPG/grqznw7/dAVaDAmXDrIiQDLViORmFe/a5Sv2uFoSbrcmSgkGQUC57gaHo8T8nqSe0conYa4uC6/jhq770Vlxy6IFFyv/mMiyfH4B3nv1Y3m+XkAXuiu3IVxbQBO8zz49nJuTRDR/QM9sEIDstyy4Rwz+2/aHb4Pg6mvPCB+EAx8+6esQRbPOlxodXZC+PDFnzRI6LJvA+XFYOYKZ28bWsNri+fzpJT6mWVkoGQXNmUeMyJ+/+Gqjzxnvf6lg4Tm0M3VIUCj4w5oPRw7ERMA6pIsdCIPVjeiGbbuHktOmeyUetP7lhkRZYXAeviBKgCrVHvwYE//uTMs7WrSnKAQbIIm01fqvTfzqxu7glK03tjAtPYX/8/dDPVeChPpfYfHW84GV1obpLQ4QwZmf9mmwLQMcavIMPEdlq7WTHrtgcqVsTINZJcPCkaSuYVDt7XcKNH/NXn/fh7jP+MQ== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:FRYP281MB0144.DEUP281.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230031)(376002)(366004)(346002)(396003)(136003)(39830400003)(230922051799003)(64100799003)(186009)(1800799012)(451199024)(8676002)(966005)(66446008)(64756008)(8936002)(66946007)(54906003)(6486002)(76116006)(66476007)(66556008)(2906002)(4744005)(478600001)(6916009)(4326008)(316002)(6512007)(6506007)(86362001)(36756003)(5660300002)(41300700001)(38100700002)(33656002)(71200400001)(2616005)(38070700009)(122000001)(45980500001);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?bEpVOTErcDVYNWpFYXRIRnZCSzBiQWpyMmNscGlNdXRKUGN3QXlvU0wrbC9v?= =?utf-8?B?V05hcFdrMEJLMDlRNkpTeHFhOHRneWt0UzZ2dnhnR3FGb2RNMXhIdmFZUmE0?= =?utf-8?B?NE0vZ285NGExU0x0QzdNeDhsd3NsZm9WWVBGeThzQlI4QlVxWlZQMWJ6YTVt?= =?utf-8?B?YVRqUlpiMlY5ODB5NFo5czdiVkhiM0habWZhWitlaHNaMGY1RHJxa1k1elpy?= =?utf-8?B?S092Z0Rpa21EY1p0L0pROVBYQUdQS2NHdDQ3d2RGWkM2cVlyTks0cXFMOXJP?= =?utf-8?B?VW8ranFXSEVycHNrQjVjODBsMFgyRVlNL2YrS0JDcG16aUdyTTF3c2JwTE5C?= =?utf-8?B?UTI1RXQrSHZsSHY5QmFpVkdsUjN2MUloM00wcXZQOC92bHppR2ZwL0pyUkh6?= =?utf-8?B?QXlLeDI1bkFQSGdBYitQL2lVdXkzU1JPWURHT0R5NzZPaFFlV1BnQm8yLzBv?= =?utf-8?B?aXFYekJHUURtQmg1WXU2a210YzlzSCs0bzEwYkdOeTdMWW9nUnh1NFhJRVox?= =?utf-8?B?VzV1VDNIN3ZNUFVZL1dsMjYxemRZWDJrcngzL3ZYR0FMdjRoOXRmWHpodU8x?= =?utf-8?B?a1JsbnNXT0lZZ1d5cUlQNzJDVms4OGcyV1ZUQXBSQUtJRmltRDBTSVRLaVFs?= =?utf-8?B?QjlqbTRVTkFSS0s5d0dxYjJsTHJRNm90UUYyMVhlT251YWp3Ykt5TSt5V2x2?= =?utf-8?B?akc1N1ZsQzJLWXZOVFZPZjlzd21Ra3FtcWZKRFNLTHdIdnhqVm1vSzhDODcy?= =?utf-8?B?ZWduQ0R5WlFiOXBpZVBKVDAvZGt5MmEzS1hKbWJ2aU1BcjN4VXczazlEa1ZR?= =?utf-8?B?d3VxOHhPN3NwMnVRSU03cmlxOFBIQ1Z2ODcwaWpSbW5id2JWdFJvV0RvT2Zx?= =?utf-8?B?YlNSL3JSZFl3WWg5UE1CSitWMWlXSzdaZDUvU2RrWGwvKzhNcGlLdFI2SHl0?= =?utf-8?B?ZkpGdzVuR3pTUXdTWjMvU2tDcDZHMDdtUFh2TUp2UHpFaGlsQTZzdmp4Q29a?= =?utf-8?B?bzl2Z2tVWStHQnhNOU5UeE4xQkZYSS95NzdZalVuaDh6NElrR1Z4V3R0YWNT?= =?utf-8?B?NVBicTRyNzlZWFVvVWNBeVNGV2k1RlA0NHRXUHpRWFBvajFXaXRHZ3kyYmlx?= =?utf-8?B?dmlZWjNmVyttUXNrVTF6bWF1RTZma3hvTDlTVEhocHEycFFYTjdkMG5rU3h4?= =?utf-8?B?a3dsVTVXNUdWSXNrRmlIZWhyZGFva1BkYzFPVHRpZGVxK3NLM0JGcjQyTm0r?= =?utf-8?B?eUNZaE1PeUd3RHRpY2JrT0RSVEFYUjJoNFIzalpmNFl3NXhERW5oM0pWc2t3?= =?utf-8?B?NkJLajdOSis3amZLQVhDbzB6RUhKbTlzWnhDeFJqb2s3QjAvSGRRRTc4R2NH?= =?utf-8?B?RjBDTWYzc0V3QmhXakFTWG42cEhCSTFMejgxYXlOUzUxcThDKy81VEZPMTE4?= =?utf-8?B?RTAvV2NXOXpVUmJYajYrMmh5ZVdTRlV2OXo1eDRYdWhzQjNMMzZ0cGlJdDNm?= =?utf-8?B?K280S1B5THcwaEE5cFF0bVhhcGhaNE52UmVlUkxIMiszOUxJbXRiajIyTkdJ?= =?utf-8?B?NUtVbnNEajF1OGVXNkJreHRjR1dSYWJHNkFtbFF6MDg0bGdWT2NKZWRSVDV5?= =?utf-8?B?S1FlSUlGOEJEZ2VGWE9YYlpIVjUwd0RtaTRVbVREb3dLYkZiSGR4L3N1MG1x?= =?utf-8?B?d3dMemE1NldpSWlEME84cDgrRDFTOXBxMTAyQnZxZUd5MEpGK2dqaTgxbzJw?= =?utf-8?B?VzR4ZUNmeWlCZlE4OFNRalg5M2JQcWRwZlU2Q3JTYlUzMXk1Y0Y2K0NZUGUv?= =?utf-8?B?cmFSUWpBRWowL2Z6cjIrQU4xQUdjY3hhOU94WnlSMEloNkF6SVFZV1FiZ084?= =?utf-8?B?RWkxOGdkU3J0aUt2N0NhL1lxOGhmWmtWaG5kazc2ZVUzMVF0djhYMmJ4cEJQ?= =?utf-8?B?YlJpenhCYWxRazZ6UUtQV3VsVHQ4ZUJ3MlJDTll6NWFFYUN3eVhSYWpscklD?= =?utf-8?B?UndZWjhwSy9UWEIrYk1sd2d1R2crRjhoZkIybUpHYTZzYmJnYWZIRXdYNnQ2?= =?utf-8?B?NGtGRGMwcUYyNTVCN2UyR2ZNUDE2VjNDTzhmMWZPOHFITTY0VFVpelVSTWE2?= =?utf-8?B?YVd6T2Ftd1ozN2I1ak8vTFZZcDhHcVpHL285Szc4VnZpeHBRU2JYdHYvMmxO?= =?utf-8?Q?1bytwaLo2LVvgtxc45KPV3YgsdVaCFCZAl5gIVO+xJQx?= Content-Type: text/plain; charset="utf-8" Content-ID: Content-Transfer-Encoding: base64 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 X-OriginatorOrg: punkt.de X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: FRYP281MB0144.DEUP281.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: defc45d6-a771-47ee-db4d-08dc2335f0dc X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Feb 2024 14:56:16.0879 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: d1aa1808-3734-45fc-a490-f8ba49028756 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: MLjca8yaUkAu7duypuRp7QkM1IXBSHy9xlFHD6GmiyNOEXg0tkFVl21X0nurZ7ly X-MS-Exchange-Transport-CrossTenantHeadersStamped: FRYP281MB2558 X-Rspamd-Queue-Id: 4TQhn25PWBz4lMw X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US] SGkgYWxsLA0KDQo+IEFtIDAxLjAyLjIwMjQgdW0gMTU6NDcgc2NocmllYiBBbGVqYW5kcm8gSW1h c3MgPGFpbWFzc0B5YWJhcmFuYS5jb20+Og0KPiBJTU8sIHRoZSBpc3N1ZSBpcyBub3Qgc28gbXVj aCB0aGUgY29udGFpbmVyIHRlY2gsIHdlIGludmVudGVkIHRoYXQgaW4gdGhlIGZpcnN0IHBsYWNl LiBUaGUgaXNzdWUgaXMgbm90IGhhdmluZyBhIGRlY2VudCBvcmNoZXN0cmF0b3IgLyBhdXRvc2Nh bGluZyBzZXJ2aWNlLg0KDQpUaGVyZSdzIE5vbWFkIChodHRwczovL3d3dy5ub21hZHByb2plY3Qu aW8vKSBpbnRlZ3JhdGlvbg0Kd2l0aCBwb3Q6IGh0dHBzOi8vZ2l0aHViLmNvbS9ic2Rwb3QvcG90 DQoNClRoZSByaWdodCBwZXJzb24gdG8gY29udGFjdCBpcyBwcm9iYWJseSBMdWNhIChodHRwczov L2dpdGh1Yi5jb20vcGl6emFtaWcpLg0KSSdsbCBzZW5kIGhpbSBhIGhpbnQgYXQgdGhpcyB0aHJl YWQgaW5zdGVhZCBvZiBDYydpbmcgaGltIHdpdGhvdXQgYXNraW5nLg0KDQpLaW5kIHJlZ2FyZHMs DQpQYXRyaWNrDQotLSANCnB1bmt0LmRlIEdtYkgNClBhdHJpY2sgTS4gSGF1c2VuDQouaW5mcmFz dHJ1Y3R1cmUNCg0KU29waGllbnN0ci4gMTg3DQo3NjE4NSBLYXJsc3J1aGUNCg0KVGVsLiArNDkg NzIxIDkxMDk1MDANCg0KaHR0cHM6Ly9pbmZyYXN0cnVjdHVyZS5wdW5rdC5kZQ0KaW5mb0BwdW5r dC5kZQ0KDQpBRyBNYW5uaGVpbSAxMDgyODUNCkdlc2Now6RmdHNmw7xocmVyOiBEYW5pZWwgTGll bmVydCwgRmFiaWFuIFN0ZWluDQoNCg== From nobody Thu Feb 1 15:03:31 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TQhxZ3WDqz58pWD for ; Thu, 1 Feb 2024 15:03:46 +0000 (UTC) (envelope-from aimass@yabarana.com) Received: from mail-il1-x12f.google.com (mail-il1-x12f.google.com [IPv6:2607:f8b0:4864:20::12f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TQhxY6SWJz4mDh for ; Thu, 1 Feb 2024 15:03:45 +0000 (UTC) (envelope-from aimass@yabarana.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-il1-x12f.google.com with SMTP id e9e14a558f8ab-363898391beso2593315ab.1 for ; Thu, 01 Feb 2024 07:03:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yabarana-com.20230601.gappssmtp.com; s=20230601; t=1706799825; x=1707404625; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=TIBRff+517DvwDJNIhh44orLg3F6Q1Ay2OaOKc6+DRg=; b=iiJPJFY47aLhcJbBGLZIInxVwDIj5sGAWRkAkfpIOe5HgInqNswxA0Fbl4P6Stu2Cn /XNeEL7xKig3pPuPMx/vH9mDMCvzJIKgVpeXMIjOdmZxNfApTOIvuHLcww6jcSwoSg8/ g5zixCdn91z7iij0wbGmBwkC565hoPL23lnrTu6eb6/o2fimlVeS28RVzXUobHT5WyNe 70xAAs4ERiwqiNOOZE90CwkFBTv737x0Yu4wtASot+lcxTvwe/XmnnXjoj14Qwv/fIyn IAgmBCn7B5ZchNqgEgRL9vwmUVW54h84qDiE5qUtQLOH5EIDRJ2uGPCpkbeZVcbF3bK9 fJdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706799825; x=1707404625; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=TIBRff+517DvwDJNIhh44orLg3F6Q1Ay2OaOKc6+DRg=; b=bHGnV8lYZ33HfQSm4BPYO+5jXCHBHRh4Eng88gIZHbKgWFFU8WAKu/JE7wWxKLnX3l KaTjyTxuurzihxQvrn9Rj63FLIuyjAavm+gQFyzg60Yei4D630yoOU0wiu3q3Uqrz0GL FAsJWOOA4L59ibQ25SxcvmUPRn3pLgIM7biGHhZBtc+VPHRIWbQ5PDVFK52VWWIVQ1rI pOAnAnhGIaphzEpkNlRLZUGbfcvTNWm1lS4W+Ob+x0XeWL/M++z2Ked8zovRV2eTBgsj BG5OASdzZ21ab8WI+qEzg3kfgTqmnQ7wrfjliFpmKxySgZDKkRhdSDeUVDEuhspvzWHB Trkw== X-Gm-Message-State: AOJu0YzZXYjW3WKAn1SpRERV8KXnfRPBxuXEwZGb4ogU8CTUQ23d44S2 ybH03e6pz9CwgDdHI8WavT6xHID7+teOcNt4l8BHCwlCwidijhFxwMZ1rRTi/7DSqFWBiLYvb89 Fv8/9siBcdBwclhyv99Ahm1MoJLAEdZ0MVrQgMw== X-Google-Smtp-Source: AGHT+IG1xbURIFus5cCWAV+lZPye1rUeWX8y/jZVfmNKN8FMiVq+OZGmauiCwGOtZJ4vI1ktWKB9FHwjnNfSsjDNW7E= X-Received: by 2002:a05:6e02:2184:b0:363:8166:ed86 with SMTP id j4-20020a056e02218400b003638166ed86mr4691634ila.8.1706799822397; Thu, 01 Feb 2024 07:03:42 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: <39C7327E-5A5A-4074-9EDA-1AC1BBE38A56@punkt.de> In-Reply-To: <39C7327E-5A5A-4074-9EDA-1AC1BBE38A56@punkt.de> From: Alejandro Imass Date: Thu, 1 Feb 2024 16:03:31 +0100 Message-ID: Subject: Re: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? To: "Patrick M. Hausen" Cc: FreeBSD virtualization , Mario Marietto , Odhiambo Washington Content-Type: multipart/alternative; boundary="0000000000001e0bc20610534d3e" X-Rspamd-Queue-Id: 4TQhxY6SWJz4mDh X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] --0000000000001e0bc20610534d3e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Feb 1, 2024 at 3:56=E2=80=AFPM Patrick M. Hausen = wrote: > Hi all, > > > Am 01.02.2024 um 15:47 schrieb Alejandro Imass : > > IMO, the issue is not so much the container tech, we invented that in > the first place. The issue is not having a decent orchestrator / > autoscaling service. > > There's Nomad (https://www.nomadproject.io/) integration > with pot: https://github.com/bsdpot/pot > > The right person to contact is probably Luca (https://github.com/pizzamig > ). > I'll send him a hint at this thread instead of Cc'ing him without asking. > > Thank you! I saw Nomad but I am worried about the move to BUSL. I didn't know it had integration with pot. Shouldn't be that hard to support Bastille, CBSD and IOCage if the latter is still alive. May forking Nomad is an option, they did it for Terraform and pretty quick: https://opentofu.org Best, --=20 Alex > Kind regards, > Patrick > -- > > --0000000000001e0bc20610534d3e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Feb 1, 2024 at 3:56=E2=80=AFP= M Patrick M. Hausen <hausen@punkt.de<= /a>> wrote:
Hi all,

> Am 01.02.2024 um 15:47 schrieb Alejandro Imass <aimass@yabarana.com>:
> IMO, the issue is not so much the container tech, we invented that in = the first place. The issue is not having a decent orchestrator / autoscalin= g service.

There's Nomad (https://www.nomadproject.io/) integration
with pot: https://github.com/bsdpot/pot

The right person to contact is probably Luca (https://github.com/pizzamig= ).
I'll send him a hint at this thread instead of Cc'ing him without a= sking.


Thank you! I saw Nomad but I am worrie= d about the move to BUSL.=C2=A0
I didn't know it had integrat= ion with pot. Shouldn't be that hard to support Bastille, CBSD and IOCa= ge if the latter is still alive.
May forking Nomad is an option, = they did it for Terraform and pretty quick:=C2=A0https://opentofu.org


Bes= t,

--=C2=A0
Alex
=C2=A0
<= blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l= eft-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);pa= dding-left:1ex"> Kind regards,
Patrick
--

--0000000000001e0bc20610534d3e-- From nobody Thu Feb 1 15:05:21 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TQj065tkPz58pNF for ; Thu, 1 Feb 2024 15:05:58 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TQj0642KMz4mVd for ; Thu, 1 Feb 2024 15:05:58 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x635.google.com with SMTP id a640c23a62f3a-a35e9161b8cso141897066b.3 for ; Thu, 01 Feb 2024 07:05:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706799957; x=1707404757; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=bkzQ9q0c/9fHbs0ks+mlErfWt2yPRT3R8LPAtoTlSZo=; b=H0F85Q18ktpDI6hQGaoloFGXu0uGYyAT8JzpFWxQeWsPoCTloDG7toh/A+8j5SFs2s /3tsh1uVZ4cs89YHGy3ERS0tyIupd4Ci8B2I96tVNwbT7rvyow4NmjlMXK3DX5oQ5QbU +/aBqkXvIVjfBqAWgeKksyo/ZEC5Hb6EO+o7oU5/BGG3ootAJHM8CfZLfn9rHUVy+VgV 2UZ6XYyO7AGZ6nzsNlWHgmkLFBBQCRaiiVd1mkQRosPwRWTYdlZPFcnggxAr3qEaGnji hF3bJKdcoIR5/RUBc0MF4GE9wlcnkhVuLjiaunQBverF3CppW6G43WtSTFSDUmSxEhyy Jzkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706799957; x=1707404757; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=bkzQ9q0c/9fHbs0ks+mlErfWt2yPRT3R8LPAtoTlSZo=; b=vpWQCQ+uSdTVlmIVAkqBy/F+6qSCtlFGZd2VW1zNiGjUP47BDwraLlkUsbteEN/nCv 9Dl5xANQWk8l77B/YggcsXVf797unFQYpmR9T1VxRU+wcXR1wY++S75XXveD2mw1M6YS QZVFZenb0u57PYiVwo3ZWwn//8AExhLxVKh+QdCYLzNoL/9cJoJlwU2OxLzZ2tKfmJ0h lp25hMcrfIi2gZ22CrMB5qFaBq0Vs+eas5ZR47ppsdC9EyBKWHhZMbVjNrT+eeiSN94x tarD3uZIp43MFl0Dl4cs63I6epD8jYUisekKprzOSz/brCPt9f6Mp1Jpa9HUkyOy1NgR MKng== X-Gm-Message-State: AOJu0Yz6m7vWjsynbLuHkGgpd4bUOHPbKG9Ak1mIDqkkLtTWKQW4cSJz AarmYmfP7BaYPKXGq82ocYjUSUyjgXj43c3yPfeCaqK6hWcYxJY9PM5uZuAjhOv1FhDHR2ZeKub xc/lV8P6jeZD7zRYjyJeIOSejKUpGcNIx X-Google-Smtp-Source: AGHT+IGxRzqMx03X1RMPSclW/jNNn+0ImiqjkV/hpHtwIIJ23p8oAaiVX9GhKo+AnOEsydd2+OE5ZXBaYIejZZesmTE= X-Received: by 2002:a17:906:1c4e:b0:a36:7e17:8929 with SMTP id l14-20020a1709061c4e00b00a367e178929mr3385573ejg.69.1706799957252; Thu, 01 Feb 2024 07:05:57 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Mario Marietto Date: Thu, 1 Feb 2024 16:05:21 +0100 Message-ID: Subject: Re: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? To: Alejandro Imass Cc: Odhiambo Washington , FreeBSD virtualization Content-Type: multipart/alternative; boundary="000000000000279f7006105355b5" X-Rspamd-Queue-Id: 4TQj0642KMz4mVd X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] --000000000000279f7006105355b5 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable ---> I understand that Linux native containers can already run on BSD with the Linuxulator layer. Docker can't run on the linuxulator. Curious to see if kubernetes and or runj do it. Me too I know that native code is better,but I know that no one on FreeBSD wants,so instead of using bhyve,if I can,I prefer the lighter Linuxulator. On Thu, Feb 1, 2024 at 3:47=E2=80=AFPM Alejandro Imass wrote: > > > On Thu, Feb 1, 2024 at 1:44=E2=80=AFPM Mario Marietto > wrote: > >> Would be cool to investigate if the Linuxulator supports Kubernetes.... >> >> > > I understand that Linux native container can already run on BSD with > Linuxulator layer. But I would rather traduce Dockerfile to Bastillefile = of > CBSDFile and go with native code. > > IMO, the issue is not so much the container tech, we invented that in the > first place. The issue is not having a decent orchestrator / autoscaling > service. > > Although I think that runj is the correct starting point: > > https://github.com/samuelkarp/runj > > Although I am not a k8s expert not sure if runj counts as an alternative > to k8s or more like container runtime for k8s. Would be nice that actual > k8s experts could chime in here ... > > > > >> On Thu, Feb 1, 2024 at 1:40=E2=80=AFPM Odhiambo Washington >> wrote: >> >>> >>> >>> On Thu, Feb 1, 2024 at 3:13=E2=80=AFPM Alejandro Imass >>> wrote: >>> >>>> >>>> > ... > > >> You're missing the fact that you could virtualize a Linux host and run >>> all those Docker/k8s inside it and just proxy access to them. >>> For example, I am running https://ushahidi.kictanet.or.ke/views/map >>> inside an Ubuntu VM under bhyve. Apache on the FreeBSD host is doing th= e >>> proxy. >>> >>> PS: I am lucky to be the 1st one to pour more fuel into your flame :-) >>> >>> >>> -- >>> Best regards, >>> Odhiambo WASHINGTON, >>> Nairobi,KE >>> +254 7 3200 0004/+254 7 2274 3223 >>> In an Internet failure case, the #1 suspect is a constant: DNS. >>> "Oh, the cruft.", egrep -v '^$|^.*#' =C2=AF\_(=E3=83=84)_/=C2=AF :-) >>> [How to ask smart questions: >>> http://www.catb.org/~esr/faqs/smart-questions.html] >>> >> >> >> -- >> Mario. >> > --=20 Mario. --000000000000279f7006105355b5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
---> I understand that Linux native containers can= already run on BSD with the Linuxulator layer.

Docker can't run on the linuxulator. Curious to see if kubernetes an= d or runj do it. Me too I know that native code is better,but I know that n= o one on FreeBSD wants,so instead of using bhyve,if I can,I prefer the ligh= ter Linuxulator.

On Thu, Feb 1, 2024 at 3:47=E2=80=AFPM Alejandro= Imass <aimass@= yabarana.com> wrote:


On Thu, Feb 1, 2024 at 1:= 44=E2=80=AFPM Mario Marietto <marietto2008@gmail.com> wrote:
Would be cool to i= nvestigate if the Linuxulator supports Kubernetes....



I understand that Linux native con= tainer can already run on BSD with Linuxulator layer. But I would rather tr= aduce Dockerfile to Bastillefile of CBSDFile and go with native code.
=

IMO, the issue is not so much the container tech, we in= vented that in the first place. The issue is not having a decent orchestrat= or / autoscaling service.

Although I think that ru= nj=C2=A0is the correct starting point:


Although I am not a k8s expe= rt not sure if runj=C2=A0counts as an alternative to k8s or more like conta= iner runtime for k8s. Would be nice that actual k8s experts could chime in = here ...


=C2=A0
On Thu, Feb 1, 2024 at 1:40=E2=80=AFPM Odhiambo W= ashington <odhia= mbo@gmail.com> wrote:


On Thu, Feb 1, 2024 at 3= :13=E2=80=AFPM Alejandro Imass <aimass@yabarana.com> wrote:


...
<= div>=C2=A0
You're missing the= fact that you could virtualize a Linux host and run all those Docker/k8s i= nside it and just proxy access to them.
For example, I am running= =C2=A0https://ushahidi.kictanet.or.ke/views/map inside an Ubuntu VM under = bhyve. Apache on the FreeBSD host is doing the proxy.

<= div>PS: I am lucky to be the 1st one to pour more fuel into your flame :-)<= /div>


--
Best regards,
Odhiambo WASHINGTON,
Nairobi,= KE
+254 7 3200 0004/+254 7 2274 3223
=C2=A0In=C2=A0an Inte= rnet failure case, the #1 suspect is a constant: DNS.
"Oh, the cruft.",=C2=A0egrep -v '^= $|^.*#'=C2=A0=C2=AF\_(=E3=83=84)_/=C2=AF=C2=A0:-)
[How to ask smart questions:=C2=A0http://www.catb.org/~esr/faqs/smart-questions.h= tml]


--
Mario.


--
Mario.
--000000000000279f7006105355b5-- From nobody Thu Feb 1 15:11:09 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TQj6r47Pgz58pq5 for ; Thu, 1 Feb 2024 15:11:48 +0000 (UTC) (envelope-from sweekly44@gmail.com) Received: from mail-qt1-x831.google.com (mail-qt1-x831.google.com [IPv6:2607:f8b0:4864:20::831]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TQj6r2JYMz4nbj for ; Thu, 1 Feb 2024 15:11:48 +0000 (UTC) (envelope-from sweekly44@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-qt1-x831.google.com with SMTP id d75a77b69052e-42aa4a9d984so9165171cf.1 for ; Thu, 01 Feb 2024 07:11:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706800306; x=1707405106; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=V/y8U4AvVkcnz6a8qWCszu+EhPiBi8iQ2tlaPe6bdO8=; b=biIxXdIW0f3LU9ndYPx8cxK08aJoLMKE+aKhEjwQsfMsI9ryF/p3/xLD74rBFiOw/5 ZjxSTyluAFTFw+N5TaucoriQUm6KDgDDCrSY6o5Z4dMTFmtCmCd36SLgbc/x4VADc4QW 6UawUiJjl4hVFaQG1E2iTVcjcNhQrXw8QAGujeHaEz9PHXKtMQhXRikWkPHJmyJSgbuR hQ5SaFNNr2oyUmb1V1KWB21HDZhcVZA+5mQXMNCbRP5eXuZhLhVnoVayoxEVpgv6BbQk LJxHbHnqjsO7Z9umkn9IG7+QIEjJMvrqPcjbB0wifDs7Zd7G2xMNq477DS5k6i2Yg2wE bsOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706800306; x=1707405106; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=V/y8U4AvVkcnz6a8qWCszu+EhPiBi8iQ2tlaPe6bdO8=; b=l9eufLIk8fbxuWZHIOC+haTJf+iHw1Mj5SCwG1kLRAEump1AxedErkclJjyHJ9+VS1 mJYBFX+xf2sfh1sAbqVG/FCuB4+XPdFnhvETnVfUaxurnY0pAHe/hVinhCy8TCcqBTgk TFnw1i39xCLKspeUwh6otibWCvTulX+jbXmNFm3cua+qX+3L56KlZcAjoXFeRVcZs4b0 NZyKGdZd31E07LGl2frhqrOb8N34pUy0E69v1+eUEJzZww3QVRorgw510Na9U4e6JGgs mibGC7zPRvZ8v5c6vvHedCzN1F6MAxTGtdCwRzEPIdE01RGvNWlAUYbkAdz/ZBcWMJJX uGyg== X-Gm-Message-State: AOJu0YwlNyh/ncBr9v/iv+xcRbC8m38JgCZD44MAK6UANQuHmcakQ/GC mx3hKf+q7c4Ee6i8pcXbdFxtI3xMHsAGwV8IZ8igAIkCf6XstvvixlWIVm3hSoPuhBQLaZuA11Y 5XKnk9fJ8kOOOS4p7igqmrSnqKvqorrSt X-Google-Smtp-Source: AGHT+IHl8D5Hceb6oN5+w24AZ7mIkz2Ut/NVWdjv0iuLU6Ahe0uuWsGlMUarH/Z1EfVC2SqeunHN2OO0M+F+3DgR+Zs= X-Received: by 2002:a0c:a999:0:b0:68c:7f46:8327 with SMTP id a25-20020a0ca999000000b0068c7f468327mr1114892qvb.3.1706800305646; Thu, 01 Feb 2024 07:11:45 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: <39C7327E-5A5A-4074-9EDA-1AC1BBE38A56@punkt.de> In-Reply-To: From: Sam W Date: Thu, 1 Feb 2024 10:11:09 -0500 Message-ID: Subject: Re: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? To: Alejandro Imass Cc: "Patrick M. Hausen" , FreeBSD virtualization , Mario Marietto , Odhiambo Washington Content-Type: multipart/alternative; boundary="000000000000ebaf960610536962" X-Rspamd-Queue-Id: 4TQj6r2JYMz4nbj X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] --000000000000ebaf960610536962 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi all, I came across this k8s fork recently, but haven't had a chance to give it a try. Perhaps it's also worth investigating. https://github.com/tnorlin/kubernetes/releases Regards, Sam On Thu, Feb 1, 2024 at 10:04=E2=80=AFAM Alejandro Imass wrote: > > > On Thu, Feb 1, 2024 at 3:56=E2=80=AFPM Patrick M. Hausen wrote: > >> Hi all, >> >> > Am 01.02.2024 um 15:47 schrieb Alejandro Imass : >> > IMO, the issue is not so much the container tech, we invented that in >> the first place. The issue is not having a decent orchestrator / >> autoscaling service. >> >> There's Nomad (https://www.nomadproject.io/) integration >> with pot: https://github.com/bsdpot/pot >> >> The right person to contact is probably Luca (https://github.com/pizzami= g >> ). >> I'll send him a hint at this thread instead of Cc'ing him without asking= . >> >> > Thank you! I saw Nomad but I am worried about the move to BUSL. > I didn't know it had integration with pot. Shouldn't be that hard to > support Bastille, CBSD and IOCage if the latter is still alive. > May forking Nomad is an option, they did it for Terraform and pretty > quick: https://opentofu.org > > > Best, > > -- > Alex > > >> Kind regards, >> Patrick >> -- >> >> --000000000000ebaf960610536962 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi all,

I came across this k8s fork recen= tly, but haven't had a chance to give it a try. Perhaps it's also w= orth investigating.

https://github.com/tnorlin/kubernetes/releas= es

Regards,
Sam

On Thu, Feb = 1, 2024 at 10:04=E2=80=AFAM Alejandro Imass <aimass@yabarana.com> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l= eft:1px solid rgb(204,204,204);padding-left:1ex">


On Thu, Feb 1, 2024 at 3:56=E2=80=AFPM Patrick M. Hausen <<= a href=3D"mailto:hausen@punkt.de" target=3D"_blank">hausen@punkt.de>= wrote:
Hi all,<= br>
> Am 01.02.2024 um 15:47 schrieb Alejandro Imass <aimass@yabarana.com>:
> IMO, the issue is not so much the container tech, we invented that in = the first place. The issue is not having a decent orchestrator / autoscalin= g service.

There's Nomad (https://www.nomadproject.io/) integration
with pot: https://github.com/bsdpot/pot

The right person to contact is probably Luca (https://github.com/pizzamig= ).
I'll send him a hint at this thread instead of Cc'ing him without a= sking.


Thank you! I saw Nomad but I am worrie= d about the move to BUSL.=C2=A0
I didn't know it had integrat= ion with pot. Shouldn't be that hard to support Bastille, CBSD and IOCa= ge if the latter is still alive.
May forking Nomad is an option, = they did it for Terraform and pretty quick:=C2=A0https://opentofu.org

=
Best,

--=C2=A0
Alex
=
=C2=A0
Kind regards,
Patrick
--

--000000000000ebaf960610536962-- From nobody Thu Feb 1 15:18:50 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TQjHC2tz0z58qYr for ; Thu, 1 Feb 2024 15:19:03 +0000 (UTC) (envelope-from aimass@yabarana.com) Received: from mail-il1-x135.google.com (mail-il1-x135.google.com [IPv6:2607:f8b0:4864:20::135]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TQjHB20qZz4pJc for ; Thu, 1 Feb 2024 15:19:02 +0000 (UTC) (envelope-from aimass@yabarana.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-il1-x135.google.com with SMTP id e9e14a558f8ab-3638e3ace68so3935235ab.2 for ; Thu, 01 Feb 2024 07:19:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yabarana-com.20230601.gappssmtp.com; s=20230601; t=1706800741; x=1707405541; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ogwzF2flHk4gHXdQaiukRNSk7teGiXQleqCXKgRjosQ=; b=M1UbePkyTjAgQYMzaTz0/7zY64y50S7MzSxsF4ew1IGyBAx1UD6VXMrSeyE9jan/M/ bC4rQ1jY3EAGJMGd1D04vbKDm6n4o3h8vbxj/rzSpsrGOP0b7llkWrlz0sH3q72ry/pD +ld//2iLKWKaC+ZPGS0CVEWsY89GuOHrc3Kq41Lp2FXs7bkAgVrOIOT3j/m8J+iTvoG5 ZgL9yXx0MCQ4fq9oIei2hts8QbYLeaXIcAGp7mWDy40HnJ2ouNJVebzbMXZEhib3egmR mkgXQO13pnY52wHBbikC5XsybcUD3LZv8Q1a8yBYW6Yh5qZA3VzKWkfZ7QsH/i+q3xUa uPBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706800741; x=1707405541; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ogwzF2flHk4gHXdQaiukRNSk7teGiXQleqCXKgRjosQ=; b=KXW6Zv8Tn6VtbZ5Shl9orcl0T3dzbEXzEUzpH7lmDilmcXGDXk4H2/ae5/Ed5XYoNl 7q2+SSdxuPQblhUAo9gASh0u4MLjx5ycaQ4UucWPI/9cJXvrqWX7eI2JxtAU84N7jLad Rjm7uf+o5POESFSssqECcXl7oQ3wAvo01bvrnV4yJBEmNrp6/UqfG/QElVEvQ8TSMlI8 p0HiroZHEDDjUcseDd/7PWVXZpReF0URyfqXYLLIDDvv1cylvtiBf1LBgj3BzuY215xf T0wki5J2PCmHGK8Vn/ePdWxEVoyxB1/22NP8IQRGD2I5suf2RZCxeW3kdnkp09w7Rz36 FC+Q== X-Gm-Message-State: AOJu0Ywr96v6Mu5NgZnT0auS291beojnKOSmKQG9eMNqaEBUcz1NRpFs eIzC6epVdWQFQur/ijKkfSJed+GQnhLFlTunHGZmHyjaW2fDSVT5XKcxXmzUbykKK7flEd2Aois iRHwhWZMNttvXPevodmAzRMHO8/8w1HkXWsBofexjCVipOQf++HE= X-Google-Smtp-Source: AGHT+IFfgrKcy4xcpQNcQfSAElg4Xs+RJrwv7x4vdBdTN0LmNHg50TNaNsnDfZTjPdX9gQ33U3vP3eCe5SESyQhDWv4= X-Received: by 2002:a92:c8cc:0:b0:363:8618:87ce with SMTP id c12-20020a92c8cc000000b00363861887cemr5626796ilq.12.1706800741080; Thu, 01 Feb 2024 07:19:01 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alejandro Imass Date: Thu, 1 Feb 2024 16:18:50 +0100 Message-ID: Subject: Re: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? To: Mario Marietto Cc: Odhiambo Washington , FreeBSD virtualization Content-Type: multipart/alternative; boundary="000000000000dffb5a061053837f" X-Rspamd-Queue-Id: 4TQjHB20qZz4pJc X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] --000000000000dffb5a061053837f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Feb 1, 2024 at 4:05=E2=80=AFPM Mario Marietto wrote: > ---> I understand that Linux native containers can already run on BSD wit= h > the Linuxulator layer. > > Docker can't run on the linuxulator. Curious to see if kubernetes and or > runj do it. Me too I know that native code is better,but I know that no o= ne > on FreeBSD wants,so instead of using bhyve,if I can,I prefer the lighter > Linuxulator. > Well, it has run in the past: https://wiki.freebsd.org/Docker But I don't think anyone really wants Docker and there's the hypervisor for that. The intent of the thread is to deliberate on native freebsd orchestration and autoscaling. I don't think anyone would mind re-writing Dockerfile to Bastillefile or whatever. What's missing is the other part, the k8s equivalent. --000000000000dffb5a061053837f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Feb 1, 2024 at 4:05=E2=80=AFP= M Mario Marietto <marietto2008= @gmail.com> wrote:
= ---> I understand that Linux native containers can already run on BSD wi= th the Linuxulator layer.

Docker can't ru= n on the linuxulator. Curious to see if kubernetes and or runj do it. Me to= o I know that native code is better,but I know that no one on FreeBSD wants= ,so instead of using bhyve,if I can,I prefer the lighter Linuxulator.
<= /div>

Well, it has run in the past:= =C2=A0https://wiki.freebsd.org/= Docker

But I don't think anyone really wan= ts Docker and there's the hypervisor for that. The intent of the thread= is to deliberate on native freebsd orchestration and autoscaling.=C2=A0
I don't think anyone would mind re-writing Dockerfile to Bastil= lefile or whatever. What's missing is the other part, the k8s equivalen= t.=C2=A0


=C2=A0
--000000000000dffb5a061053837f-- From nobody Thu Feb 1 16:07:00 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TQkLq70hBz58vY7; Thu, 1 Feb 2024 16:07:15 +0000 (UTC) (envelope-from antranigv@freebsd.am) Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TQkLp69Jwz4tbw; Thu, 1 Feb 2024 16:07:14 +0000 (UTC) (envelope-from antranigv@freebsd.am) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=freebsd.am header.s=fm2 header.b=arwJTWSR; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=Ta3AHeRU; dmarc=none; spf=pass (mx1.freebsd.org: domain of antranigv@freebsd.am designates 64.147.123.20 as permitted sender) smtp.mailfrom=antranigv@freebsd.am Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id 9E73F3200B1B; Thu, 1 Feb 2024 11:07:13 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Thu, 01 Feb 2024 11:07:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.am; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:message-id:mime-version:reply-to :subject:subject:to:to; s=fm2; t=1706803633; x=1706890033; bh=I8 YTIycc5F2Xa1tQWaL91VCimu2G1EozsTBp3Qizlm0=; b=arwJTWSRJONrHOltfv T1Wu6qbiF6a8jNR8c/3BJmndIMiFm/BtVHER7M7uhoeIWhV589zVwEbMDHN3NND4 B4D68iAcp1l5DlyjMh0QXh/E3bXG45P8he0jMYhseN1LI8PU07ZiHAyBysy6zgTG kMjqZOSPSzdYNkj6o8Rj+SP8iA/01TPHsvAmPN/PThSQTEGnMP8kNSeBmaqfbC1N 5FG6iKdF/K4GCjnF9j+6e3EF2taBTxvaxG6tjIW9odGzZWzCrylSA0wn1ksjOosl 32xiYGKDxdGO2sjTUbYH7mpyrg/G+9zvdl1TMWp4hpoZb5aj5/Uz+o8YzxT+7uO1 qF2w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:message-id:mime-version:reply-to:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1706803633; x=1706890033; bh=I8YTIycc5F2Xa 1tQWaL91VCimu2G1EozsTBp3Qizlm0=; b=Ta3AHeRUXctk2+gI3l2g1Xwb5MCG9 Lru0cMvvOCx/VPM07CI+J0xfu+rtqlGBqUat7E8/BMhVCx7KQU+Pz5uk5kza4j0M nwKMpK2B1/EITu/+z3BFfcIX5MejA02U9mYCTDJVeNeXALjD4G0pCIt9HaIQhgz/ x2om+zX8PtGH9RGKl/4NY+/irIfKhOa+xzS62w7zMhUEaL962iCzELP/JdxOedR+ 6CBbfYmIqr0agLYh602+qPSKeQKpllyQZH1AGzAKgYeVXxF0kvdyhRB0hul77iiT EkZA8SIiPH7EA9pEgHlZOhFH8AUS3leWUdW9v9OXGFGjdhQn7/9qwZ+Vg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrfeduuddgkeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhtgfgggfukfffvefvofesthhqmh dthhdtjeenucfhrhhomheptehnthhrrghnihhgucggrghrthgrnhhirghnuceorghnthhr rghnihhgvhesfhhrvggvsghsugdrrghmqeenucggtffrrghtthgvrhhnpeeggfeuueffie dvgfefueeifeehheeuvdegheffledufefhlefhkefftdektdettdenucffohhmrghinhep rghnthhrrghnihhgvhdrrghmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpe hmrghilhhfrhhomheprghnthhrrghnihhgvhesfhhrvggvsghsugdrrghm X-ME-Proxy: Feedback-ID: ibc494664:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 1 Feb 2024 11:07:11 -0500 (EST) From: Antranig Vartanian Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: if_ixl: Admin Queue memory allocation issue hangs the server Message-Id: <7F49AE24-D3BC-4103-A522-75CB7BE477D0@freebsd.am> Date: Thu, 1 Feb 2024 20:07:00 +0400 Cc: "freebsd-net@freebsd.org" , "freebsd-hardware@freebsd.org" To: "freebsd-virtualization@freebsd.org" X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.95 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_SHORT(-0.95)[-0.946]; RWL_MAILSPIKE_EXCELLENT(-0.40)[64.147.123.20:from]; R_DKIM_ALLOW(-0.20)[freebsd.am:s=fm2,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.20:c]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.20:from]; DKIM_TRACE(0.00)[freebsd.am:+,messagingengine.com:+]; TO_DN_EQ_ADDR_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[freebsd.am]; RCVD_COUNT_THREE(0.00)[3]; RCPT_COUNT_THREE(0.00)[3]; MLMMJ_DEST(0.00)[freebsd-hardware@FreeBSD.org,freebsd-net@FreeBSD.org,freebsd-virtualization@FreeBSD.org]; ARC_NA(0.00)[]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+] X-Rspamd-Queue-Id: 4TQkLp69Jwz4tbw Greetings y=E2=80=99all, We have a massive server with two IXL NICs (Ethernet Controller X710 for = 10GBASE-T). One of them is on the host and the other is passed to a = guest running in a bhyve VM. Every once in a while, the system hangs. We cannot SSH, we cannot ping = nor we can login using console (IPMI remote screen). However, on the console, I see the following messages: "ixl0: ixl_process_adminq: Unable to allocate memory for Admin Queue = event!=E2=80=9D And it keeps repeating. The only way to get access to the host is to=E2=80=A6 reboot it. We don=E2=80=99t use SR-IOV (although when we did, the same thing = happened), we don=E2=80=99t use large MTUs (1500) nor we have a memory = issue (the system has 2TB of RAM, and right before hanging around 250G = is available). Any tip would be appreciated. P.S. we also tried the kmod from ports, still seeing the same issue. Kind regards, =E2=80=94 Antranig Vartanian https://antranigv.am/ PGP Key ID: 0x2D59F21C From nobody Thu Feb 1 23:18:03 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TQvw15dTlz58tCP for ; Thu, 1 Feb 2024 23:18:09 +0000 (UTC) (envelope-from paul@redbarn.org) Received: from util.redbarn.org (util.redbarn.org [24.104.150.222]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "*.redbarn.org", Issuer "RapidSSL TLS RSA CA G1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TQvw04sBFz4s9w for ; Thu, 1 Feb 2024 23:18:08 +0000 (UTC) (envelope-from paul@redbarn.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=redbarn.org header.s=util header.b=kKrMlnWF; dmarc=pass (policy=reject) header.from=redbarn.org; spf=pass (mx1.freebsd.org: domain of paul@redbarn.org designates 24.104.150.222 as permitted sender) smtp.mailfrom=paul@redbarn.org Received: from family.redbarn.org (family.redbarn.org [24.104.150.213]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "*.redbarn.org", Issuer "RapidSSL TLS RSA CA G1" (not verified)) by util.redbarn.org (Postfix) with ESMTPS id 04CF919CCAE for ; Thu, 1 Feb 2024 23:18:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=redbarn.org; s=util; t=1706829485; bh=WscEUpGUNUq605foPEhQmJ+qsXWOMuTQLsJBkNNha8c=; h=Subject:To:References:From:Date:In-Reply-To; b=kKrMlnWFf3qKpAnlItwV77GwOaV4329cDZV9vz7Z9SS8IY5B1F1U5WtbZYdCpdNap HtnaZ2mSpOCtSU6DCyi6A2oPZoBc3SpBDc72Pud1OWm2WcOVF60uvadN6Tcsa0Aqmz nxveGMDs6qRlMV9SRYO+ABBSABccqgSwYNfTsZDA= Received: from [24.104.150.166] (dhcp-166.access.rits.tisf.net [24.104.150.166]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by family.redbarn.org (Postfix) with ESMTPSA id BA11EC3F1F for ; Thu, 1 Feb 2024 23:18:04 +0000 (UTC) Subject: Re: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? To: FreeBSD virtualization References: From: Paul Vixie Message-ID: Date: Thu, 1 Feb 2024 15:18:03 -0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 PostboxApp/7.0.60 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.70 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; DWL_DNSWL_LOW(-1.00)[redbarn.org:dkim]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; DMARC_POLICY_ALLOW(-0.50)[redbarn.org,reject]; RCVD_IN_DNSWL_MED(-0.20)[24.104.150.213:received]; R_SPF_ALLOW(-0.20)[+ip4:24.104.150.0/24]; R_DKIM_ALLOW(-0.20)[redbarn.org:s=util]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_ALL(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[redbarn.org:+]; FREEFALL_USER(0.00)[paul]; ASN(0.00)[asn:33651, ipnet:24.104.150.0/24, country:US]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[] X-Rspamd-Queue-Id: 4TQvw04sBFz4s9w Alejandro Imass wrote on 2024-02-01 07:18: > ... > > But I don't think anyone really wants Docker and there's the hypervisor > for that. The intent of the thread is to deliberate on native freebsd > orchestration and autoscaling. > I don't think anyone would mind re-writing Dockerfile to Bastillefile or > whatever. What's missing is the other part, the k8s equivalent. I think Docker and K8S and other successful / dominant forms of containers in the OSS world are platform-specific simply because that's what their creators and early adopters cared about. Adding more forms of platform-specific container technology (for example, taking explicit advantage of Bastille or other FreeBSD features) would not be a value add since its adopters would likely still have to support other platforms. To be worth doing, the outcome should be platform-agnostic, allowing a container creator to not-have-to-care what the underlying operating system was. "Write Once Run Anywhere." seems to be an example of putting the container-maker first and insulating them from details they won't care about such as what the underlying platform is running. Good abstraction boundaries make good neighbors, as they say. "We" should not be opposed to Docker per se nor K8S. Linux became dominant by focusing on what its users wanted to be able to do. Docker and K8S likewise. If we have value to add to that mix, it won't be in the form of bespoke or BSD-lockin alternatives. It might be in inclusive and platform-agnostic alternatives. -- P Vixie From nobody Thu Feb 1 23:39:34 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TQwPW0Cllz58w1B for ; Thu, 1 Feb 2024 23:40:15 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TQwPV5Y2rz4vQh for ; Thu, 1 Feb 2024 23:40:14 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x52b.google.com with SMTP id 4fb4d7f45d1cf-55f19a3ca7aso5111121a12.1 for ; Thu, 01 Feb 2024 15:40:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706830810; x=1707435610; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=XQinCytRmEpbShtwCGGRsRGNObLnYwhWm7HmA7XElD0=; b=NU/6O05RP4PTIIW2TOioEhA2IhrfbLPwGFQesWue102EPYb+3HFgb/LH/9D2YvmZQD 4DgabWVJ10QEgWRmBMYj3jZqUMApkAzRniMybXb+3cSsCEcvyQOK4ivTUyBcr8T+dX7o sHqHPLL9fFkih65r0krv2Kuj9P0KYE3PFusU/6BJYQJ5fX8MnBptJTbZOepXZSHgYl0O Ic2NAdvW3lohT/S2L3oP8QdbZ5D5Tnjut7dAe+cvldxs+ieU3gdCKtcZMY5StYV3UC4p JlomptmelPFR9k5Hq6vfh2FyHyLWKz0KKp7m+2aZybbJlX4fFg29Minz4hK6Wb0nvOvb MD8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706830810; x=1707435610; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=XQinCytRmEpbShtwCGGRsRGNObLnYwhWm7HmA7XElD0=; b=SPzTHgoAAIK0FA42t6/ezze5AdPFJMEQ+cPoOFddf0ycZSYwSI7V1xkCWdijUpYpXp Pgz41SxM10nhRq7x27VF78/9bfbxJqlvnKup61hPIxzduY2wX12/rdyX+hiAUqqmMeGf sHG+ORIVo00aQym2VBIbzEuULmGaRBzLgVtM6iBp3CogAG1QWisks7N+CLOmcrzyWFI2 0imCxh909+myCaKnYcISWRgkX5tkA1EmEfEeNcoX7IDFHLKGQY7mXpCuxIIKb3R7Gu/1 dCi9GsEBRyL8zq3kWqfto7LPj1HYuBxL3UrglKHij6Z+jr9Sdig4vQyhzRv9OrOG7Y8F O6TQ== X-Gm-Message-State: AOJu0YxGF7moD5WjasT89ifv66IPQcyjFR7chDHwZf0U8CUU2AhnDzEv k1+ZxUDgOApvIWKplMY7n2NfHO8kBYm4QFTvprwmkEPSeqSRy0NS5etsw4wMXEGMqfopfm5fjmI LUuSnev6P5FAEiDTvAhhm0pJljYJXBUST5T0= X-Google-Smtp-Source: AGHT+IEn5sEwJW1IH+5IefBbhnf2CaUTUOca+uvrF9Pa7s3P5qAnqj2+LZDrQoR8kEpFYWvtQwIkm9RjnMBB3rdewWw= X-Received: by 2002:a17:907:7e9c:b0:a37:60e:3cfc with SMTP id qb28-20020a1709077e9c00b00a37060e3cfcmr66032ejc.0.1706830810395; Thu, 01 Feb 2024 15:40:10 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Mario Marietto Date: Fri, 2 Feb 2024 00:39:34 +0100 Message-ID: Subject: Re: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? To: Paul Vixie Cc: FreeBSD virtualization Content-Type: multipart/alternative; boundary="00000000000025380d06105a84eb" X-Rspamd-Queue-Id: 4TQwPV5Y2rz4vQh X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] --00000000000025380d06105a84eb Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Excuse me. I'm not very experienced,but isn't the L4 or any other microkernel a valid alternative to containers ? If I have understood correctly how it works,it allows multiple instances of the various services implemented within the microkernel OS. How many instances can we have ? For example in the L4 Linux kernel webpage it is explained that it can boot FreeBSD in cooperation with Linux. It works like xen. WIth xen we can have multiple virtual machines. But xen today has been preferred to kvm. And anyway,we always talk about monolithic kernels. So,I want to ask : is a microkernel OS a valid alternative to the containers ? If it allows to run only some services of the "virtualized" os,why not use it ? Why not invest effort and time to implement this solution as an alternative to the containers that FreeBSD already has ? Personally I like the idea of seeing Linux and FreeBSD work together. On Fri, Feb 2, 2024 at 12:18=E2=80=AFAM Paul Vixie wrote= : > > > Alejandro Imass wrote on 2024-02-01 07:18: > > ... > > > > But I don't think anyone really wants Docker and there's the hypervisor > > for that. The intent of the thread is to deliberate on native freebsd > > orchestration and autoscaling. > > I don't think anyone would mind re-writing Dockerfile to Bastillefile o= r > > whatever. What's missing is the other part, the k8s equivalent. > > I think Docker and K8S and other successful / dominant forms of > containers in the OSS world are platform-specific simply because that's > what their creators and early adopters cared about. Adding more forms of > platform-specific container technology (for example, taking explicit > advantage of Bastille or other FreeBSD features) would not be a value > add since its adopters would likely still have to support other > platforms. To be worth doing, the outcome should be platform-agnostic, > allowing a container creator to not-have-to-care what the underlying > operating system was. "Write Once Run Anywhere." > > seems to be an example > of putting the container-maker first and insulating them from details > they won't care about such as what the underlying platform is running. > Good abstraction boundaries make good neighbors, as they say. > > "We" should not be opposed to Docker per se nor K8S. Linux became > dominant by focusing on what its users wanted to be able to do. Docker > and K8S likewise. If we have value to add to that mix, it won't be in > the form of bespoke or BSD-lockin alternatives. It might be in inclusive > and platform-agnostic alternatives. > > -- > P Vixie > > > --=20 Mario. --00000000000025380d06105a84eb Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Excuse me. I'm not very experienced,but isn't the = L4 or any other microkernel a valid alternative to containers ? If I have u= nderstood correctly how it works,it allows multiple instances of the variou= s services implemented within the microkernel OS. How many instances can we= have ? For example in the L4 Linux kernel webpage it is explained that it = can=C2=A0boot FreeBSD in cooperation with Linux. It works like xen. WIth xe= n we can have multiple virtual machines. But xen today has been preferred t= o kvm. And anyway,we always talk about monolithic kernels. So,I want to ask= : is a microkernel OS a valid alternative to the containers ? If it allows= to run only some services of the "virtualized" os,why not use it= ? Why not invest effort and time to implement this solution as an alternat= ive to the containers that FreeBSD already has ? Personally I like the idea= of seeing Linux and FreeBSD work together.

On Fri, Feb 2, 2024 at 12:1= 8=E2=80=AFAM Paul Vixie <paul@redbar= n.org> wrote:


Alejandro Imass wrote on 2024-02-01 07:18:
> ...
>
> But I don't think anyone really wants Docker and there's the h= ypervisor
> for that. The intent of the thread is to deliberate on native freebsd =
> orchestration and autoscaling.
> I don't think anyone would mind re-writing Dockerfile to Bastillef= ile or
> whatever. What's missing is the other part, the k8s equivalent.
I think Docker and K8S and other successful / dominant forms of
containers in the OSS world are platform-specific simply because that's=
what their creators and early adopters cared about. Adding more forms of platform-specific container technology (for example, taking explicit
advantage of Bastille or other FreeBSD features) would not be a value
add since its adopters would likely still have to support other
platforms. To be worth doing, the outcome should be platform-agnostic,
allowing a container creator to not-have-to-care what the underlying
operating system was. "Write Once Run Anywhere."

<https://github.com/tnorlin/kubernetes/releases= > seems to be an example
of putting the container-maker first and insulating them from details
they won't care about such as what the underlying platform is running. =
Good abstraction boundaries make good neighbors, as they say.

"We" should not be opposed to Docker per se nor K8S. Linux became=
dominant by focusing on what its users wanted to be able to do. Docker
and K8S likewise. If we have value to add to that mix, it won't be in <= br> the form of bespoke or BSD-lockin alternatives. It might be in inclusive and platform-agnostic alternatives.

--
P Vixie




--
Mario.
--00000000000025380d06105a84eb-- From nobody Fri Feb 2 07:54:33 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TR7Mv4xdzz58CjC for ; Fri, 2 Feb 2024 07:54:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TR7Mv249tz4fCd for ; Fri, 2 Feb 2024 07:54:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706860475; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nn1dnO8H175ODLxxUWQa77vKbIZB0h4hfEtZIAbtlm0=; b=bITfh8B+i/80fcvV3gUpJ/U5RZige6viZHuo8zZuhi4hz4WvFOVC8QOU8eslAg6pK0MiQ1 QBJ0lwy1JxK6gGvttzGGJ2m076hShBoGIpSyL70Udf1t8fomeg2XHZrQWKeImdeCTdFYZH NX6z1CPSyntnmZzdu/UWLpp/mKHkwAiUpFIG1dDbC5c5wdEhbGTX+UBErSgZQ58ER6DL+q aHdVbuGlpVq/7vQb1FuHWiWpfI8ZjZyU5ZllVG1/tjYbc0B+u3eD4ELBm96TooYBxPPcBN EhCRa1JjakSk8bCxcaV2X0U6iiscnIdU49rR3cEfWeHdfgi4G/ecI7DW3dgXEQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706860475; a=rsa-sha256; cv=none; b=qeLE/DKFmt9GB32yTTPU6hY24xuDgqo6sYyF3mli0LaHuvurOh/ei36W1w6jhJPk49liqm IVXkqid0Dc68LjC8NGjqlshR6HmT2Z/wrGv7XoHflcOMyEkAZSFE3Fvhaz2cW6C2ZGT5rl Ci68Snoto/br7+j0m+iKLRUbcoiFdOqV6qrUIGgCXaGjTjGGvbnINOSc82umBfGbyb9A2x DjWq/yBwIWoMHx9w0g3DwHm74Cug8WmH6mFn+DJC/NhKY8/LrAr47ztykbwPeEs6FY2wpU rTxWIKMPaPpg7GglinHsDPdH4vZpHoKSvqKVYZvE520NY0veqH4H7AlebXTTgQ== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TR7Mt68hkzM1F for ; Fri, 2 Feb 2024 07:54:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 4127sYf1031314 for ; Fri, 2 Feb 2024 07:54:34 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 4127sYAR031312 for virtualization@FreeBSD.org; Fri, 2 Feb 2024 07:54:34 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 273772] zpool nested in ZVOL becomes inaccessible when zfsd(8) is running on the host Date: Fri, 02 Feb 2024 07:54:33 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 14.0-STABLE X-Bugzilla-Keywords: needs-qa, regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: zarychtam@plan-b.pwste.edu.pl X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D273772 --- Comment #11 from Marek Zarychta --- Now both: bhyve host and guest are running FreeBSD stable/14. ZFS pools on = host and guest were upgraded. The report is still valid and the issue is persist= ent. Maybe the transition to mirror (the guest is nested on a ZVOL of pool with= a top-level-vdev that had previously been removed) has something to do with it (see bug 273663). The guest is using host's ZVOL zroot/ZVOL/virtbsd as vtbd= 0. This is how ZFS looks on the host:=20 # zpool status zroot pool: zroot state: ONLINE scan: scrub repaired 0B in 00:11:06 with 0 errors on Fri Jan 26 03:21:55 = 2024 scan warning: skipped blocks that are only referenced by the checkpoint. remove: Removal of vdev 1 copied 15.1G in 0h0m, completed on Fri May 15 20:04:34 2020 119K memory used for removed device mappings checkpoint: created Mon Dec 4 20:10:12 2023, consumes 49.6G config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 da0p2 ONLINE 0 0 0 da1p2 ONLINE 0 0 0 errors: No known data errors # zpool get all zroot NAME PROPERTY VALUE SOURCE zroot size 476G - zroot capacity 63% - zroot altroot - default zroot health ONLINE - zroot guid 6762271100094851702 - zroot version - default zroot bootfs zroot/ROOT/default14 local zroot delegation on default zroot autoreplace off default zroot cachefile - default zroot failmode wait default zroot listsnapshots off default zroot autoexpand off default zroot dedupratio 1.00x - zroot free 176G - zroot allocated 300G - zroot readonly off - zroot ashift 0 default zroot comment - default zroot expandsize - - zroot freeing 0 - zroot fragmentation 58% - zroot leaked 0 - zroot multihost off default zroot checkpoint 49.6G - zroot load_guid 2254167222130845727 - zroot autotrim on local zroot compatibility off default zroot bcloneused 0 - zroot bclonesaved 0 - zroot bcloneratio 1.00x - zroot feature@async_destroy enabled local zroot feature@empty_bpobj active local zroot feature@lz4_compress active local zroot feature@multi_vdev_crash_dump enabled local zroot feature@spacemap_histogram active local zroot feature@enabled_txg active local zroot feature@hole_birth active local zroot feature@extensible_dataset active local zroot feature@embedded_data active local zroot feature@bookmarks enabled local zroot feature@filesystem_limits enabled local zroot feature@large_blocks enabled local zroot feature@large_dnode enabled local zroot feature@sha512 enabled local zroot feature@skein enabled local zroot feature@edonr enabled local zroot feature@userobj_accounting active local zroot feature@encryption enabled local zroot feature@project_quota active local zroot feature@device_removal active local zroot feature@obsolete_counts active local zroot feature@zpool_checkpoint active local zroot feature@spacemap_v2 active local zroot feature@allocation_classes enabled local zroot feature@resilver_defer enabled local zroot feature@bookmark_v2 enabled local zroot feature@redaction_bookmarks enabled local zroot feature@redacted_datasets enabled local zroot feature@bookmark_written enabled local zroot feature@log_spacemap active local zroot feature@livelist enabled local zroot feature@device_rebuild enabled local zroot feature@zstd_compress enabled local zroot feature@draid enabled local zroot feature@zilsaxattr active local zroot feature@head_errlog active local zroot feature@blake3 enabled local zroot feature@block_cloning enabled local zroot feature@vdev_zaps_v2 active local # zfs get all zroot/ZVOL/virtbsd NAME PROPERTY VALUE SOURCE zroot/ZVOL/virtbsd type volume - zroot/ZVOL/virtbsd creation Wed Mar 10 8:59 2021 - zroot/ZVOL/virtbsd used 109G - zroot/ZVOL/virtbsd available 144G - zroot/ZVOL/virtbsd referenced 11.3G - zroot/ZVOL/virtbsd compressratio 1.13x - zroot/ZVOL/virtbsd reservation none default zroot/ZVOL/virtbsd volsize 50G local zroot/ZVOL/virtbsd volblocksize 8K - zroot/ZVOL/virtbsd checksum on default zroot/ZVOL/virtbsd compression lz4 inherit= ed from zroot zroot/ZVOL/virtbsd readonly off default zroot/ZVOL/virtbsd createtxg 5153440 - zroot/ZVOL/virtbsd copies 1 default zroot/ZVOL/virtbsd refreservation 51.6G local zroot/ZVOL/virtbsd guid 1428736673261547338 - zroot/ZVOL/virtbsd primarycache all default zroot/ZVOL/virtbsd secondarycache all default zroot/ZVOL/virtbsd usedbysnapshots 45.8G - zroot/ZVOL/virtbsd usedbydataset 11.3G - zroot/ZVOL/virtbsd usedbychildren 0B - zroot/ZVOL/virtbsd usedbyrefreservation 51.5G - zroot/ZVOL/virtbsd logbias latency default zroot/ZVOL/virtbsd objsetid 66109 - zroot/ZVOL/virtbsd dedup off default zroot/ZVOL/virtbsd mlslabel none default zroot/ZVOL/virtbsd sync standard default zroot/ZVOL/virtbsd refcompressratio 1.21x - zroot/ZVOL/virtbsd written 85.1M - zroot/ZVOL/virtbsd logicalused 64.4G - zroot/ZVOL/virtbsd logicalreferenced 13.6G - zroot/ZVOL/virtbsd volmode default default zroot/ZVOL/virtbsd snapshot_limit none default zroot/ZVOL/virtbsd snapshot_count none default zroot/ZVOL/virtbsd snapdev hidden default zroot/ZVOL/virtbsd context none default zroot/ZVOL/virtbsd fscontext none default zroot/ZVOL/virtbsd defcontext none default zroot/ZVOL/virtbsd rootcontext none default zroot/ZVOL/virtbsd redundant_metadata all default zroot/ZVOL/virtbsd encryption off default zroot/ZVOL/virtbsd keylocation none default zroot/ZVOL/virtbsd keyformat none default zroot/ZVOL/virtbsd pbkdf2iters 0 default zroot/ZVOL/virtbsd snapshots_changed Fri Feb 2 4:20:00 2024 - This is how ZFS looks inside the guest:=20 [virtbsd] ~# gpart show =3D> 40 104857520 vtbd0 GPT (50G) 40 1024 1 freebsd-boot (512K) 1064 94371840 2 freebsd-zfs (45G) 94372904 10484656 3 freebsd-swap (5.0G) [virtbsd] ~# zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEA= LTH=20 ALTROOT vmzroot 44.5G 8.98G 35.5G - - 32% 20% 1.00x ONL= INE=20 - [virtbsd] ~# zpool status pool: vmzroot state: ONLINE scan: scrub repaired 0B in 00:00:16 with 0 errors on Sat Jan 27 03:01:33 = 2024 config: NAME STATE READ WRITE CKSUM vmzroot ONLINE 0 0 0 gpt/vmzfs0 ONLINE 0 0 0 errors: No known data errors [virtbsd] ~# zpool get all vmzroot NAME PROPERTY VALUE SOUR= CE vmzroot size 44.5G - vmzroot capacity 20% - vmzroot altroot - defa= ult vmzroot health ONLINE - vmzroot guid 14496911069470365237 - vmzroot version - defa= ult vmzroot bootfs vmzroot/ROOT/default local vmzroot delegation on defa= ult vmzroot autoreplace off defa= ult vmzroot cachefile - defa= ult vmzroot failmode wait defa= ult vmzroot listsnapshots off defa= ult vmzroot autoexpand off defa= ult vmzroot dedupratio 1.00x - vmzroot free 35.5G - vmzroot allocated 8.98G - vmzroot readonly off - vmzroot ashift 0 defa= ult vmzroot comment - defa= ult vmzroot expandsize - - vmzroot freeing 0 - vmzroot fragmentation 32% - vmzroot leaked 0 - vmzroot multihost off defa= ult vmzroot checkpoint - - vmzroot load_guid 9652613418712187666 - vmzroot autotrim on local vmzroot compatibility off defa= ult vmzroot bcloneused 0 - vmzroot bclonesaved 0 - vmzroot bcloneratio 1.00x - vmzroot feature@async_destroy enabled local vmzroot feature@empty_bpobj active local vmzroot feature@lz4_compress active local vmzroot feature@multi_vdev_crash_dump enabled local vmzroot feature@spacemap_histogram active local vmzroot feature@enabled_txg active local vmzroot feature@hole_birth active local vmzroot feature@extensible_dataset active local vmzroot feature@embedded_data active local vmzroot feature@bookmarks enabled local vmzroot feature@filesystem_limits enabled local vmzroot feature@large_blocks enabled local vmzroot feature@large_dnode enabled local vmzroot feature@sha512 enabled local vmzroot feature@skein enabled local vmzroot feature@edonr enabled local vmzroot feature@userobj_accounting active local vmzroot feature@encryption enabled local vmzroot feature@project_quota active local vmzroot feature@device_removal enabled local vmzroot feature@obsolete_counts enabled local vmzroot feature@zpool_checkpoint enabled local vmzroot feature@spacemap_v2 active local vmzroot feature@allocation_classes enabled local vmzroot feature@resilver_defer enabled local vmzroot feature@bookmark_v2 enabled local vmzroot feature@redaction_bookmarks enabled local vmzroot feature@redacted_datasets enabled local vmzroot feature@bookmark_written enabled local vmzroot feature@log_spacemap active local vmzroot feature@livelist enabled local vmzroot feature@device_rebuild enabled local vmzroot feature@zstd_compress enabled local vmzroot feature@draid enabled local vmzroot feature@zilsaxattr active local vmzroot feature@head_errlog active local vmzroot feature@blake3 enabled local vmzroot feature@block_cloning enabled local vmzroot feature@vdev_zaps_v2 active local --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Fri Feb 2 12:30:03 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TRFV15c5Kz58fmf for ; Fri, 2 Feb 2024 12:30:17 +0000 (UTC) (envelope-from aimass@yabarana.com) Received: from mail-il1-x12b.google.com (mail-il1-x12b.google.com [IPv6:2607:f8b0:4864:20::12b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TRFV139XDz429t for ; Fri, 2 Feb 2024 12:30:17 +0000 (UTC) (envelope-from aimass@yabarana.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-il1-x12b.google.com with SMTP id e9e14a558f8ab-363793e88abso7849115ab.1 for ; Fri, 02 Feb 2024 04:30:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yabarana-com.20230601.gappssmtp.com; s=20230601; t=1706877015; x=1707481815; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=fY9Q4W35/VPDxtF4GzyLovwt9AT9C/Sn5Riav9ppJHI=; b=WMjj36nM+tT7Vc5BWWCXP+UCFD4We9cSK1S7GrZZlhoh5nVuDAxFNitvBecwTbpqD4 q1xD8nlDz4jElqkkQT5HesW38hrc/XKvMmOODSTp/1SDEpyKRU9/Gq7EO2l5ZssYwwu+ bnw2tKwdL1EKRn61gmR1VXZMX7ID2qujx9XOs26j5Fd9hHUoNFQgGrrfQIap/IVEJpA8 W/qNFSn14pLFj/8vojOdFclbMQOWQyrqozPWHMQLyNg873TXaOVcBQoFjGjdLahpA3J+ ASNczhB5e6wiBsLMqJH+yLpj0pGXMcPBfSVIULtXL+gK3gmgbufH4NcbZg643sMcidXT P6XA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706877015; x=1707481815; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=fY9Q4W35/VPDxtF4GzyLovwt9AT9C/Sn5Riav9ppJHI=; b=IYzcee9lEMuS2gG6A1Rmup8GC7Lcn8zvwelXAXdZw3BRFpI5fMD+c45/ggdvOP2dlz Wqt5RN1u86w97YDmtUVXV2e+9tCLubdGgnXVUo/i4ir2FPHBoDrDpdfWKWlh/FivKfrF dEtIzXhCjHAiWMyFjodeG7QIBUxfs/xdI5CGGnkpfZxy4yQJM4mYb1cEfqePNMEmKIur tf/IfbZZhiI6sgkm1qt1H1uHdDcC6YTkKpCT7MNPN8VGIU5dvz/NJQ+7Sxs4uucovs8h MP9aHDTh+Fv5aRp/lr+7XsV4Il6r3exk2MBSKztvh0LRLezAap6Wi9GAVzh46V9JQu57 a4RQ== X-Gm-Message-State: AOJu0Yww7XH/WkC09iXKCHLzuAS9czBiqQluCmbH6sQr1RuYbet9aH4P cYzSWMrFSmKSyh1+3KMiwwSmkCS0fVf0dK75vrN+YDj/lvJfM3f1pXrJzyJk/+TFSxHK/WyBN3o fYWBg8DsxV/QBB7ncoX0XtbnYutsJrk3y8Xvmkw== X-Google-Smtp-Source: AGHT+IFSNfiy+iCXHDdcFwSphAJdAEetGe0UNhrKylMl+TFObgFxntXFFd4QiWb6iuw8WJyJAhU0kU0exW734VH81nw= X-Received: by 2002:a05:6e02:1aaa:b0:363:8ce0:c8c0 with SMTP id l10-20020a056e021aaa00b003638ce0c8c0mr7790742ilv.15.1706877014927; Fri, 02 Feb 2024 04:30:14 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alejandro Imass Date: Fri, 2 Feb 2024 13:30:03 +0100 Message-ID: Subject: Re: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? To: Mario Marietto Cc: Paul Vixie , FreeBSD virtualization Content-Type: multipart/alternative; boundary="00000000000026833806106546e8" X-Rspamd-Queue-Id: 4TRFV139XDz429t X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] --00000000000026833806106546e8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Feb 2, 2024 at 12:40=E2=80=AFAM Mario Marietto wrote: > Excuse me. I'm not very experienced,but isn't the L4 or any other > microkernel a valid alternative to containers ? If I have understood > correctly how it works,it allows multiple instances of the various servic= es > implemented within the microkernel OS. How many instances can we have ? F= or > example in the L4 Linux kernel webpage it is explained that it can boot > FreeBSD in cooperation with Linux. It works like xen. WIth xen we can hav= e > multiple virtual machines. But xen today has been preferred to kvm. And > anyway,we always talk about monolithic kernels. So,I want to ask : is a > microkernel OS a valid alternative to the containers ? If it allows to ru= n > only some services of the "virtualized" os,why not use it ? Why not inves= t > effort and time to implement this solution as an alternative to the > containers that FreeBSD already has ? Personally I like the idea of seein= g > Linux and FreeBSD work together. > > IMHO, way beyond the scope of this thread. AFAIK neither Linux nor FreeBSD are microkernels. The original question is more like what would it take to run k8s natively using different jail managers like CBSD and BastilleBSD as drivers? Does it already exist? maybe: https://github.com/tnorlin/kubernetes What is the state? What are the limitations? etc. etc. --00000000000026833806106546e8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Fri, Feb 2, 2024 at 12:40=E2=80=AF= AM Mario Marietto <marietto200= 8@gmail.com> wrote:
Excus= e me. I'm not very experienced,but isn't the L4 or any other microk= ernel a valid alternative to containers ? If I have understood correctly ho= w it works,it allows multiple instances of the various services implemented= within the microkernel OS. How many instances can we have ? For example in= the L4 Linux kernel webpage it is explained that it can=C2=A0boot FreeBSD = in cooperation with Linux. It works like xen. WIth xen we can have multiple= virtual machines. But xen today has been preferred to kvm. And anyway,we a= lways talk about monolithic kernels. So,I want to ask : is a microkernel OS= a valid alternative to the containers ? If it allows to run only some serv= ices of the "virtualized" os,why not use it ? Why not invest effo= rt and time to implement this solution as an alternative to the containers = that FreeBSD already has ? Personally I like the idea of seeing Linux and F= reeBSD work together.
=

IMHO, way beyond the scope of th= is thread. AFAIK neither Linux nor FreeBSD are microkernels.=C2=A0

The original question is more like what would it take to r= un k8s natively using different jail managers like CBSD and BastilleBSD as = drivers?
Does it already exist? maybe:=C2=A0https://github.com/tnorlin/kubernetes
What is the state? What are the limitations?
etc. etc.


=C2=A0
--00000000000026833806106546e8-- From nobody Fri Feb 2 13:28:51 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TRGpL3tWkz58llL for ; Fri, 2 Feb 2024 13:29:30 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TRGpK6njXz46Pj for ; Fri, 2 Feb 2024 13:29:29 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x635.google.com with SMTP id a640c23a62f3a-a318ccfe412so236712366b.1 for ; Fri, 02 Feb 2024 05:29:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706880568; x=1707485368; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=zZy2Kpt+I0vdR2sAUxDZ/t9iacBXb+f3cvJi0JbN/Rs=; b=JC/lirbbJHbLxs0ZQ2Td+cdZWcyquvF+un5IM60R3pvjwb08PgFJapPnC7ZBb4TKvH 4OjgIADXfOtSHywwpW9U9QhfSoR8rpszp1Lo5wSenR2rkPFU/iCRTt8V+vNcfhi5eIny OUVEPEd9lybS6O3sV9HTiQ5SuVeNIyKsSw78tuz8QWs37y+cklYM5806Cqoot8zj+Ehy NqnsWpWqRpcG/jnjrrgRcHQ6TFsSQXEfcw/Zc+ySy15/Ncb4rYXwaMs5qzFXQJrYe0e5 BTpP+u2+kTEvVQqQcpgnZMw0/dTmcu/QLoxGmQf5IIt4L2DbvOTtDjk+Vtw4IsA4CEPs pEoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706880568; x=1707485368; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=zZy2Kpt+I0vdR2sAUxDZ/t9iacBXb+f3cvJi0JbN/Rs=; b=KBxf4yK/l8I9mAKqCi0hDd8jX8kB2RJDqP/T2l5VyZ4/y7UgunRp4zFfKEhygGR1UQ fb1c/yoPp89E5lR40RCXBDZTwxg3b4q6+Q+FDv8OIAqra7DU0ylai4WZB1mHxbp+HAj5 wfp2r82cppyohFCPbqVNHjEPFoIAve1AYrywVJYNjBO9BneCwLRxdzavUtmKGiuZ5SNX BDpPl3qwO7lvqrQsVypj4eB1XnyLvE1Mvwd2O3OkoZtFxA8bmJIvr1bzQm7yfjWIbWZG kHpvwDnmsaL1HHkh3jDAqtOt0Xy+zlCLJnR2//L2+A/FH1kgiILISZE2aNvEnBXf3Gc/ WmcA== X-Gm-Message-State: AOJu0Yyy+rksMNyKbYXgzQUXhZb2MAIW0d+VD3Nnk8TH45TptW13CuLy /ViSRvUaQGyVk/Jb1OKMi6p55Rw+k/9msk+EBimXad25uil/CgNftQh7S6fmzqeJuxUR7KYlO7T 2MeiTH73t871Sr+4tEbANRQQDJ1zQXX6o2aUslw== X-Google-Smtp-Source: AGHT+IHNxTQxP7mMQh3HwKG+5nii19wAmEV1PSgP1cshY4B3VRpSxWC31TpXjPgiIRIO+j/hVYXF2setNBRpuG9OD+0= X-Received: by 2002:a17:907:2da6:b0:a35:6a61:e3dc with SMTP id gt38-20020a1709072da600b00a356a61e3dcmr1861378ejc.16.1706880567466; Fri, 02 Feb 2024 05:29:27 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Mario Marietto Date: Fri, 2 Feb 2024 14:28:51 +0100 Message-ID: Subject: Re: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? To: Alejandro Imass Cc: Paul Vixie , FreeBSD virtualization Content-Type: multipart/alternative; boundary="000000000000e5dee20610661905" X-Rspamd-Queue-Id: 4TRGpK6njXz46Pj X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] --000000000000e5dee20610661905 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Alejandro,I'm not sure if you are aware that the discussion at a certain point has broadened and deepened. I don't understand why I can't propose an alternative to the actual jail and jail managers that you are talking about. And no. On the L4 website there is a userland based on new versions of Ubuntu and Debian that can be used with the microkernel offered. They also say that FreeBSD can be compiled,too,using the same technique used to build the Linux userland proposed. If you want to talk about containers,why not also add the microkernel topic,that's not a container,but its working can be compared to the microservices offered by Linux and FreeBSD if they run with a microkernel. On Fri, Feb 2, 2024 at 1:30=E2=80=AFPM Alejandro Imass wrote: > > > On Fri, Feb 2, 2024 at 12:40=E2=80=AFAM Mario Marietto > wrote: > >> Excuse me. I'm not very experienced,but isn't the L4 or any other >> microkernel a valid alternative to containers ? If I have understood >> correctly how it works,it allows multiple instances of the various servi= ces >> implemented within the microkernel OS. How many instances can we have ? = For >> example in the L4 Linux kernel webpage it is explained that it can boot >> FreeBSD in cooperation with Linux. It works like xen. WIth xen we can ha= ve >> multiple virtual machines. But xen today has been preferred to kvm. And >> anyway,we always talk about monolithic kernels. So,I want to ask : is a >> microkernel OS a valid alternative to the containers ? If it allows to r= un >> only some services of the "virtualized" os,why not use it ? Why not inve= st >> effort and time to implement this solution as an alternative to the >> containers that FreeBSD already has ? Personally I like the idea of seei= ng >> Linux and FreeBSD work together. >> >> > IMHO, way beyond the scope of this thread. AFAIK neither Linux nor FreeBS= D > are microkernels. > > The original question is more like what would it take to run k8s natively > using different jail managers like CBSD and BastilleBSD as drivers? > Does it already exist? maybe: https://github.com/tnorlin/kubernetes > What is the state? What are the limitations? > etc. etc. > > > > --=20 Mario. --000000000000e5dee20610661905 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Alejandro,I'm not sure if you are aware that the discu= ssion at a certain point has broadened a= nd deepened. I don't understand why I can't propose an alternative = to the actual jail and jail managers that you are talking about. And no. On= the L4 website there is a userland based on new versions of Ubuntu and Deb= ian that can be used with the microkernel offered. They also say that FreeB= SD can be compiled,too,using the same technique used to build the Linux use= rland proposed. If you want to talk about containers,why not also add the m= icrokernel topic,that's not a container,but its working can be compared= to the microservices offered by Linux and FreeBSD if they run with a micro= kernel.


On Fri, Feb 2, 2024 at 1:30=E2=80=AFPM Alejan= dro Imass <aimass@yabarana.com> wrote:


On Fri, Feb 2, 2024 at 12:40=E2=80=AFAM = Mario Marietto <marietto2008@gmail.com> wrote:
Excuse me. I'm not very expe= rienced,but isn't the L4 or any other microkernel a valid alternative t= o containers ? If I have understood correctly how it works,it allows multip= le instances of the various services implemented within the microkernel OS.= How many instances can we have ? For example in the L4 Linux kernel webpag= e it is explained that it can=C2=A0boot FreeBSD in cooperation with Linux. = It works like xen. WIth xen we can have multiple virtual machines. But xen = today has been preferred to kvm. And anyway,we always talk about monolithic= kernels. So,I want to ask : is a microkernel OS a valid alternative to the= containers ? If it allows to run only some services of the "virtualiz= ed" os,why not use it ? Why not invest effort and time to implement th= is solution as an alternative to the containers that FreeBSD already has ? = Personally I like the idea of seeing Linux and FreeBSD work together.
<= /div>


IMHO, way beyond the scope of this thread. AFAIK neither Li= nux nor FreeBSD are microkernels.=C2=A0

The origin= al question is more like what would it take to run k8s natively using diffe= rent jail managers like CBSD and BastilleBSD as drivers?
Does it = already exist? maybe:=C2=A0https://github.com/tnorlin/kubernetes
Wha= t is the state? What are the limitations?
etc. etc.

=C2=A0


--
Mario.
--000000000000e5dee20610661905-- From nobody Fri Feb 2 14:32:49 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TRJCf534qz58s8M for ; Fri, 2 Feb 2024 14:33:02 +0000 (UTC) (envelope-from aimass@yabarana.com) Received: from mail-il1-x131.google.com (mail-il1-x131.google.com [IPv6:2607:f8b0:4864:20::131]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TRJCf3Csdz4G3F for ; Fri, 2 Feb 2024 14:33:02 +0000 (UTC) (envelope-from aimass@yabarana.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-il1-x131.google.com with SMTP id e9e14a558f8ab-363923728aaso9106635ab.0 for ; Fri, 02 Feb 2024 06:33:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yabarana-com.20230601.gappssmtp.com; s=20230601; t=1706884381; x=1707489181; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=7OK6Y4g7/eDWuNrCgRvBEnyEMf2yb9vxYnl2LNXcudo=; b=XCKj9Z08mWb9gB0D01EXiZpSXPvNatzXf5bUfvYBtfUowUVxyWrYDvZMM8U6PTcy9n dvXEvqdPetZfSJt9UqmUTQJxJDvWfLInm8mVW119sM+7RUOAXGcXndEfdlVYYsUpx0te +gEVq//atwHi1xEVSJXmZMPdXt6Sa/SbvlH9+mpMiiMrOzp/4pzgjhe56LKqOT/1/BcS iVmEZ7ZfQg1KcKNvG6K+0bv3rPs1/geJzuJuHJc2Arue2nMIe/epfjXLNNNRyaqlL5dR A1y0FfXvsjBhtXvfj1/k2PzD2D8H2NgTtRsF+dzCslJvispFkrog95RK6PQNRDsZ2J+Z OkQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706884381; x=1707489181; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7OK6Y4g7/eDWuNrCgRvBEnyEMf2yb9vxYnl2LNXcudo=; b=E2pK1jl7MSb0S6dgo82tDMlnNWmu8UFYtbOn0G/9Xq+uBxMyQec8IU48aZr/3CDadH qTzjDh6Jm7xYS4m9EHU84G9B6eRcFBfYmDWmo8NIRq1tgCHpT/tcllUcHt5uW9i0nokE At+oYLZnLrIgAAq59m6QNfVFX7trzW6L2JfhGUMKCYDb9wFOMLE/a41kimZutI0K6VK5 HZUBicXVdeVA6zkFxpNWI1pb2sn6U5m+1I9NpHVYfOlvuQ60IhneehdLg6ILoZePaKba gMXW3YB+0bp3NLzmDGXCoCWGrT6MTfZRPhJzfBDkUnyVOEw4csdRZfvaNGWHHPj0z69w X3cA== X-Gm-Message-State: AOJu0YwobiSNCmcIPKNNdCJVZwwihFyLS65msv8aDtZN2p3d0DFlUBuL eez5Z6qA1GJmau9idotOnISy6+L/Ib/XQinSi34g/jeif9bkS34JUDAOAlNGfxD/R48UNDeX/+v q+TOap7mqxwCYIuXzIl4fjJk7tXOLWzr5bwumBw== X-Google-Smtp-Source: AGHT+IE435S4PZqxzpYk0uQ9RJJUGXqL6yGCybjcBnRIxxuavx+6a9/CcfQriY8+0LZU5ehDvIfaThWTIGc/BCw4yPc= X-Received: by 2002:a92:dacf:0:b0:363:905b:d7e1 with SMTP id o15-20020a92dacf000000b00363905bd7e1mr3333877ilq.13.1706884380965; Fri, 02 Feb 2024 06:33:00 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alejandro Imass Date: Fri, 2 Feb 2024 15:32:49 +0100 Message-ID: Subject: Re: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? To: Mario Marietto Cc: Paul Vixie , FreeBSD virtualization Content-Type: multipart/alternative; boundary="000000000000334dfd061066fdf6" X-Rspamd-Queue-Id: 4TRJCf3Csdz4G3F X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] --000000000000334dfd061066fdf6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Feb 2, 2024 at 2:29=E2=80=AFPM Mario Marietto wrote: > Alejandro,I'm not sure if you are aware that the discussion at a certain > point has broadened and deepened. I don't understand why I can't propose > an alternative to the actual jail and jail managers that you are talking > about. And no. On the L4 website there is a userland based on new version= s > of Ubuntu and Debian that can be used with the microkernel offered. They > also say that FreeBSD can be compiled,too,using the same technique used t= o > build the Linux userland proposed. If you want to talk about containers,w= hy > not also add the microkernel topic,that's not a container,but its working > can be compared to the microservices offered by Linux and FreeBSD if they > run with a microkernel. > > Never said you can't propose it, it just seems off-topic to this thread. Maybe start a new thread on microkernel versus X. This thread is about containers and container configuration, auto-scaling, discovery, updating and monitoring, etc. What I usually do in this case is start a new thread with a new subject and (was: xxx). That way, people interested in that branch of the discussion can continue on its own topic, and the mail archives can do a better job for later research. BTW, I think what you are saying already exists and macOS is exactly that: the FreeBSD core and the mach microkernel. Best, --=20 Alex --000000000000334dfd061066fdf6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Fri, Feb 2, 2024 at 2:29=E2=80=AFP= M Mario Marietto <marietto2008= @gmail.com> wrote:
Aleja= ndro,I'm not sure if you are aware that the discussion at a certain poi= nt has broadened and deepened. I don't un= derstand why I can't propose an alternative to the actual jail and jail= managers that you are talking about. And no. On the L4 website there is a = userland based on new versions of Ubuntu and Debian that can be used with t= he microkernel offered. They also say that FreeBSD can be compiled,too,usin= g the same technique used to build the Linux userland proposed. If you want= to talk about containers,why not also add the microkernel topic,that's= not a container,but its working can be compared to the microservices offer= ed by Linux and FreeBSD if they run with a microkernel.


Never = said you can't propose it, it just seems off-topic to this thread. Mayb= e start a new thread on microkernel versus X.=C2=A0
This thread i= s about containers and container configuration, auto-scaling, discovery, up= dating and monitoring, etc.

What I usually do in t= his case is start a new thread with a new subject and (was: xxx).=C2=A0
That way, people interested in that branch of the discussion can con= tinue on its own topic, and the mail archives can do a better job for later= research.

BTW, I think what you are saying alread= y exists and macOS is exactly that: the FreeBSD core and the mach microkern= el.=C2=A0

Best,

--=C2=A0<= /div>
Alex
--000000000000334dfd061066fdf6-- From nobody Fri Feb 2 14:54:09 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TRJhm0T5Kz58tdB for ; Fri, 2 Feb 2024 14:54:48 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TRJhl5b7pz4JPh for ; Fri, 2 Feb 2024 14:54:47 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x632.google.com with SMTP id a640c23a62f3a-a2f79e79f0cso306259666b.2 for ; Fri, 02 Feb 2024 06:54:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706885686; x=1707490486; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Br6s5t0pxM1y452qRxfxtMTxU9/ugXMazmtNJfLhdFs=; b=I2aGOrvmef/CBodIqXgJzAEuNDkvdizuO/cg+ONZLGnpiqQ7KacBBcepBPgaiT02lb nm9XPF5e5ay9jl1DPfo9PAiS1eomX3POVP1sUZTrji7OZP7vIczDucee9euKu3vLdNak AeYxFwMvn7F0gX9TdRvvOcAYXHiMRVYwpwOS5mClhb60LiHg1D3ctUVjv+MnDDmzkjoi TgpTXxCO8UwEcEREVn1GtwGNA0t1J8vy4v0uR7q4yhra0qDcDT+el0buBsmI9kJCLQhI QZYUi72a1BieXfq2AO77BjgEXNzuXjppQ8glPPIVdKOjLmVZtiEikXxgQ0rafvkzdBvQ JtxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706885686; x=1707490486; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Br6s5t0pxM1y452qRxfxtMTxU9/ugXMazmtNJfLhdFs=; b=uD+NGbFGg79OL4UpyPAdIQa+3c50ykDBb/xI9MtUgUIMjx3eZta2cL8lqo/M+lmrC+ rKJRVEZS65l/L1DvyJcQ/fChqb1azbQC0eWMdlnHIx063Co7mmWQyVPhHmyaHbH9CJq3 QoWOzM08W4fOHA4m/ySrbhmj+/A6/nCpdhqnwYp8ytDyy7/E6emOKMfFbTkUG1WHXSgp Umn5LWoEhu5CkYzKhtMtugg0Cxyb1JYQ5jX3N9WkeoLJ6rSnGQDUUnHGZJ9+nZgTXJIe YAcXNHeKBw8tcxB+lTRKCRtjy9UEK0H+W5fCX2M+wGj6nsAM+eZmf3PgcS+cQThsnhPY 7hWQ== X-Gm-Message-State: AOJu0YxwPpw06STVkuSXt9pc991G/nDd/9VjfmVW0AX0GCU8ds9cDMpq cel/DtnubSxsHzYLA8nTACjHOaX1KZZ94p/aHgt54pvSmAV/SK/U2Ffxo2ZiDPH+pHdy6xi3eVd 3qc0rjapDVnmW72VFu5y5RZo+bjMdpYfM5qnk5zOx X-Google-Smtp-Source: AGHT+IEAuPYa9I6l4o8n7bQoq4vUEbm0a/eH81gzGDntvURn56IgN4INx8iPJKIlO5/PGZ7lF0BrjWLicVpntmLgByc= X-Received: by 2002:a17:907:7802:b0:a37:23e1:63ed with SMTP id la2-20020a170907780200b00a3723e163edmr623740ejc.33.1706885685584; Fri, 02 Feb 2024 06:54:45 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Mario Marietto Date: Fri, 2 Feb 2024 15:54:09 +0100 Message-ID: Subject: Re: If we are so opposed to Docker and Kubernetes, what is the real alternative on BSD? To: Alejandro Imass Cc: Paul Vixie , FreeBSD virtualization Content-Type: multipart/alternative; boundary="000000000000f629e30610674a46" X-Rspamd-Queue-Id: 4TRJhl5b7pz4JPh X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] --000000000000f629e30610674a46 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable ---> BTW, I think what you are saying already exists and macOS is exactly that: the FreeBSD core and the mach microkernel. Please don't compare FreeBSD to MacOS. On Fri, Feb 2, 2024 at 3:33=E2=80=AFPM Alejandro Imass wrote: > > > On Fri, Feb 2, 2024 at 2:29=E2=80=AFPM Mario Marietto > wrote: > >> Alejandro,I'm not sure if you are aware that the discussion at a certain >> point has broadened and deepened. I don't understand why I can't propose >> an alternative to the actual jail and jail managers that you are talking >> about. And no. On the L4 website there is a userland based on new versio= ns >> of Ubuntu and Debian that can be used with the microkernel offered. They >> also say that FreeBSD can be compiled,too,using the same technique used = to >> build the Linux userland proposed. If you want to talk about containers,= why >> not also add the microkernel topic,that's not a container,but its workin= g >> can be compared to the microservices offered by Linux and FreeBSD if the= y >> run with a microkernel. >> >> > Never said you can't propose it, it just seems off-topic to this thread. > Maybe start a new thread on microkernel versus X. > This thread is about containers and container configuration, auto-scaling= , > discovery, updating and monitoring, etc. > > What I usually do in this case is start a new thread with a new subject > and (was: xxx). > That way, people interested in that branch of the discussion can continue > on its own topic, and the mail archives can do a better job for later > research. > > BTW, I think what you are saying already exists and macOS is exactly that= : > the FreeBSD core and the mach microkernel. > > Best, > > -- > Alex > --=20 Mario. --000000000000f629e30610674a46 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
---> BTW, I think what you are saying already exis= ts and macOS is exactly that: the FreeBSD core and the mach microkernel.=C2= =A0

Please don't compare FreeBSD to MacOS.

On Fri, Feb 2, 2024 at 3:33=E2=80=AFPM Alejandro Imass <aimass@yabarana.com> wrote:


On Fri, Feb 2, 2024 at 2:29=E2=80=AFPM Mario Marietto <<= a href=3D"mailto:marietto2008@gmail.com" target=3D"_blank">marietto2008@gma= il.com> wrote:
Alejandro,I'm not sure if you are aware that the= discussion at a certain point has broadened = and deepened. I don't understand why I can't propose an alternative= to the actual jail and jail managers that you are talking about. And no. O= n the L4 website there is a userland based on new versions of Ubuntu and De= bian that can be used with the microkernel offered. They also say that Free= BSD can be compiled,too,using the same technique used to build the Linux us= erland proposed. If you want to talk about containers,why not also add the = microkernel topic,that's not a container,but its working can be compare= d to the microservices offered by Linux and FreeBSD if they run with a micr= okernel.


Never said you can't propose it, it just seems of= f-topic to this thread. Maybe start a new thread on microkernel versus X.= =C2=A0
This thread is about containers and container configuratio= n, auto-scaling, discovery, updating and monitoring, etc.

What I usually do in this case is start a new thread with a new sub= ject and (was: xxx).=C2=A0
That way, people interested in that br= anch of the discussion can continue on its own topic, and the mail archives= can do a better job for later research.

BTW, I th= ink what you are saying already exists and macOS is exactly that: the FreeB= SD core and the mach microkernel.=C2=A0

Best,

--=C2=A0
Alex


--
Mario.
--000000000000f629e30610674a46-- From nobody Fri Feb 2 20:20:46 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TRRwt4Wprz58C1j for ; Fri, 2 Feb 2024 20:20:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TRRwt3TLpz4xd0 for ; Fri, 2 Feb 2024 20:20:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706905246; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5YqSIZ1z7w+H9dvlrE+rlNhCYUIMamoYj0NbUOKjVWc=; b=hHX8Cz4HgG6e/LwYU03yCzsOhRiK+52nv13Zt4bdrnUGWfKmpQCPBXNLiiwaYKWV9u6g02 uys2HnhjEc7THb6Og4sQXEHBTww3gE/c8kJhqVcbPALDzC2Cy5MU8n2DQhmLc4PaaKD3oF L1qszdBVfAAFVnHo8eleqh2d09zIX8riLp1ESSmq9SbJNISpuN6Nw9vQ6bYY2I1dcweJEu 9CUg5gParkjSrrMm202uR985Sla9wYbirDigZ/mcB6OEVgeELccHNl5SDEYtkepj47WBML wHbXqhF0YaVvk7TBJgSvnm/nsYuGTzm4ih3Ti1EmHaGpLmqcqz8SfT4suIrZ0A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706905246; a=rsa-sha256; cv=none; b=Zs6m4XZA5To6qiP8cLC5t+cPIsX4a0KBmmZ3emb2dphL2OG8Fm2XKcOWag/NHQy9fgCfCz /ngo5Py2jvfnkW/e74vRHQIIwsvMqLj2NVVCQaChWsQuSyNhzgzXGjT6mgoM6YOswA6Xht yKRuLlc02hZBXvDmv5w8rDzT7oBgSq2vuLeTkxEYmHJXDaE24vcmlSiDQEu2c62qHe4JDu cP3JxNoegdvZx05DAFLWkL5Y0i7knpJAUjU69N2bIBLIrC3lrWBiJp5h7JRigk6hj1JGIb Ao5Kaf9L9aoYb///StCqLcKBs2V5QcW9MBcqkc6Zp2BktG5mdirnwkoAW8WMHQ== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TRRwt2YvdzkqR for ; Fri, 2 Feb 2024 20:20:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 412KKkvI016443 for ; Fri, 2 Feb 2024 20:20:46 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 412KKkPU016436 for virtualization@FreeBSD.org; Fri, 2 Feb 2024 20:20:46 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 273372] SR-IOV Networking in Bhyve Causes Chelsio T520-SO-CR to Fail on Host, Kernel Panic if Reset Date: Fri, 02 Feb 2024 20:20:46 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 14.0-STABLE X-Bugzilla-Keywords: bhyve, crash X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: mark@markmcb.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D273372 --- Comment #10 from Mark McBride --- Problem still exists. Adding this link as it is a very detailed walk-throug= h of exactly what I'm doing and the hardware I'm using: https://markmcb.com/freebsd/vs_linux/sriov_is_first_class/ --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Sun Feb 4 00:34:06 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TS9Vk6Mbvz58mjj for ; Sun, 4 Feb 2024 00:34:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TS9Vk3qdvz4VTT for ; Sun, 4 Feb 2024 00:34:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707006846; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=M4DZPQxX4qiAYsrglH6W5pfKyGCumV/9iJ+mlpzKQDw=; b=sDm8LujppXnOIyYOYUDFBEfVqrt6aQFBBUasZLWSEw7EtZyvBDReG4WWuesOSZ2AVMEYhw cs+blP07k6wIOn78pUSSGQ0a3vR7e1wCR+DSh9dKrNUgwP4B+L1t0yXa/l4zqC1gpoC0oJ dTsZnFhG2p1UlZQy5dhFjo6+spPNsySznIZrGrMh1/sWCMxiSDC8VW00kHNfxFvwkRhifT iOR6v67N3Q0XJ7olEgOlOC9V26inO3psGQ6/kidBoODXjbrtDsS42dOTQGh+W4SQFWHWy+ M0rd5SiHa171rfvnJQmA8Bx5uLBFPyJnIK2EQm+r5xz89ET5vz4MC6R4/8Bdsw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1707006846; a=rsa-sha256; cv=none; b=hqAqG4mlzZf6imXn0D9EAKnL8ADH2AMlbkYlwxFY/i3ZkVryjQI3hUHhRbKw4MKziRkwDb 9+TysdKVxhDUNf1Nke/6A7gGi94iYVP6sHBytHV70kcRFG8Jlfh2pFCceKV7gchsl4NynT 7jlgEIMRmKA2wq8uqYDzDIxhjFF4AJtiOdOOtfD0SLm/BXtCub0kgI5+QGVXFjIQ69n945 Ho+XtM6xvw8jryKI/rfyOWB1+HLQhW/DDB/pQIb5xiaxq+TvsghbnUa5dF5lUASpk8/p9I 2hKV4W4y2IGKn9rVmbgCcM0UxZuQoC667gkAOdax7szRooS5FiL79X3aI1xF4g== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TS9Vk2t5VzbwG for ; Sun, 4 Feb 2024 00:34:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 4140Y679045868 for ; Sun, 4 Feb 2024 00:34:06 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 4140Y6Lt045867 for virtualization@FreeBSD.org; Sun, 4 Feb 2024 00:34:06 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 276815] loader.efi: failure to load a module causes general protection fault Date: Sun, 04 Feb 2024 00:34:06 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: lexi.freebsd@le-fay.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276815 Bug ID: 276815 Summary: loader.efi: failure to load a module causes general protection fault Product: Base System Version: CURRENT Hardware: amd64 OS: Any Status: New Severity: Affects Only Me Priority: --- Component: bhyve Assignee: virtualization@FreeBSD.org Reporter: lexi.freebsd@le-fay.org using src 49619f73151aeaca4cef5adf631253da04a46e19=20 i built a new kernel for a bhyve VM on a 14.0 host. i forgot to include a necessary module in the kernel, so it failed to load. however, rather than drop to the loader prompt, the loader produced a fatal exception: Loading kernel... /boot/kernel/kernel text=3D0x963f0 text=3D0x38eed8 text=3D0x7b974 data=3D0x= 140 data=3D0x8c780+0x172880 0x8+0x9db78+0x8+0xbb463| Loading configured modules... /boot/kernel/zfs.ko size 0x5e8828 at 0xb59000 loading required module 'crypto' /boot/kernel/crypto.ko size 0x5e338 at 0x1142000 loading required module 'acl_nfs4' can't find 'acl_nfs4' !!!! X64 Exception Type - 0D(#GP - General Protection) CPU Apic ID - 00000= 000 !!!! ExceptionData - 0000000000000000 RIP - 00000000BE2B818A, CS - 0000000000000038, RFLAGS - 0000000000010246 RAX - 000000000000002F, RCX - 0000000000000000, RDX - 000000000000006B RBX - 0000000000000000, RSP - 00000000BFF40200, RBP - 00000000BFF40200 RSI - 4000000000000080, RDI - 00000000B9A6B940 R8 - 0000000000000018, R9 - FFFFFFFFFFFFFFFD, R10 - 656B2F746F6F622F R11 - 0000000000000140, R12 - 0000000000000000, R13 - 0000000000000000 R14 - 00000000B9A6CE70, R15 - 00000000B9A6B940 DS - 0000000000000030, ES - 0000000000000030, FS - 0000000000000030 GS - 0000000000000030, SS - 0000000000000030 CR0 - 0000000080010033, CR2 - 0000000000000000, CR3 - 00000000BFC01000 CR4 - 0000000000000668, CR8 - 0000000000000000 DR0 - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000 DR3 - 0000000000000000, DR6 - 00000000FFFF0FF0, DR7 - 0000000000000400 GDTR - 00000000BFBDE000 0000000000000047, LDTR - 0000000000000000 IDTR - 00000000BF1D4018 0000000000000FFF, TR - 0000000000000000 FXSAVE_STATE - 00000000BFF3FE60 !!!! Find image based on IP(0xBE2B818A) (No PDB) (ImageBase=3D00000000BE27= 6000, EntryPoint=3D00000000BE2800D0) !!!! bhyve command line: Feb 04 00:23:32: booting Feb 04 00:23:32: [bhyve options: -c 8 -m 4g -Hwl bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -U 4701bec6-c139-11ee-9702-9c6b0006ba72 -u] Feb 04 00:23:32: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,nvme,/data/vm/lily/disk0.img -s 5:0,nvme,/data/vm/lily/disk1.img -s 6:0,virtio-net,tap1,mac=3D58:9c:fc:0e:e8:95] Feb 04 00:23:32: [bhyve console: -l com1,stdio] --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Sun Feb 4 20:27:41 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TSgzx1pSMz58kgf for ; Sun, 4 Feb 2024 20:27:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TSgzw6WSLz4CQR for ; Sun, 4 Feb 2024 20:27:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707078460; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3SEfue/qr6lNiPAgnzH1EgNStFsNoTbkrynLhO2qkBM=; b=svQjdTAXQnQUWtOpx8L1OqsRhXnDBsFJqA7BHGCb+ZBgK8UIleDwRzZpprqibbIGgusdXf 5dSMMs6guXQPDRBZHzYFUsj2sEw621XTWAq9DytrX2//b21d4eIYkzzBHdJvI78lbIOz1k 8SV8nUJoaPqtkaT/h0wNzahGDpIy/4cfmRbaSw5AOvtedTu5FfsQrG9CauXJo9Wy4uOVpD Jwg2/sEdVBUD463tTXRs9LmidQ+TXsy9k0hV9jWqlCDPJij3zHRnjB7vuU10IienLhmTDO FT65XoLpiaoQXwZ2HtXPjMCvf0QInpOzxLCaYUGrBMcZ6gm7uy4PJn54sBn9ZA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1707078460; a=rsa-sha256; cv=none; b=xpssFnA8ZsPxH+huWskt9ZpCQdf9X1cZLMLZmZKE2uVOct/FAb0hf0BUqEg4y4Zsz1R7Zc rfgHs1DORy5Ykd1gD8f0hC1Aa2lmd7gKacBFax/Duc9v7ESvzeHYwF6N5O/RLQl0DerKqf Yp24xjoqQP36RHQ+64Cx+eWUo0dPnCetXGZsvQq7tyB81wNEZJbwVHIOhSsvuf2Ayu7QDR t/x5Hcogt9ATiZvzoV85DWViGHtG6CralkNUqcT6e/+9yv1aqddgkSdP0FKsHVr1q5W83a KiobGC8S7CH9v+QwQX4DXNhMvKiDg63HHp/Z3JVjUxFehvhSdzBBzsiT/g+Wsg== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TSgzw5ZH5z1B3K for ; Sun, 4 Feb 2024 20:27:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 414KReXK065505 for ; Sun, 4 Feb 2024 20:27:40 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 414KReb8065504 for virtualization@FreeBSD.org; Sun, 4 Feb 2024 20:27:40 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 276734] bhyve gets stuck during VM boot Date: Sun, 04 Feb 2024 20:27:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 14.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: markj@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D276734 Mark Johnston changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |markj@FreeBSD.org Status|New |Open --- Comment #2 from Mark Johnston --- 67937 117360 bhyve - mi_switch _sleep vm_wait_doms vm_wait_domain vm_page_alloc_noobj_domain uma_small_alloc keg_alloc_slab zone_import zone_alloc_item malloc amdvi_update_mapping iommu_create_mapping vm_iommu_modify vm_assign_pptdev vmmdev_ioctl devfs_io= ctl vn_ioctl devfs_ioctl_f This means that one of the bhyve threads is stuck in the kernel waiting for memory. Is this running on a NUMA system? --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Sun Feb 4 21:00:08 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TShjP4KCHz58nfJ for ; Sun, 4 Feb 2024 21:00:09 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TShjP0dl8z4Hnq for ; Sun, 4 Feb 2024 21:00:09 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707080409; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=n+GV/elyU/QkCgLxymdaGnW6ls2AZunRH7KpO6Rn8hc=; b=XjLQE+K1ZBTts3cx+f9dbiv5yR6D9c2nChez145cCnA9jayu0U2mHWnyRN5RXlktf1VCYw tG0iVHbBykhmXsJpKJfliHYQUmD6Zf1rKbFjO3GieNxIe91jDkw7VuE/auKF2foSDEQh94 17F55Pd72QKcV3r1Z9iMAQc3il1dojbY5crNouydcHWO/HvdNBS+VdBfI+9t+OzSaue/kE 6DL0oMhpSFeA3FV0zwSY91j15XOK+ZNes5kxZ1/pFwOfsnNRGSptxXL1LC9rqOKCKQHNwb 8S0eoTaHocZvjsk9mx+/fsaVzm+lLxlX5OqitPsCl0cnIss46q9WAEjfbJbBfg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1707080409; a=rsa-sha256; cv=none; b=Zo7VnR0LF03pnAgABau1XFkFoV/7EBZH9pAR+Agjt/A5BjEZodcjnOIQ3eecorQfRMEnZ0 DILIP5jNEAdBDeTDgDDKvlLHd8DSR78afpuzqIYdfOrl8gFq1Sg/JpEVOLIc85KE4PF4SM xJDfvioFizAB1BGbUSdr0qnf1ljO+jYT2cNwbXVhrbYrJl+P7+PKScof0eal7VIleafo6q otAOyqeXJ9VS4lKdtNbnM2EUQx65HtuFyVmQxi6bZBY2ZAJ1l9ipcQUanFixIxOoriyBq4 YGNI53s9WEiejGNZJYJBX/+KlTzMMLvhqBnIjy3IuoSGK3mAoJVSiCpK9cTuqg== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TShjN6rGcz1Bp7 for ; Sun, 4 Feb 2024 21:00:08 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 414L08m5027592 for ; Sun, 4 Feb 2024 21:00:08 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 414L08GU027591 for virtualization@FreeBSD.org; Sun, 4 Feb 2024 21:00:08 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202402042100.414L08GU027591@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: virtualization@FreeBSD.org Subject: Problem reports for virtualization@FreeBSD.org that need special attention Date: Sun, 4 Feb 2024 21:00:08 +0000 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="17070804088.5cf6d.24290" Content-Transfer-Encoding: 7bit --17070804088.5cf6d.24290 Date: Sun, 4 Feb 2024 21:00:08 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 264253 | hyperv: ifconfig media autoselect on FreeBSD 13.0 Open | 264267 | UEFI Booting on Azure Generation 2 VMs crashes 2 problems total for which you should take action. --17070804088.5cf6d.24290 Date: Sun, 4 Feb 2024 21:00:08 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" 
The following is a listing of current problems submitted by FreeBSD users,
which need special attention. These represent problem reports covering
all versions including experimental development code and obsolete releases.

Status      |    Bug Id | Description
------------+-----------+---------------------------------------------------
Open        |    264253 | hyperv: ifconfig media autoselect on FreeBSD 13.0
Open        |    264267 | UEFI Booting on Azure Generation 2 VMs crashes

2 problems total for which you should take action.
--17070804088.5cf6d.24290--