From owner-freebsd-hackers@freebsd.org Sun May 16 04:05:40 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 93591633BF6 for ; Sun, 16 May 2021 04:05:40 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4FjTFc0zy0z3krm for ; Sun, 16 May 2021 04:05:40 +0000 (UTC) (envelope-from ian@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 21B16633C1F; Sun, 16 May 2021 04:05:40 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 217A5633971 for ; Sun, 16 May 2021 04:05:40 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound5c.ore.mailhop.org (outbound5c.ore.mailhop.org [54.244.192.240]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FjTFb4W1wz3kmn for ; Sun, 16 May 2021 04:05:39 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1621137938; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=v67KOuqowaliV+ckFsNHI/cQl/aNIlnV2P5sQjou0VkQ5JNEQFS0cHruknqC2Mwjg9Qd6Gi3Zwxol d/NBAhR6kysqPNVkF/1RpaCstRmfLR5NOKQhSCbYt9ZeGLyry1Wb9bxoQhTbT8cOJxDgjtKlTeeWBK FdSfUUOTf6w3eIx+oMeeqXup37wBOl9bqwwGB0N/NL78OlfHs7lxI6DLOTpCocsfIzKPOf9rLWAp1C MX7ruQ/TNQHwlt4w1tbipmDDVkZSZqh4rO081GQfG4OdV8TMx6j/jWJRVVokxV3ZXDY9HCM4UayF0H 94WwCaqeUXouv8yAC4SM+umq2b6s0Rw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:dkim-signature:from; bh=BWeNhiyABlPUCwvxeK007QNbyBrqkKPKy5RNkZ1x2Us=; b=MYojLej14oiwkZWZw+KhPMioluzgBBhQEGs5ZT78Er5WQ0WhjdhcSFcSY2ntjXGnwDrD44gy38AmU lUEAxKQka2ZRzC3TN0XXeLnUUR3ZBlM4Jpi/LXJN0M8Oy+qXnOF+9g5Gpzemt8NlF9HgRxrjD1VWKL 3q+zhjrKF26Cp+LIWh406pAG1YWRs+uUlGpeBdWu8trKOXwAfEK89pcS+Aa7sOMRMi9jPTyrRg5umI tHqoUhDK3dMZ5aB+Kw56CBFY3IJZvFp5vYV7DXyhD2kR393E2X59Px3brOQnfk3k67au05FiWqWLO0 JVnv8FZjm04nVvU8e1e/F2jFVfvz7Wg== ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:from; bh=BWeNhiyABlPUCwvxeK007QNbyBrqkKPKy5RNkZ1x2Us=; b=JcUDGVvOVkBM1TNiy6HhWuiyTUwTP1Ksnc8GHvOtCKmxMM5MoPFrfSWHmzKP2HbLh1vIx6fqnGa3E u23IBn0OTNtOxAzLg9/8aomr3WtzKIYK35ZW2iA2JpwkhqUvvLPBPxfPyynbRjwmRpl//ZVlP/pbsr boYUbwf7a0oeVH9V7L+NBpBRFFczJDo9f36g4rbHp/YH9Aq0oQ+v07QTuEdVU+NRRbE6F4FDNf+M6c h01y3tpwFODwQ+34lJI6Ato4jUqKPBxKTG2XjycViJ2AkdiaaPIcXVSfGUCuKLrcsKkNh5IXpC2sUN eOY3zJW79JIGw6PbvTOQ11p7aQf+xxw== X-Originating-IP: 67.177.211.60 X-MHO-RoutePath: aGlwcGll X-MHO-User: f6aa57f0-b5fb-11eb-8406-bf9d68d023b6 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (c-67-177-211-60.hsd1.co.comcast.net [67.177.211.60]) by outbound3.ore.mailhop.org (Halon) with ESMTPSA id f6aa57f0-b5fb-11eb-8406-bf9d68d023b6; Sun, 16 May 2021 04:05:37 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id 14G45Xup062390; Sat, 15 May 2021 22:05:34 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <8d232feb354bf404426c51efe4953551ae2476c9.camel@freebsd.org> Subject: Re: patch: make i2c(8) usable for scripting From: Ian Lepore To: Poul-Henning Kamp , hackers@freebsd.org Date: Sat, 15 May 2021 22:05:33 -0600 In-Reply-To: <12056.1621109641@critter.freebsd.dk> References: <12056.1621109641@critter.freebsd.dk> Content-Type: text/plain; charset="ASCII" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4FjTFb4W1wz3kmn X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2021 04:05:40 -0000 On Sat, 2021-05-15 at 20:14 +0000, Poul-Henning Kamp wrote: > The /dev/iic%d filedescriptor is opened and closed for every single > command, in the hope that it will aid multiple separate programs > sharing I2C busses. I have not actually tried that yet, so it may > need more work (exclusive opens etc.). There should be no need to do that. The i2c bus is arbitrated in iicbus/iiconf.c and most all the in-tree drivers now use either explicit iicbus_request_bus() calls, or use iicbus_transfer_excl(). Certainly users of iic(4) don't need to be protected from each other; everything in the driver that uses the bus does so inside of request_bus / release_bus calls. -- Ian From owner-freebsd-hackers@freebsd.org Sun May 16 06:51:33 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C843163792A for ; Sun, 16 May 2021 06:51:33 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4FjXx14RFgz4YBC for ; Sun, 16 May 2021 06:51:33 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: by mailman.nyi.freebsd.org (Postfix) id 9836963763B; Sun, 16 May 2021 06:51:33 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 97F82637929 for ; Sun, 16 May 2021 06:51:33 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FjXx1349Hz4Y5Z; Sun, 16 May 2021 06:51:32 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (v-critter.freebsd.dk [192.168.55.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by phk.freebsd.dk (Postfix) with ESMTPS id CAF6D89290; Sun, 16 May 2021 06:51:30 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.16.1/8.16.1) with ESMTPS id 14G6pUtT013974 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sun, 16 May 2021 06:51:30 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.16.1/8.16.1/Submit) id 14G6pUS5013973; Sun, 16 May 2021 06:51:30 GMT (envelope-from phk) To: Ian Lepore cc: hackers@freebsd.org Subject: Re: patch: make i2c(8) usable for scripting In-reply-to: <8d232feb354bf404426c51efe4953551ae2476c9.camel@freebsd.org> From: "Poul-Henning Kamp" References: <12056.1621109641@critter.freebsd.dk> <8d232feb354bf404426c51efe4953551ae2476c9.camel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <13971.1621147890.1@critter.freebsd.dk> Content-Transfer-Encoding: quoted-printable Date: Sun, 16 May 2021 06:51:30 +0000 Message-ID: <13972.1621147890@critter.freebsd.dk> X-Rspamd-Queue-Id: 4FjXx1349Hz4Y5Z X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2021 06:51:33 -0000 -------- Ian Lepore writes: > On Sat, 2021-05-15 at 20:14 +0000, Poul-Henning Kamp wrote: > > The /dev/iic%d filedescriptor is opened and closed for every single > > command, in the hope that it will aid multiple separate programs > > sharing I2C busses. I have not actually tried that yet, so it may > > need more work (exclusive opens etc.). > > There should be no need to do that. The i2c bus is arbitrated in > iicbus/iiconf.c and most all the in-tree drivers now use either > explicit iicbus_request_bus() calls, or use iicbus_transfer_excl(). = > Certainly users of iic(4) don't need to be protected from each other; > everything in the driver that uses the bus does so inside of > request_bus / release_bus calls. How does that work when i2c(8) (by default) issues discrete start/stop com= mands ? Related to that: Should i2c(8)'s -m default be changed to 'tr' ? -- = Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe = Never attribute to malice what can adequately be explained by incompetence= . From owner-freebsd-hackers@freebsd.org Sun May 16 11:29:04 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DAF5C63E889 for ; Sun, 16 May 2021 11:29:04 +0000 (UTC) (envelope-from markm@FreeBSD.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Fjg5D5pSBz3MT3 for ; Sun, 16 May 2021 11:29:04 +0000 (UTC) (envelope-from markm@FreeBSD.org) Received: by mailman.nyi.freebsd.org (Postfix) id C73C763E888; Sun, 16 May 2021 11:29:04 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C704C63E5FD for ; Sun, 16 May 2021 11:29:04 +0000 (UTC) (envelope-from markm@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fjg5D5HdSz3MKy; Sun, 16 May 2021 11:29:04 +0000 (UTC) (envelope-from markm@FreeBSD.org) Received: from smtpclient.apple (unknown [IPv6:2a02:8011:300b:42:d9f8:503e:4e8a:3be2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: markm) by smtp.freebsd.org (Postfix) with ESMTPSA id 37354A4CD; Sun, 16 May 2021 11:29:04 +0000 (UTC) (envelope-from markm@FreeBSD.org) Content-Type: multipart/signed; boundary="Apple-Mail=_0B802C0C-624D-4CA7-81F7-1D15902E1450"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.80.0.2.43\)) Subject: Re: patch: make i2c(8) usable for scripting From: Mark Murray In-Reply-To: <12056.1621109641@critter.freebsd.dk> Date: Sun, 16 May 2021 12:29:01 +0100 Cc: hackers@freebsd.org Message-Id: <5F4DC788-99A8-49D7-939E-3353823C5425@FreeBSD.org> References: <12056.1621109641@critter.freebsd.dk> To: Poul-Henning Kamp X-Mailer: Apple Mail (2.3654.80.0.2.43) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2021 11:29:04 -0000 --Apple-Mail=_0B802C0C-624D-4CA7-81F7-1D15902E1450 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 15 May 2021, at 21:14, Poul-Henning Kamp wrote: > These modes always use the I2CRDWR ioctl, which works on all the = hardware I have. Please could you list what you have, for the record? Thanks! M -- Mark R V Murray --Apple-Mail=_0B802C0C-624D-4CA7-81F7-1D15902E1450 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.2 Comment: GPGTools - http://gpgtools.org iQEzBAEBCgAdFiEEyzPHvybPbOpU9MCxQlsJDh9CUqAFAmChAf0ACgkQQlsJDh9C UqAfVQgApT/DsuBXFltDRqzJgrGFETl2RxuaOj7z8iVKTUTuMyFFhgFiC27ROEk+ jMFOSO2qK+JQTUWTuzc/wg/54xtzDqHA9Wqk9IkLargSttCw3vuMnC9EP+8tVIA1 6ZZFolXM1A6QU8WicvbM5iRDlhZlBONo5SFyo05gaWSFUyxsezCoit9LR62SUDXY ueHfXCcB8Svd/EiYqjSthmy1gzbmtgeSAwEfYGvNaZhyPojPORAMFPnnwkddMt/D VS3qT07w+w4QIWU3VacnzTryltZ9tnHCbvOjB9gwpKx3GoQhgIo5kKpkEiuPOZ2g ICt/A6MQWGjYoJKzuggVrdsJtFfzQQ== =dNvA -----END PGP SIGNATURE----- --Apple-Mail=_0B802C0C-624D-4CA7-81F7-1D15902E1450-- From owner-freebsd-hackers@freebsd.org Sun May 16 12:53:05 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6FA4D6413C8 for ; Sun, 16 May 2021 12:53:05 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Fjhy91N6Kz3mmn for ; Sun, 16 May 2021 12:53:05 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: by mailman.nyi.freebsd.org (Postfix) id 2F2CD64128B; Sun, 16 May 2021 12:53:05 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2EF61641243 for ; Sun, 16 May 2021 12:53:05 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fjhy9068pz3mfj; Sun, 16 May 2021 12:53:04 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (v-critter.freebsd.dk [192.168.55.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by phk.freebsd.dk (Postfix) with ESMTPS id BB3C889290; Sun, 16 May 2021 12:53:02 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.16.1/8.16.1) with ESMTPS id 14GCr2Ih002562 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sun, 16 May 2021 12:53:02 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.16.1/8.16.1/Submit) id 14GCr2Nx002561; Sun, 16 May 2021 12:53:02 GMT (envelope-from phk) To: Mark Murray cc: hackers@freebsd.org Subject: Re: patch: make i2c(8) usable for scripting In-reply-to: <5F4DC788-99A8-49D7-939E-3353823C5425@FreeBSD.org> From: "Poul-Henning Kamp" References: <12056.1621109641@critter.freebsd.dk> <5F4DC788-99A8-49D7-939E-3353823C5425@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <2559.1621169582.1@critter.freebsd.dk> Date: Sun, 16 May 2021 12:53:02 +0000 Message-ID: <2560.1621169582@critter.freebsd.dk> X-Rspamd-Queue-Id: 4Fjhy9068pz3mfj X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2021 12:53:05 -0000 -------- Mark Murray writes: > On 15 May 2021, at 21:14, Poul-Henning Kamp wrote: > > These modes always use the I2CRDWR ioctl, which works on all the > hardware I have. > > Please could you list what you have, for the record? Rpi4 & BBB -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-hackers@freebsd.org Sun May 16 13:18:11 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EFD496417B4 for ; Sun, 16 May 2021 13:18:11 +0000 (UTC) (envelope-from markm@FreeBSD.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4FjjW75Jc2z3pmg for ; Sun, 16 May 2021 13:18:11 +0000 (UTC) (envelope-from markm@FreeBSD.org) Received: by mailman.nyi.freebsd.org (Postfix) id B4541641836; Sun, 16 May 2021 13:18:11 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B41E1641999 for ; Sun, 16 May 2021 13:18:11 +0000 (UTC) (envelope-from markm@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FjjW74krgz3q1M; Sun, 16 May 2021 13:18:11 +0000 (UTC) (envelope-from markm@FreeBSD.org) Received: from smtpclient.apple (unknown [IPv6:2a02:8011:300b:42:d9f8:503e:4e8a:3be2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: markm) by smtp.freebsd.org (Postfix) with ESMTPSA id 1E59EBC9B; Sun, 16 May 2021 13:18:11 +0000 (UTC) (envelope-from markm@FreeBSD.org) Content-Type: multipart/signed; boundary="Apple-Mail=_5605A8EC-1E90-4B2C-B9A5-F5BBD80B6650"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.80.0.2.43\)) Subject: Re: patch: make i2c(8) usable for scripting From: Mark Murray In-Reply-To: <2560.1621169582@critter.freebsd.dk> Date: Sun, 16 May 2021 14:18:07 +0100 Cc: hackers@freebsd.org Message-Id: <350E25AE-180C-4F13-B75B-32711225F7AA@FreeBSD.org> References: <12056.1621109641@critter.freebsd.dk> <5F4DC788-99A8-49D7-939E-3353823C5425@FreeBSD.org> <2560.1621169582@critter.freebsd.dk> To: Poul-Henning Kamp X-Mailer: Apple Mail (2.3654.80.0.2.43) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2021 13:18:12 -0000 --Apple-Mail=_5605A8EC-1E90-4B2C-B9A5-F5BBD80B6650 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On 16 May 2021, at 13:53, Poul-Henning Kamp = wrote: >=20 > -------- > Mark Murray writes: >=20 >=20 >> On 15 May 2021, at 21:14, Poul-Henning Kamp = wrote: >>> These modes always use the I2CRDWR ioctl, which works on all the >> hardware I have. >>=20 >> Please could you list what you have, for the record? >=20 > Rpi4 & BBB Thank you! I had to jump through hoops to get I2C to work at all on RPi4 = 8GB. What was your method for getting this right? At some point, when I have enough circular to-its, I want to attend to = the differences between BCM2835 and BCM2711, the list of which is = growing. M -- Mark R V Murray --Apple-Mail=_5605A8EC-1E90-4B2C-B9A5-F5BBD80B6650 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.2 Comment: GPGTools - http://gpgtools.org iQEzBAEBCgAdFiEEyzPHvybPbOpU9MCxQlsJDh9CUqAFAmChG48ACgkQQlsJDh9C UqAcPQf+LMtn2WANZPNbND991SQ9iUQ8O//WhLiO+ld+PjV1B+oCnWtRFmKmr0YU WgD6SsmJPem7WJJkzaEzGFOpeoeS/EAU6hULTMNimKCBcX8QnkhswZafrqfOvU5w fQuDj7BTGTyX3Jvnm8CjExorGt+JjszJPiCV7544zz8DCWAt5uRrfZGMB/Pcpprh fs1OiWcr16OyznP01RmtxQLZZzmIJkFMG3LN8UkGX0eayv83hG6QRMNEjmhIGh1+ HKBRDkiiDXe3sQ4A8LglIi9/nv8vHZIqBLoiFZvohB1nkfMorb9BJqUemZ753ChB 9AgBzNEkwWCe+aXyXjj/gXmayVXFEA== =o3T9 -----END PGP SIGNATURE----- --Apple-Mail=_5605A8EC-1E90-4B2C-B9A5-F5BBD80B6650-- From owner-freebsd-hackers@freebsd.org Sun May 16 16:43:18 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B14ED6287B8 for ; Sun, 16 May 2021 16:43:18 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Fjp3p4Fjnz4llN for ; Sun, 16 May 2021 16:43:18 +0000 (UTC) (envelope-from ian@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 91DD462855A; Sun, 16 May 2021 16:43:18 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 919F36287B6 for ; Sun, 16 May 2021 16:43:18 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound5c.ore.mailhop.org (outbound5c.ore.mailhop.org [54.244.192.240]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fjp3p2SCNz4lyc for ; Sun, 16 May 2021 16:43:18 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1621183397; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=ctHdaX/L9JcZUxsNxD7xeaBDMJClAT1cndXYembmeD1ZYIAY72byr5ISB42NdmlhVL1pJSo5STfmU gQzRVCeigWpXvTpU4y1X4+fQd7ItNvOm6FuxLF8234SBVWJCxaLn7i8uSdR2j+rZMWKE9Ih3JC/daS poueKN7JywxnwV9sIdVNlZTVjF1xgGpct/hMSI0ufPvSq32CsjclsaDVBid2Ocu5CCxTqy2g2fYTD1 28H/Xk410g0lmhk9UrUbF7ee9aPQdxe+AUSsjBaPqsp5ubDAEPr+sdw4fF45O8afRyY8mlk+LUWWQc 5T8gS5JrP6ACMFkzOK9v23F1IJ7qlrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=KipnR5+uWqdG8G7fxxoEhMq/ysDpZRebE6zxp9Kd7bw=; b=ZcE83wx6VjmXzMShmvTVmcet6+8ATCZ/6QjozGHZPLcd24MoX+Kou+f6BCbnHWTz0+p89I9jRuM/J sIWQdN+kIBuqbleR6W8uuQ2LcVor8VUfZoh0DxhEeMkJ1UfTNBJQJqcysXqvgJNzI7k+7MDz8dSQlm gL/AOMHYh8q0EsTuFcian3q+yaq29BnFuIi63uciPgsUBiswYisyssdHyeEaXOwg4kC/3dsIcbrlu4 hIRaNpR9OgZO715hIMA1N+nLFSXVBC8P9na7JEs1Al0PdIkCtLal3+rWErKHCoLH33FX9ymbj5899h /vSKzCThLIyEHDFJiAGR9+m9Rrtmo3A== ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=KipnR5+uWqdG8G7fxxoEhMq/ysDpZRebE6zxp9Kd7bw=; b=AmiJOVEPxVFKKEfZh62ILphDDRHt4NVfQjTL63qsydN986YbRgxT51UgM4DnK5Ls0XhUJfvpGgNid SaRqggF3q7k+2mYbGiX9nlrlY3hTgfZ2IABxjHid3KO/dGaEVGZwl+7l/4ruuWrTrPaXkrSGQLn3/e VdJ34USJ0+uDsYZrIUw+GVQVLymy8K8DpWn+0c8qZO3RGfQofKcGF6e6x3HFHWFVK4Wqqu+LJNuLGP I6WRXLfuFQCtXX1FnjJez7yGTzk93DwvdFKtOxp7SxHFNDDbXj3QM9UqGVfMJahjggGdDAXYn7RHup RrUI0j1kUGpHvuHiIg5nx0nuJzDHxTw== X-Originating-IP: 67.177.211.60 X-MHO-RoutePath: aGlwcGll X-MHO-User: cec407d5-b665-11eb-841b-bf9d68d023b6 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (c-67-177-211-60.hsd1.co.comcast.net [67.177.211.60]) by outbound3.ore.mailhop.org (Halon) with ESMTPSA id cec407d5-b665-11eb-841b-bf9d68d023b6; Sun, 16 May 2021 16:43:16 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id 14GGhDiE064418; Sun, 16 May 2021 10:43:14 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <058b3056a0adf05ac6e124a989b9b8250ff36ce3.camel@freebsd.org> Subject: Re: patch: make i2c(8) usable for scripting From: Ian Lepore To: Poul-Henning Kamp Cc: hackers@freebsd.org Date: Sun, 16 May 2021 10:43:13 -0600 In-Reply-To: <13972.1621147890@critter.freebsd.dk> References: <12056.1621109641@critter.freebsd.dk> <8d232feb354bf404426c51efe4953551ae2476c9.camel@freebsd.org> <13972.1621147890@critter.freebsd.dk> Content-Type: text/plain; charset="ASCII" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Fjp3p2SCNz4lyc X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2021 16:43:18 -0000 On Sun, 2021-05-16 at 06:51 +0000, Poul-Henning Kamp wrote: > -------- > Ian Lepore writes: > > > On Sat, 2021-05-15 at 20:14 +0000, Poul-Henning Kamp wrote: > > > The /dev/iic%d filedescriptor is opened and closed for every > > > single > > > command, in the hope that it will aid multiple separate programs > > > sharing I2C busses. I have not actually tried that yet, so it > > > may > > > need more work (exclusive opens etc.). > > > > There should be no need to do that. The i2c bus is arbitrated in > > iicbus/iiconf.c and most all the in-tree drivers now use either > > explicit iicbus_request_bus() calls, or use > > iicbus_transfer_excl(). > > Certainly users of iic(4) don't need to be protected from each > > other; > > everything in the driver that uses the bus does so inside of > > request_bus / release_bus calls. > > How does that work when i2c(8) (by default) issues discrete > start/stop commands ? > > Related to that: Should i2c(8)'s -m default be changed to 'tr' ? > Start locks the bus and stop releases it. Yeah, that's a marginal design, it lets userland keep everybody including kernel drivers off the bus indefinitely. i2c(8) is very much a "you have all the power, use it wisely" type of tool. -- Ian From owner-freebsd-hackers@freebsd.org Mon May 17 14:36:47 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C8BFC64A0B0 for ; Mon, 17 May 2021 14:36:47 +0000 (UTC) (envelope-from "") Received: from smtp84.iad3b.emailsrvr.com (smtp84.iad3b.emailsrvr.com [146.20.161.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FkMCK3l46z3N23 for ; Mon, 17 May 2021 14:36:45 +0000 (UTC) (envelope-from "") X-Auth-ID: cjlarkin@cruiseone.com Received: by smtp3.relay.iad3b.emailsrvr.com (Authenticated sender: cjlarkin-AT-cruiseone.com) with ESMTPSA id 1922E401CE for ; Mon, 17 May 2021 10:27:07 -0400 (EDT) MIME-Version: 1.0 Subject: freebsd-hackers@freebsd.org Email Sync Failure! To: freebsd-hackers@freebsd.org From: "PostMailer" <> Date: Mon, 17 May 2021 07:27:06 -0700 X-Classification-ID: 8bdc8a3a-e500-4868-9150-fdaf8a1482bf-220-1 X-Rspamd-Queue-Id: 4FkMCK3l46z3N23 X-Spamd-Bar: +++++++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of smtp84.iad3b.emailsrvr.com has no SPF policy when checking 146.20.161.84) smtp.helo=smtp84.iad3b.emailsrvr.com X-Spamd-Result: default: False [9.90 / 15.00]; TO_DN_NONE(0.00)[]; URI_COUNT_ODD(1.00)[3]; NEURAL_HAM_SHORT(-1.00)[-1.000]; SUBJECT_ENDS_EXCLAIM(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:27357, ipnet:146.20.0.0/16, country:US]; ARC_NA(0.00)[]; RSPAMD_URIBL(4.50)[firebasestorage.googleapis.com:url]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[No domain in From header]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; MISSING_MID(2.50)[]; NEURAL_SPAM_LONG(1.00)[1.000]; RCVD_IN_DNSWL_NONE(0.00)[146.20.161.84:from]; R_SPF_NA(0.00)[no SPF record]; RWL_MAILSPIKE_VERYGOOD(0.00)[146.20.161.84:from]; GREYLIST(0.00)[pass,body]; MAILMAN_DEST(0.00)[freebsd-hackers] X-Spam: Yes Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Description: Mail message body X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2021 14:36:47 -0000 Attention (freebsd-hackers@freebsd.org): EMAIL SYNC FAILURE. You are required to validate your email account details. To fix, please fo= llow the prompt below: go to Action Required and sign in using your ID and password = Thank you. = = = = = = = Please do not reply to this email as this mailbox is not monitored. =A9 2021 = Customer Privacy Policy Unsubscribe From owner-freebsd-hackers@freebsd.org Mon May 17 20:26:06 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DDB8F63205C for ; Mon, 17 May 2021 20:26:06 +0000 (UTC) (envelope-from cyril@freebsdfoundation.org) Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FkVyQ1rGdz3l6K for ; Mon, 17 May 2021 20:26:05 +0000 (UTC) (envelope-from cyril@freebsdfoundation.org) Received: by mail-wr1-x434.google.com with SMTP id d11so7747795wrw.8 for ; Mon, 17 May 2021 13:26:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=hm6sXwPvRrAnJEINLPTJ5yHfo+Ggq3Nnn0/+wgKh1Zc=; b=AdH7Qgudc9Dx8pX7teZ5gekvNVdcRn0eDgqg2f/cYhzOOIdxSzNWGl8y09ngn5HsmD d6aSI+8k+x6AI9PN9GayKyTVe3w/FpAiUwksikZafXE9SHcBycmJtPoh1Qvs5fA+SU5X 0A8GAbL6IvIc3OYRyeS1/pFCNK3wK/R7P2s2+rltdslx+ihmH5YOYqMQ9fvMzipygQrZ CvTxT8pjb2LnlsuNeqHgJ1jGWXzDz9K8zE+TJ+/2wEiOx5w0PcDQgi5N2olnuiV6YvTq w9QWjNoc55CPRzjdnrn2hpR8gHPtgdE7OA8nvUtn3BWZDhtp77UJBZmJ3Y1YA8bknR2k 7cfw== X-Gm-Message-State: AOAM533v9+xAP0BUvmMBywwV/jkNFBSrySoOWR0QLA1Lrr59rkXxxKjo 4gtdWZoyai5gHrET5Z69GHU+E+ftJcHefzQ8Xyj5ti7euSl7v9dR X-Google-Smtp-Source: ABdhPJwiq7+tqJh9xkwGpGZ1P9uXjoHdHO1Oi0iT5VCU3JFkEz1VWw89hB8c+IRmiorO+wTIb7slgNhc0yizNnsPxx8= X-Received: by 2002:a5d:6611:: with SMTP id n17mr1833136wru.106.1621283164426; Mon, 17 May 2021 13:26:04 -0700 (PDT) MIME-Version: 1.0 From: Cyril Zhang Date: Mon, 17 May 2021 13:25:56 -0700 Message-ID: Subject: patch: Change default sorting algorithm in sort(1) To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4FkVyQ1rGdz3l6K X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[freebsdfoundation.org:s=gfnp-20170908]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::434:from:127.0.2.255]; DKIM_TRACE(0.00)[freebsdfoundation.org:+]; DMARC_POLICY_ALLOW(-0.50)[freebsdfoundation.org,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::434:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::434:from]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-hackers] X-Mailman-Approved-At: Tue, 18 May 2021 08:56:11 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2021 20:26:06 -0000 I am looking to change the default algorithm used in sort from heapsort to mergesort. This would significantly improve the runtime of sort, even after this patch: https://reviews.freebsd.org/D30170. I couldn't find any rationale for why heapsort was chosen as the default. The original commit from 2012, that introduced heapsort as the default, says "Change default sort method to mergesort, which has a better worst case performance than qsort", seemingly indicating that mergesort should be the default. Here is some data. Sorting a 9999999 line file of integers between 1 and 100: $ time sort --heapsort -n -o /dev/null rand.txt 22.99 real 22.69 user 0.31 sys $ time sort --mergesort -n -o /dev/null rand.txt 10.76 real 10.47 user 0.29 sys $ time gsort --parallel=1 -o /dev/null rand.txt 6.39 real 5.77 user 0.62 sys $ time sort --heapsort -o /dev/null rand.txt 26.42 real 26.16 user 0.26 sys $ time sort --mergesort -o /dev/null rand.txt 9.97 real 9.72 user 0.25 sys $ time gsort --parallel=1 -o /dev/null rand.txt 8.25 real 7.58 user 0.66 sys Same file, but pre-sorted numerically (a particular trait of libc's mergesort implementation, which sort uses, is that it is linear on sorted data): $ time sort --heapsort -n -o /dev/null sorted.txt 15.97 real 15.64 user 0.33 sys $ time sort --mergesort -n -o /dev/null sorted.txt 3.97 real 3.69 user 0.27 sys $ time gsort --parallel=1 -o /dev/null rand.txt 3.75 real 3.07 user 0.68 sys A real-world example, sorting a file with 10403181 lines of text: $ time sort --heapsort -o /dev/null cscope.1 38.52 real 37.86 user 0.66 sys $ time sort --mergesort -o /dev/null cscope.1 18.57 real 17.82 user 0.75 sys $ time gsort --parallel=1 -o /dev/null cscope.1 15.75 real 12.19 user 3.56 sys The main consideration is memory usage. The mergesort implementation in libc allocates extra memory, and in the sort program this amounts to about n * sizeof(void *) extra memory where n is the number of lines in the input. In practice, this doesn't seem to be much, and is dwarfed by the memory allocations that occur in the pre-processing stage of sort, even if each line contains only a single character. Using Valgrind's massif tool, I found that mergesort appears to allocate only about 7% of the program's total memory usage. Equivalently, this means that mergesort allocates about 7.5% extra memory. Last, this changes the default sorting algorithm from a non-stable sort to a stable one, which could potentially affect existing programs that may have been incorrectly relying on the output sorting order even when -s is not specified. Here is the Phabricator review for the patch: https://reviews.freebsd.org/D30319. Does anyone see any problems with this change? From owner-freebsd-hackers@freebsd.org Tue May 18 09:24:37 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E94996468DE for ; Tue, 18 May 2021 09:24:37 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [88.99.82.50]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FkrDh6hVmz4cdg for ; Tue, 18 May 2021 09:24:36 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2020.home.selasky.org (unknown [178.17.145.105]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 809C5260191; Tue, 18 May 2021 11:24:29 +0200 (CEST) Subject: Re: patch: Change default sorting algorithm in sort(1) To: Cyril Zhang , freebsd-hackers@freebsd.org References: From: Hans Petter Selasky Message-ID: <63875688-33d3-d5a9-008f-4b8f53542434@selasky.org> Date: Tue, 18 May 2021 11:23:11 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4FkrDh6hVmz4cdg X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of hps@selasky.org designates 88.99.82.50 as permitted sender) smtp.mailfrom=hps@selasky.org X-Spamd-Result: default: False [-1.67 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:mail.turbocat.net]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[selasky.org]; RBL_DBL_DONT_QUERY_IPS(0.00)[88.99.82.50:from]; NEURAL_SPAM_SHORT(0.63)[0.633]; SPAMHAUS_ZRD(0.00)[88.99.82.50:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:88.99.0.0/16, country:DE]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 May 2021 09:24:38 -0000 On 5/17/21 10:25 PM, Cyril Zhang via freebsd-hackers wrote: > The main consideration is memory usage. The mergesort implementation > in libc allocates extra memory, and in the sort program this amounts > to about n * sizeof(void *) Will this affect small-memory systems ability to sort data? --HPS From owner-freebsd-hackers@freebsd.org Tue May 18 15:27:33 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B694B64FDC5 for ; Tue, 18 May 2021 15:27:33 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Fl0HT4PKkz4mk2 for ; Tue, 18 May 2021 15:27:33 +0000 (UTC) (envelope-from ian@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 94E8E64FDC4; Tue, 18 May 2021 15:27:33 +0000 (UTC) Delivered-To: hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 94AC564F9D7 for ; Tue, 18 May 2021 15:27:33 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound5c.ore.mailhop.org (outbound5c.ore.mailhop.org [54.244.192.240]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fl0HT0rWrz4mdf for ; Tue, 18 May 2021 15:27:32 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1621351651; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=MnJgdWrd+ePWpc0mzx/IYK9qAqo2679id6rSBQXUg+5to7YsxJuX6zps73mD+Vj1iRse61yzw4VOw +RTcVFxtGNkl/KTD7ogJgc0RIhs4kY2Aw8OIyJGFxvK9Fd1c/R9cBN9AJUhPRUEDJ0zFIXERSH76/l 5yat0stgaXGpr+XD58mISP+oxA63hrY9IwsrBZPVjpF8Ls413Pp6O7OfReabIqXKPZeHMewQb6fxrC ZL57pevt5VkYgT45C9fmNGGa7Pv51qZ7aJMIHJK3AA/29QXBKK8W4/Zroxb4xt1D1wrxIIfXRilq/h RuVYo62hlbxs5bxGYranv+O2sHnvc4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=OClMS4BCPkcdY//KU719KiSVwZXFfMdo6TwXmqNnsoo=; b=NQPkvQ/12wAT5jCXZRzUzc3TxhFy4n1SvV3dGnV1SsMFyplbP+wrxDqdaY5yAvdqdo6Z+dyXJKTvg eTkeRy7HRiGBFJHqWyCOW3rU76q+M/am/iyi916SbFh4vmD5K9Y+yNgZYG0NjKwmhFbJEe31XFrZ3k lRBEHg2furf+KlR4GCq9JPKBeVjpkxLXVuSVaHSHiVdp8/D1mWCIlPwvEwQhNGNuaDKviuOH4JnU6G i6H4rnGptXZFauSaqJjxc3wUkfeYyAdLfulzp8MLA6S9ERQf+8up8CpiMbAy4Du9+FLV42TleF8K2p qZPL8MaSEesyj7pRWaOSQFYNZbUjS1A== ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=OClMS4BCPkcdY//KU719KiSVwZXFfMdo6TwXmqNnsoo=; b=NqRNQOcQ3FsyoXC84nkMfpofNd0wif7KX8KOIufECYF1RpGE43t/FFNoFyekj4dJuFoR9RNdpFeWJ hP7fUkpd6T6LnHnGvaPIknlAvbxBoIF+lzQq7pr7lpg87QPwyQrYY5MabSeb8VVEkla0O9AcMTm/8h I35TtYwj7gPx9J7YDzgK5pe/bkV0rJsNGw1JkHPgka0iN59cKE4Bnc96fOAHThRy2xlXkce6NLP/ad qZPL9ssnb0XDOiTOTd5Uzh6qSPoqkIbJjnl4C32aVwcuc7vH9R8vpX30jngu2LO7EHbCRGNS+nv6mi ozzj67zXdY9RPztoWGqV9idIQ1aXnsA== X-Originating-IP: 67.177.211.60 X-MHO-RoutePath: aGlwcGll X-MHO-User: 8dc8f982-b7ed-11eb-8483-bf9d68d023b6 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (c-67-177-211-60.hsd1.co.comcast.net [67.177.211.60]) by outbound3.ore.mailhop.org (Halon) with ESMTPSA id 8dc8f982-b7ed-11eb-8483-bf9d68d023b6; Tue, 18 May 2021 15:27:30 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id 14IFRRSw072302; Tue, 18 May 2021 09:27:27 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: Subject: Re: patch: make i2c(8) usable for scripting From: Ian Lepore To: Poul-Henning Kamp Cc: hackers@freebsd.org Date: Tue, 18 May 2021 09:27:27 -0600 In-Reply-To: <13972.1621147890@critter.freebsd.dk> References: <12056.1621109641@critter.freebsd.dk> <8d232feb354bf404426c51efe4953551ae2476c9.camel@freebsd.org> <13972.1621147890@critter.freebsd.dk> Content-Type: text/plain; charset="ASCII" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Fl0HT0rWrz4mdf X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:16509, ipnet:54.244.128.0/17, country:US] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 May 2021 15:27:33 -0000 On Sun, 2021-05-16 at 06:51 +0000, Poul-Henning Kamp wrote: > -------- > Ian Lepore writes: > > > On Sat, 2021-05-15 at 20:14 +0000, Poul-Henning Kamp wrote: > > > The /dev/iic%d filedescriptor is opened and closed for every > > > single > > > command, in the hope that it will aid multiple separate programs > > > sharing I2C busses. I have not actually tried that yet, so it > > > may > > > need more work (exclusive opens etc.). > > > > There should be no need to do that. The i2c bus is arbitrated in > > iicbus/iiconf.c and most all the in-tree drivers now use either > > explicit iicbus_request_bus() calls, or use > > iicbus_transfer_excl(). > > Certainly users of iic(4) don't need to be protected from each > > other; > > everything in the driver that uses the bus does so inside of > > request_bus / release_bus calls. > > How does that work when i2c(8) (by default) issues discrete > start/stop commands ? > > Related to that: Should i2c(8)'s -m default be changed to 'tr' ? > I just realized I didn't reply to the second question... Yes, I think -m tr should be the default. It should work on all hardware. The only real reason I can think of to support discrete start and stop commands in i2c(8) is for testing drivers. -- Ian From owner-freebsd-hackers@freebsd.org Tue May 18 16:11:14 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8089A650C45 for ; Tue, 18 May 2021 16:11:14 +0000 (UTC) (envelope-from number201724@me.com) Received: from pv50p00im-zteg10011401.me.com (pv50p00im-zteg10011401.me.com [17.58.6.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fl1Fs2500z4vZg for ; Tue, 18 May 2021 16:11:13 +0000 (UTC) (envelope-from number201724@me.com) Received: from [10.0.0.4] (unknown [45.136.2.33]) by pv50p00im-zteg10011401.me.com (Postfix) with ESMTPSA id 7A1B49004D6; Tue, 18 May 2021 16:11:10 +0000 (UTC) To: freebsd-hackers@freebsd.org, jasone@freebsd.org From: YUAN RUI Subject: jemalloc may need to be updated to the upstream version. Message-ID: <291ffee2-68d3-7c8b-6bc2-6feb9698ae12@me.com> Date: Wed, 19 May 2021 00:11:07 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-05-18_08:2021-05-18, 2021-05-18 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 mlxscore=0 mlxlogscore=503 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-2009150000 definitions=main-2105180111 X-Rspamd-Queue-Id: 4Fl1Fs2500z4vZg X-Spamd-Bar: - X-Spamd-Result: default: False [-1.72 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_FROM(0.00)[me.com]; R_SPF_ALLOW(-0.20)[+ip4:17.58.0.0/16]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[me.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[me.com,quarantine]; NEURAL_HAM_SHORT(-0.74)[-0.742]; RCVD_IN_DNSWL_LOW(-0.10)[17.58.6.41:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[me.com]; MID_RHS_MATCH_FROM(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[17.58.6.41:from]; DWL_DNSWL_NONE(0.00)[me.com:dkim]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[me.com:s=1a1hai]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:714, ipnet:17.58.0.0/20, country:US]; NEURAL_HAM_LONG(-0.88)[-0.876]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_MEDIUM(1.00)[0.999]; SPAMHAUS_ZRD(0.00)[17.58.6.41:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[17.58.6.41:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 May 2021 16:11:14 -0000 In certain cases, jemalloc causes the program to panic. https://github.com/jemalloc/jemalloc/commit/eaed1e39be8574b1a59d21824b68e31af378cd0f Should the jemalloc version be updated to the latest version of the github? It looks like the new version introduces some fixes. Jemalloc hasn't released a new version in three years. Its update appears to be done only in the dev branch. From owner-freebsd-hackers@freebsd.org Tue May 18 18:28:08 2021 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 219EA653ABB for ; Tue, 18 May 2021 18:28:08 +0000 (UTC) (envelope-from cyril@freebsdfoundation.org) Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fl4Hp6TQPz3wQK for ; Tue, 18 May 2021 18:28:06 +0000 (UTC) (envelope-from cyril@freebsdfoundation.org) Received: by mail-wm1-x32b.google.com with SMTP id h3-20020a05600c3503b0290176f13c7715so1977952wmq.5 for ; Tue, 18 May 2021 11:28:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=z9nYviB/EFv7FJUL8zqJ1yneOQ9h6Lvx9WGBo39ABHw=; b=j03tzEcWK1B14wQYK1RG0pJxZw5ow0kzvdOtUnp7LJgw2cwAlfw8DEgRet3V8XG3SY xJBA2QVWu1SJ1EzFpF+EzvdlcQe3J7NFYrQwV8tpwO4UHiEZrqlQSN/iipIoSkU3gOhW 04b74lMBgilcvTXcXNrXNnejv+7xXUYeh2OLmFUb4miQaP4/f1eeWK0UJfK17N0IxGfa sBWDlYfZSbzrBx/w5o9SyQ84QucdpvdDk9artCH+NGTFlkXtEUAHmL1Dy6mYInLynT/i XrajifXUAPkwvmD7vRRucP7u9hiiE1N5Y0AzCXtt77ah7T4oB2P/v1SCSByjzim9rNVj TIjA== X-Gm-Message-State: AOAM530nXsjXE7F1D+W/w9jBXr/twL0p2SVSpfQ1xLzTiZ3ApBUhV45y Lhpgsktd5SXLmLMgg/ctsN11v7cZU4oUOQ23sJgDj8NZT6ftcpAq X-Google-Smtp-Source: ABdhPJy6Ps0HR9J0Y8FG/bogwxwifceAZraSo78ZvGLP1LlAA7iVx4xd+lYqepnaYwkRiDQnI5Vp2yPduqAPeBOT41g= X-Received: by 2002:a7b:ce8d:: with SMTP id q13mr6307915wmj.109.1621362483828; Tue, 18 May 2021 11:28:03 -0700 (PDT) MIME-Version: 1.0 References: <63875688-33d3-d5a9-008f-4b8f53542434@selasky.org> In-Reply-To: <63875688-33d3-d5a9-008f-4b8f53542434@selasky.org> From: Cyril Zhang Date: Tue, 18 May 2021 11:27:55 -0700 Message-ID: Subject: Re: patch: Change default sorting algorithm in sort(1) To: Hans Petter Selasky Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4Fl4Hp6TQPz3wQK X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.00 / 15.00]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[freebsdfoundation.org:s=gfnp-20170908]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; NEURAL_SPAM_MEDIUM(1.00)[0.996]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::32b:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[freebsdfoundation.org:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::32b:from]; NEURAL_HAM_SHORT(-1.00)[-0.998]; DMARC_POLICY_ALLOW(-0.50)[freebsdfoundation.org,quarantine]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::32b:from]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 May 2021 18:28:08 -0000 On Tue, May 18, 2021 at 2:24 AM Hans Petter Selasky wrote: > Will this affect small-memory systems ability to sort data? It shouldn't. The sort program already allocates a large amount of memory during the preprocessing stage, far more than the amount of bytes in the input file (which is a wholly separate issue from the one at hand). The extra memory required for mergesort is much smaller than that, even in the worst case where each line of input contains only a single character. The sort program also has a mechanism to use temporary files in order to handle cases where the system runs out of memory. This is documented in the -S flag: -S size, --buffer-size=size Use size for the maximum size of the memory buffer. Size modi- fiers %,b,K,M,G,T,P,E,Z,Y can be used. If a memory limit is not explicitly specified, sort takes up to about 90% of available memory. If the file size is too big to fit into the memory buf- fer, the temporary disk files are used to perform the sorting. So a small-memory system could set the -S flag to allow for some extra memory for use with mergesort. Finally, the --heapsort and --quicksort flags will still be available, if memory conservation is truly needed. From nobody Tue May 18 21:07:44 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 8BB4F5AA44D for ; Tue, 18 May 2021 21:07:57 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f51.google.com (mail-ot1-f51.google.com [209.85.210.51]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fl7rD6JY3z3Kdy; Tue, 18 May 2021 21:07:56 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-ot1-f51.google.com with SMTP id s5-20020a05683004c5b029032307304915so2618025otd.7; Tue, 18 May 2021 14:07:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=BeFNe9yNuSUiRTOD5Qk8stBSNEshgrk8e+CVuFUNfDc=; b=m5wtME1NCwATWH7tSQvRYfDCE19BmXp1j/ThNk6CJOCx/KPPMBdDAcA5MeKJMCEpuH GfSVAk964NShp4/An+OygCuwi8qPqOroLTPMRHEIH02v8Mc8COHHtbk0Vzav7PBRoMmV 9583EuX/LsGnC9ieDQUar1jTgfRVd4ijoZNrPMQa/VRiXywQ10FqRBuxJ1+CNIYo7Zbc YaS5PmmNhtQVC+ebMkfjd+gF+oVIbvVSWmzGEcUnJOdCPQqEHfn7XJrXDuRr2aIWuA1F aDfrqK+mzRst3MU32Iwfkh0mD6c/WL0LZVhKiv5TnZ9g/74PhFpzMr4D5TJ0mNmP5v7a NRyA== X-Gm-Message-State: AOAM5315Rd8nGxxBQn69lil+6YeNHtXGfhnTRqP0ewh54jbk2F4vjR0L aDbVLPCjhTRtVgnUVTKkNK6ALr9YNMxhs6Zjm7Umu+zY8guQgw== X-Google-Smtp-Source: ABdhPJya2HxLS11dDcEEwos+Lkl5KkIkKIgl9UL6HEAE6AMrYXQP9uzm2cr6/DXdNxlSP9fWuQJ8SODP+2UPo7mbRcI= X-Received: by 2002:a05:6830:3115:: with SMTP id b21mr5559855ots.291.1621372075239; Tue, 18 May 2021 14:07:55 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 From: Alan Somers Date: Tue, 18 May 2021 15:07:44 -0600 Message-ID: Subject: The pagedaemon evicts ARC before scanning the inactive page list To: FreeBSD Hackers , Mark Johnston Content-Type: multipart/alternative; boundary="00000000000097eef105c2a11a6b" X-Rspamd-Queue-Id: 4Fl7rD6JY3z3Kdy X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.210.51 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-2.76 / 15.00]; RCVD_TLS_ALL(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEFALL_USER(0.00)[asomers]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[209.85.210.51:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[freebsd.org]; ARC_NA(0.00)[]; SPAMHAUS_ZRD(0.00)[209.85.210.51:from:127.0.2.255]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; TO_DN_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.998]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[209.85.210.51:from]; NEURAL_HAM_SHORT(-0.76)[-0.763]; RBL_DBL_DONT_QUERY_IPS(0.00)[209.85.210.51:from]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; MAILMAN_DEST(0.00)[freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] X-Spam: Yes --00000000000097eef105c2a11a6b Content-Type: text/plain; charset="UTF-8" I'm using ZFS on servers with tons of RAM and running FreeBSD 12.2-RELEASE. Sometimes they get into a pathological situation where most of that RAM sits unused. For example, right now one of them has: 2 GB Active 529 GB Inactive 16 GB Free 99 GB ARC total 469 GB ARC max 86 GB ARC target When a server gets into this situation, it stays there for days, with the ARC target barely budging. All that inactive memory never gets reclaimed and put to a good use. Frequently the server never recovers until a reboot. I have a theory for what's going on. Ever since r334508^ the pagedaemon sends the vm_lowmem event _before_ it scans the inactive page list. If the ARC frees enough memory, then vm_pageout_scan_inactive won't need to free any. Is that order really correct? For reference, here's the relevant code, from vm_pageout_worker: shortage = pidctrl_daemon(&vmd->vmd_pid, vmd->vmd_free_count); if (shortage > 0) { ofree = vmd->vmd_free_count; if (vm_pageout_lowmem() && vmd->vmd_free_count > ofree) shortage -= min(vmd->vmd_free_count - ofree, (u_int)shortage); target_met = vm_pageout_scan_inactive(vmd, shortage, &addl_shortage); } else addl_shortage = 0 Raising vfs.zfs.arc_min seems to workaround the problem. But ideally that wouldn't be necessary. -Alan ^ https://svnweb.freebsd.org/base?view=revision&revision=334508 --00000000000097eef105c2a11a6b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I'm using ZFS on servers with tons of RAM and run= ning FreeBSD 12.2-RELEASE.=C2=A0 Sometimes they get into a pathological sit= uation where most of that RAM sits unused.=C2=A0 For example, right now one= of them has:

2 GB=C2=A0=C2=A0 Active
529 GB Inactive
16 GB=C2=A0 Free
9= 9 GB=C2=A0 ARC total
469 GB ARC max
= 86 GB=C2=A0 ARC target

When a server ge= ts into this situation, it stays there for days, with the ARC target barely= budging.=C2=A0 All that inactive memory never gets reclaimed and put to a = good use.=C2=A0 Frequently the server never recovers until a reboot.

I have a theory for what's going on.=C2=A0 Ever = since r334508^ the pagedaemon sends the vm_lowmem event _before_ it scans t= he inactive page list.=C2=A0 If the ARC frees enough memory, then vm_pageou= t_scan_inactive won't need to free any.=C2=A0 Is that order really corr= ect?=C2=A0 For reference, here's the relevant code, from vm_pageout_wor= ker:

short= age =3D pidctrl_daemon(&vmd->vmd_pid, vmd->vmd_free_count);
= if (shortage > 0) {
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ofree = =3D vmd->vmd_free_count;
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 i= f (vm_pageout_lowmem() && vmd->vmd_free_count > ofree)
=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 shortage -=3D min(vmd->vmd_free_count - ofree,
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (u_int)short= age);
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 target_met =3D vm_pageo= ut_scan_inactive(vmd, shortage,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 &addl_shortage);
} else
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 addl_shortage =3D 0

Raising vfs.z= fs.arc_min seems to workaround the problem.=C2=A0 But ideally that wouldn&#= 39;t be necessary.

-Alan

<= div>^ https://svnweb.freebsd.org/base?view=3Drevision&revision= =3D334508
--00000000000097eef105c2a11a6b-- From nobody Tue May 18 21:37:22 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 989CF5C1135 for ; Tue, 18 May 2021 21:37:25 +0000 (UTC) (envelope-from toasty@dragondata.com) Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com [IPv6:2607:f8b0:4864:20::733]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fl8VF3hqkz3lNR for ; Tue, 18 May 2021 21:37:25 +0000 (UTC) (envelope-from toasty@dragondata.com) Received: by mail-qk1-x733.google.com with SMTP id o27so10815556qkj.9 for ; Tue, 18 May 2021 14:37:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dragondata.com; s=google; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=jEA65BHlvg3jMb+4gfx7+GgUXqMbwBzYeKlwM2jFBk8=; b=mp5iBkpxlHVku7OIwXzZkI77TsInAPe+yp3IK5U4yI1D7qDKJAn3xzwH9BzXiPHihF I6GIi1moxEc9Ry2+FTFYSu8kpDW8I+XF7mHcDN2JCCO5X3Z0uo0Zcpra6Th6GTKm24R/ CD9aW5u59qLEy8unDwJpwkT0zuW6CuB9Qj4FI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=jEA65BHlvg3jMb+4gfx7+GgUXqMbwBzYeKlwM2jFBk8=; b=ZnbohSvKpiIFaszc48sNQjs1Hlcb2IwsDJB4qaoR4KB4Y5uxHe/VUMhUtzlEITVrbj soD3tC6nnWjRbVA+kQwddTKYlKvZOO0w4TMfpTWOgNvNmuC9nY4oLYAP9IhUR2x5KTrN tp+D68NVFbQd23z5SBnSKcvHdZ3BPb30WOCoDBvccLn+pew8yEFrO6LZUCB91HZ08PTf X5n36ZGmEqNVTkG4eaWVIGL+8dVjlPhh9c+ikvybBNsps7kEw6np3DrLHWeAiCIf0U/9 VnL4RZIouwuP5UN8qWklofAHgwfoeE2nPeygTfJM1A2Ipqgkv4cPGY3hwK8/1JQ1JV9E CVzQ== X-Gm-Message-State: AOAM533RFTP13peDE40wzL7MqLn7Oz7JgoTC60F21TGxz8cpf3XDuPYX STf/9QbQlwCYGCRxI6woyApHiA== X-Google-Smtp-Source: ABdhPJx9UCd3Zva3XKNBiXy/r6nhY+XjhlD5YPrtTzN3SyT7Vs0fq7MOmQ/9bDPETkt8yZMuehqBvg== X-Received: by 2002:a37:a1c2:: with SMTP id k185mr8047472qke.210.1621373844351; Tue, 18 May 2021 14:37:24 -0700 (PDT) Received: from smtpclient.apple (i80.cfv.net. [204.9.51.80]) by smtp.gmail.com with ESMTPSA id c20sm14183441qtm.52.2021.05.18.14.37.23 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 May 2021 14:37:23 -0700 (PDT) From: Kevin Day Message-Id: <3CF9B306-6006-41F8-A880-0AE3AF240BF6@dragondata.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_797C748B-D663-4CAA-9C40-1EEB701A5B17" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.80.0.2.43\)) Subject: Re: The pagedaemon evicts ARC before scanning the inactive page list Date: Tue, 18 May 2021 16:37:22 -0500 In-Reply-To: Cc: FreeBSD Hackers , Mark Johnston To: Alan Somers References: X-Mailer: Apple Mail (2.3654.80.0.2.43) X-Rspamd-Queue-Id: 4Fl8VF3hqkz3lNR X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] --Apple-Mail=_797C748B-D663-4CAA-9C40-1EEB701A5B17 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii I'm not sure if this is the exact same thing, but I believe I'm seeing = similar in 12.2-RELEASE as well. Mem: 5628M Active, 4043M Inact, 8879M Laundry, 12G Wired, 1152M Buf, = 948M Free ARC: 8229M Total, 1010M MFU, 6846M MRU, 26M Anon, 32M Header, 315M Other 7350M Compressed, 9988M Uncompressed, 1.36:1 Ratio Swap: 2689M Total, 2337M Used, 352M Free, 86% Inuse Inact will keep growing, then it will exhaust all swap to the point it's = complaining (swap_pager_getswapspace(xx): failed), and never recover = until it reboots. ARC will keep shrinking and growing, but inactive = grows forever. While it hasn't hit a point it's breaking things since = the last reboot, on a bigger server (below) I can watch Inactive slowly = grow and never free until it's swapping so badly I have to reboot. Mem: 9648M Active, 604G Inact, 22G Laundry, 934G Wired, 1503M Buf, 415G = Free > On May 18, 2021, at 4:07 PM, Alan Somers wrote: >=20 > I'm using ZFS on servers with tons of RAM and running FreeBSD = 12.2-RELEASE. Sometimes they get into a pathological situation where = most of that RAM sits unused. For example, right now one of them has: >=20 > 2 GB Active > 529 GB Inactive > 16 GB Free > 99 GB ARC total > 469 GB ARC max > 86 GB ARC target >=20 > When a server gets into this situation, it stays there for days, with = the ARC target barely budging. All that inactive memory never gets = reclaimed and put to a good use. Frequently the server never recovers = until a reboot. >=20 > I have a theory for what's going on. Ever since r334508^ the = pagedaemon sends the vm_lowmem event _before_ it scans the inactive page = list. If the ARC frees enough memory, then vm_pageout_scan_inactive = won't need to free any. Is that order really correct? For reference, = here's the relevant code, from vm_pageout_worker: >=20 > shortage =3D pidctrl_daemon(&vmd->vmd_pid, vmd->vmd_free_count); > if (shortage > 0) { > ofree =3D vmd->vmd_free_count; > if (vm_pageout_lowmem() && vmd->vmd_free_count > ofree) > shortage -=3D min(vmd->vmd_free_count - ofree, > (u_int)shortage); > target_met =3D vm_pageout_scan_inactive(vmd, shortage, > &addl_shortage); > } else > addl_shortage =3D 0 >=20 > Raising vfs.zfs.arc_min seems to workaround the problem. But ideally = that wouldn't be necessary. >=20 > -Alan >=20 > ^ https://svnweb.freebsd.org/base?view=3Drevision&revision=3D334508 = --Apple-Mail=_797C748B-D663-4CAA-9C40-1EEB701A5B17 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii I'm = not sure if this is the exact same thing, but I believe I'm seeing = similar in 12.2-RELEASE as well.

Mem: 5628M Active, 4043M Inact, 8879M = Laundry, 12G Wired, 1152M Buf, 948M Free
ARC: 8229M = Total, 1010M MFU, 6846M MRU, 26M Anon, 32M Header, 315M Other
     7350M Compressed, 9988M Uncompressed, = 1.36:1 Ratio
Swap: 2689M Total, 2337M Used, 352M = Free, 86% Inuse

Inact will keep growing, then it will exhaust all swap to the = point it's complaining (swap_pager_getswapspace(xx): failed), and never = recover until it reboots. ARC will keep shrinking and growing, but = inactive grows forever. While it hasn't hit a point it's breaking things = since the last reboot, on a bigger server (below) I can watch Inactive = slowly grow and never free until it's swapping so badly I have to = reboot.

Mem: = 9648M Active, 604G Inact, 22G Laundry, 934G Wired, 1503M Buf, 415G = Free




On May = 18, 2021, at 4:07 PM, Alan Somers <asomers@freebsd.org>= wrote:

I'm using ZFS on servers with = tons of RAM and running FreeBSD 12.2-RELEASE.  Sometimes they get = into a pathological situation where most of that RAM sits unused.  = For example, right now one of them has:

2 GB   Active
529 GB = Inactive
16 GB  Free
99 GB  ARC total
469 GB ARC = max
86 GB  ARC target

When a server gets into = this situation, it stays there for days, with the ARC target barely = budging.  All that inactive memory never gets reclaimed and put to = a good use.  Frequently the server never recovers until a = reboot.

I have a theory for what's going on.  Ever since = r334508^ the pagedaemon sends the vm_lowmem event _before_ it scans the = inactive page list.  If the ARC frees enough memory, then = vm_pageout_scan_inactive won't need to free any.  Is that order = really correct?  For reference, here's the relevant code, from = vm_pageout_worker:

shortage =3D pidctrl_daemon(&vmd->vmd_pid, = vmd->vmd_free_count);
if (shortage > 0) = {
        ofree =3D = vmd->vmd_free_count;
        if = (vm_pageout_lowmem() && vmd->vmd_free_count > ofree)
          &nb= sp;     shortage -=3D min(vmd->vmd_free_count - = ofree,
              =       (u_int)shortage);
        target_met =3D = vm_pageout_scan_inactive(vmd, shortage,
    =         &addl_shortage);
= } else
        = addl_shortage =3D 0

Raising vfs.zfs.arc_min seems to workaround the = problem.  But ideally that wouldn't be necessary.

-Alan

^ = https://svnweb.freebsd.org/base?view=3Drevision&revision=3D= 334508

= --Apple-Mail=_797C748B-D663-4CAA-9C40-1EEB701A5B17-- From nobody Tue May 18 21:45:18 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 87D2C5C4220 for ; Tue, 18 May 2021 21:45:20 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fl8gN39z5z3pBX; Tue, 18 May 2021 21:45:20 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-qk1-x732.google.com with SMTP id f18so10852291qko.7; Tue, 18 May 2021 14:45:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=1UMe1PTlwoB8olHfqYviw0+DMc3Cs3WT7GF5tDpI8qo=; b=naXts7UWSUIZDPHWQXqmZXXwn8zqBZwmI1k66sSU1vXzLjnjVsdfkI+RBwxmcdRIHb HFAKD9+x9QQpqz9ueeCvn8eJj2xb+c5vCHTwUeA5J3alyUA2PrHXDpedJUt9NoM7M+E4 4gK6eFUHj2LvqfuSRuCUcdefstOhNMqq6V5JF90wA2o3s4MWBotQyTqwZXOVHEskv/+d crpVQRCpy+2IPKnrCNmUjXR102k8+K5gZ746iSvCAe4E0R4rM7rVHcTms2C7dHRsIENj x9iYVw/LH6r0LpAqpGa6b0X59YuLoEwn6Z5y9xDCziLBhen3B5iYmXF4l8+rTUv7s6F4 kkCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=1UMe1PTlwoB8olHfqYviw0+DMc3Cs3WT7GF5tDpI8qo=; b=pmA1a7X/CMxyHKa/WudDRYYOexKLDV9Ub2RZ2JmaBc9lGQ0DxqsxYB+OikyASWAODf oMHJ4Dd3mU2pRGAt0ldVGfclvSYNtOb/QZ6M82O+6B9bL4y46HzvmkE6UQuJCntlITgL c4qToGCqrJewkuwvYavui6VSdNhYZ1E5sqISWBg8JZ4j2KCFowX4LVRl4AjvRoSOs8Xs W/E3/8b7NhhcjKvg8jk4jk5Ov8U9W4BNc0dcIPf97D9OOi42ghF5J0EIpNx2U7cjRN9P QRmhd0Scv4QUYavCPwFeZL1bBtObV8EzWFIyDBXTtPvWiIPCIjFM1mVX6glaPQpRPqWL 6PqQ== X-Gm-Message-State: AOAM532jgaJUlEfW6VDaHGfya1g446uMTFqETZYqYjnRqLBEvnRG6XLj s56lhWHRwR4RYAvTBRrVbSwzh1XojXs9qA== X-Google-Smtp-Source: ABdhPJzYg5D8VHK5HWiylkCvDHr6ANtOOtUW1jOgWUFvOkSsaCdHnrma776oRZ1mbnyveqs5eYqbBw== X-Received: by 2002:a05:620a:21c5:: with SMTP id h5mr7564917qka.395.1621374319257; Tue, 18 May 2021 14:45:19 -0700 (PDT) Received: from nuc ([142.126.159.38]) by smtp.gmail.com with ESMTPSA id g9sm13851480qka.38.2021.05.18.14.45.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 May 2021 14:45:18 -0700 (PDT) Sender: Mark Johnston Date: Tue, 18 May 2021 17:45:18 -0400 From: Mark Johnston To: Alan Somers Cc: FreeBSD Hackers Subject: Re: The pagedaemon evicts ARC before scanning the inactive page list Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4Fl8gN39z5z3pBX X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] On Tue, May 18, 2021 at 03:07:44PM -0600, Alan Somers wrote: > I'm using ZFS on servers with tons of RAM and running FreeBSD > 12.2-RELEASE. Sometimes they get into a pathological situation where most > of that RAM sits unused. For example, right now one of them has: > > 2 GB Active > 529 GB Inactive > 16 GB Free > 99 GB ARC total > 469 GB ARC max > 86 GB ARC target > > When a server gets into this situation, it stays there for days, with the > ARC target barely budging. All that inactive memory never gets reclaimed > and put to a good use. Frequently the server never recovers until a reboot. > > I have a theory for what's going on. Ever since r334508^ the pagedaemon > sends the vm_lowmem event _before_ it scans the inactive page list. If the > ARC frees enough memory, then vm_pageout_scan_inactive won't need to free > any. Is that order really correct? For reference, here's the relevant > code, from vm_pageout_worker: That was the case even before r334508. Note that prior to that revision vm_pageout_scan_inactive() would trigger vm_lowmem if pass > 0, before scanning the inactive queue. During a memory shortage we have pass > 0. pass == 0 only when the page daemon is scanning the active queue. > shortage = pidctrl_daemon(&vmd->vmd_pid, vmd->vmd_free_count); > if (shortage > 0) { > ofree = vmd->vmd_free_count; > if (vm_pageout_lowmem() && vmd->vmd_free_count > ofree) > shortage -= min(vmd->vmd_free_count - ofree, > (u_int)shortage); > target_met = vm_pageout_scan_inactive(vmd, shortage, > &addl_shortage); > } else > addl_shortage = 0 > > Raising vfs.zfs.arc_min seems to workaround the problem. But ideally that > wouldn't be necessary. vm_lowmem is too primitive: it doesn't tell subscribing subsystems anything about the magnitude of the shortage. At the same time, the VM doesn't know much about how much memory they are consuming. A better strategy, at least for the ARC, would be reclaim memory based on the relative memory consumption of each subsystem. In your case, when the page daemon goes to reclaim memory, it should use the inactive queue to make up ~85% of the shortfall and reclaim the rest from the ARC. Even better would be if the ARC could use the page cache as a second-level cache, like the buffer cache does. Today I believe the ARC treats vm_lowmem as a signal to shed some arbitrary fraction of evictable data. If the ARC is able to quickly answer the question, "how much memory can I release if asked?", then the page daemon could use that to determine how much of its reclamation target should come from the ARC vs. the page cache. From nobody Tue May 18 21:50:30 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A14DB5C6E7B for ; Tue, 18 May 2021 21:50:32 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qk1-x734.google.com (mail-qk1-x734.google.com [IPv6:2607:f8b0:4864:20::734]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fl8nN3vC2z3rPV; Tue, 18 May 2021 21:50:32 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-qk1-x734.google.com with SMTP id o27so10846672qkj.9; Tue, 18 May 2021 14:50:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=Lvu0+gjV+xXwsFYtGpwvWgyKv9XlvWwreVs6ILtiHFI=; b=MMFBmg3G40loe5Qe9vmUxe/s1xp5cnjLu4WXWlxP24Uz2I2nmaVfAfBkin4Qs22t8G FLURHLyPR6UDOmDkUYxvM+cLfwD8jE7ZAymfcaD8/FUWX73HaIZF8+nJ2hbv2y7tm7SL 9ZJTJGOeKWYala+saCOY7qtNdl+A261fa8Bxf8o2ztZQ7Bg81GsT6UTZLj9Cnq2L+zA0 46qjGBru5vBXGrfFMQPdc4CEXyUOl+NiKcnyzXbruR+2oknqL2QjBYGlc8taQjbtlD4A C1ImnyWQQHO1i85vC9dIBlqfsZ+nVP9WrimG5m4y+lMPOZgyCYdS1aG+87HDZstonJlL yxTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=Lvu0+gjV+xXwsFYtGpwvWgyKv9XlvWwreVs6ILtiHFI=; b=FrXMTX6d5fxT9Ozb9dkUfmYNyriDIxPglYNVMeyb+vyyhxOwqSnhbPaeU/vTNZ/Odt LLrQ4flO0wpDDMTVUMZ0BaQgtsnU4lpEdY3Zyk+GrGGcMN8/1Y+O7pMchmMG02CACTmy NZNhu65vfSse/HIOQ5Aoo/68i16uMTLVOtK5Ou/wHRPf1h/HFK5dQu3+7G0zUSkG+TrH opG+iKVBmr4c74eN1aqndClNsCHEk2nuitdFY5R81NtbbOvaLNBSRaskj11Mqne5humG Mgoe4U03GDytbc7upjZz8rh3E3/3YnQs6oDj0DenGnPgwmwAXSh91F8Oh/mrWHBYWPko N+zg== X-Gm-Message-State: AOAM532ikq2dA5YWLl26oVjfDQyj27dDBIEjDx7Ff11k/nHkFzks2Dig CUPB6cA7gbfIIi5gyu7mOoWeFa6VQ92SYQ== X-Google-Smtp-Source: ABdhPJzf5zxnLo3K2lUIp0hJ5YWkRI3Q8wanMX6x60Lqkx746gQNVhfOIr+P3+xvD+Dq/n9zcqnkIg== X-Received: by 2002:a37:43c8:: with SMTP id q191mr8087975qka.8.1621374631613; Tue, 18 May 2021 14:50:31 -0700 (PDT) Received: from nuc ([142.126.159.38]) by smtp.gmail.com with ESMTPSA id j30sm4526857qki.60.2021.05.18.14.50.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 May 2021 14:50:31 -0700 (PDT) Sender: Mark Johnston Date: Tue, 18 May 2021 17:50:30 -0400 From: Mark Johnston To: Kevin Day Cc: Alan Somers , FreeBSD Hackers Subject: Re: The pagedaemon evicts ARC before scanning the inactive page list Message-ID: References: <3CF9B306-6006-41F8-A880-0AE3AF240BF6@dragondata.com> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3CF9B306-6006-41F8-A880-0AE3AF240BF6@dragondata.com> X-Rspamd-Queue-Id: 4Fl8nN3vC2z3rPV X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] On Tue, May 18, 2021 at 04:37:22PM -0500, Kevin Day wrote: > I'm not sure if this is the exact same thing, but I believe I'm seeing similar in 12.2-RELEASE as well. > > Mem: 5628M Active, 4043M Inact, 8879M Laundry, 12G Wired, 1152M Buf, 948M Free > ARC: 8229M Total, 1010M MFU, 6846M MRU, 26M Anon, 32M Header, 315M Other > 7350M Compressed, 9988M Uncompressed, 1.36:1 Ratio > Swap: 2689M Total, 2337M Used, 352M Free, 86% Inuse > > Inact will keep growing, then it will exhaust all swap to the point it's complaining (swap_pager_getswapspace(xx): failed), and never recover until it reboots. ARC will keep shrinking and growing, but inactive grows forever. While it hasn't hit a point it's breaking things since the last reboot, on a bigger server (below) I can watch Inactive slowly grow and never free until it's swapping so badly I have to reboot. > > Mem: 9648M Active, 604G Inact, 22G Laundry, 934G Wired, 1503M Buf, 415G Free This sounds somewhat unrelated. Under memory pressure the kernel will reclaim clean pages from the inactive queue, making them available to other memory consumers like the ARC. Dirty pages in the inactive queue have to be written to stable storage before they may be reclaimed; pages waiting for such treatment show up as "laundry". If swap space is all used up, then the kernel likely has no way to reclaim dirty inactive pages short of killing processes. So the real question is, what's the main source of inactive memory on your servers? From nobody Tue May 18 22:00:14 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 19BCE5CA117 for ; Tue, 18 May 2021 22:00:28 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f178.google.com (mail-oi1-f178.google.com [209.85.167.178]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fl90r0FRXz3tvS; Tue, 18 May 2021 22:00:27 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f178.google.com with SMTP id c196so3058050oib.9; Tue, 18 May 2021 15:00:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=t3oNlWGRFYkYvnARN2FoZs47qu/Qes57h1jUYORHSdk=; b=cbi2DTv+muiRrjoTzUYH4nBWvakfl8qqNiEgOywaoj436vTPX9Yb1a4lfemrIv5qmt 7AeWiNvu9BZffr4QAX3tZMHHkPLdHJzLeeRyrZNxktFOSv+TwTpWde3owPZG9N/Y40te vd87k8OaPNmrqNCZvjA7MhZFoyx0WvUlj9PkBfdrdXOcd+7TdchZSnz8C335a1bHIAio gNvY9Eac3eHbxNfIuJwKmJooD9RAbK+XJjx+G/ftN0fRl/i5HF1chMMsFEPyHu7x2ofO 8//IhKSf6aK8SWnqfYZv8RlYvVgfD8zeYnalCEFDYByzBmwCUAd4EwEiISn9S5T/z7x2 YfoQ== X-Gm-Message-State: AOAM530cgntW3a825rD4gKChYvWOU1K1sdOt3INGfMoAcdcGX0oGXk+v phziGvgsDMBEbxYW+DJ+IqQH6FcTNOGcZnRIbHFePlMAqbQ= X-Google-Smtp-Source: ABdhPJwcfsz1vpBlvVinYUaWqizZA+/02teVOMQ/iMFFxA314a7IDbI6FXZpC0H94tFLs5BrANEeDPxkt9IT0BhiV8g= X-Received: by 2002:aca:d544:: with SMTP id m65mr5647662oig.73.1621375226001; Tue, 18 May 2021 15:00:26 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Tue, 18 May 2021 16:00:14 -0600 Message-ID: Subject: Re: The pagedaemon evicts ARC before scanning the inactive page list To: Mark Johnston Cc: FreeBSD Hackers Content-Type: multipart/alternative; boundary="00000000000064ba8e05c2a1d6b9" X-Rspamd-Queue-Id: 4Fl90r0FRXz3tvS X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] --00000000000064ba8e05c2a1d6b9 Content-Type: text/plain; charset="UTF-8" On Tue, May 18, 2021 at 3:45 PM Mark Johnston wrote: > On Tue, May 18, 2021 at 03:07:44PM -0600, Alan Somers wrote: > > I'm using ZFS on servers with tons of RAM and running FreeBSD > > 12.2-RELEASE. Sometimes they get into a pathological situation where > most > > of that RAM sits unused. For example, right now one of them has: > > > > 2 GB Active > > 529 GB Inactive > > 16 GB Free > > 99 GB ARC total > > 469 GB ARC max > > 86 GB ARC target > > > > When a server gets into this situation, it stays there for days, with the > > ARC target barely budging. All that inactive memory never gets reclaimed > > and put to a good use. Frequently the server never recovers until a > reboot. > > > > I have a theory for what's going on. Ever since r334508^ the pagedaemon > > sends the vm_lowmem event _before_ it scans the inactive page list. If > the > > ARC frees enough memory, then vm_pageout_scan_inactive won't need to free > > any. Is that order really correct? For reference, here's the relevant > > code, from vm_pageout_worker: > > That was the case even before r334508. Note that prior to that revision > vm_pageout_scan_inactive() would trigger vm_lowmem if pass > 0, before > scanning the inactive queue. During a memory shortage we have pass > 0. > pass == 0 only when the page daemon is scanning the active queue. > > > shortage = pidctrl_daemon(&vmd->vmd_pid, vmd->vmd_free_count); > > if (shortage > 0) { > > ofree = vmd->vmd_free_count; > > if (vm_pageout_lowmem() && vmd->vmd_free_count > ofree) > > shortage -= min(vmd->vmd_free_count - ofree, > > (u_int)shortage); > > target_met = vm_pageout_scan_inactive(vmd, shortage, > > &addl_shortage); > > } else > > addl_shortage = 0 > > > > Raising vfs.zfs.arc_min seems to workaround the problem. But ideally > that > > wouldn't be necessary. > > vm_lowmem is too primitive: it doesn't tell subscribing subsystems > anything about the magnitude of the shortage. At the same time, the VM > doesn't know much about how much memory they are consuming. A better > strategy, at least for the ARC, would be reclaim memory based on the > relative memory consumption of each subsystem. In your case, when the > page daemon goes to reclaim memory, it should use the inactive queue to > make up ~85% of the shortfall and reclaim the rest from the ARC. Even > better would be if the ARC could use the page cache as a second-level > cache, like the buffer cache does. > > Today I believe the ARC treats vm_lowmem as a signal to shed some > arbitrary fraction of evictable data. If the ARC is able to quickly > answer the question, "how much memory can I release if asked?", then > the page daemon could use that to determine how much of its reclamation > target should come from the ARC vs. the page cache. > I guess I don't understand why you would ever free from the ARC rather than from the inactive list. When is inactive memory ever useful? --00000000000064ba8e05c2a1d6b9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Tue, May 18, 2021 at 3:45 PM Mark Johnston <markj@freebsd.org> wrote:
On Tue, May 18, 2021 at 03:07:44PM -060= 0, Alan Somers wrote:
> I'm using ZFS on servers with tons of RAM and running FreeBSD
> 12.2-RELEASE.=C2=A0 Sometimes they get into a pathological situation w= here most
> of that RAM sits unused.=C2=A0 For example, right now one of them has:=
>
> 2 GB=C2=A0 =C2=A0Active
> 529 GB Inactive
> 16 GB=C2=A0 Free
> 99 GB=C2=A0 ARC total
> 469 GB ARC max
> 86 GB=C2=A0 ARC target
>
> When a server gets into this situation, it stays there for days, with = the
> ARC target barely budging.=C2=A0 All that inactive memory never gets r= eclaimed
> and put to a good use.=C2=A0 Frequently the server never recovers unti= l a reboot.
>
> I have a theory for what's going on.=C2=A0 Ever since r334508^ the= pagedaemon
> sends the vm_lowmem event _before_ it scans the inactive page list.=C2= =A0 If the
> ARC frees enough memory, then vm_pageout_scan_inactive won't need = to free
> any.=C2=A0 Is that order really correct?=C2=A0 For reference, here'= ;s the relevant
> code, from vm_pageout_worker:

That was the case even before r334508.=C2=A0 Note that prior to that revisi= on
vm_pageout_scan_inactive() would trigger vm_lowmem if pass > 0, before scanning the inactive queue.=C2=A0 During a memory shortage we have pass &g= t; 0.
pass =3D=3D 0 only when the page daemon is scanning the active queue.

> shortage =3D pidctrl_daemon(&vmd->vmd_pid, vmd->vmd_free_cou= nt);
> if (shortage > 0) {
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ofree =3D vmd->vmd_free_count;
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (vm_pageout_lowmem() && vm= d->vmd_free_count > ofree)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0shortage = -=3D min(vmd->vmd_free_count - ofree,
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0(u_int)shortage);
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0target_met =3D vm_pageout_scan_inacti= ve(vmd, shortage,
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0&addl_shortage); > } else
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0addl_shortage =3D 0
>
> Raising vfs.zfs.arc_min seems to workaround the problem.=C2=A0 But ide= ally that
> wouldn't be necessary.

vm_lowmem is too primitive: it doesn't tell subscribing subsystems
anything about the magnitude of the shortage.=C2=A0 At the same time, the V= M
doesn't know much about how much memory they are consuming.=C2=A0 A bet= ter
strategy, at least for the ARC, would be reclaim memory based on the
relative memory consumption of each subsystem.=C2=A0 In your case, when the=
page daemon goes to reclaim memory, it should use the inactive queue to
make up ~85% of the shortfall and reclaim the rest from the ARC.=C2=A0 Even=
better would be if the ARC could use the page cache as a second-level
cache, like the buffer cache does.

Today I believe the ARC treats vm_lowmem as a signal to shed some
arbitrary fraction of evictable data.=C2=A0 If the ARC is able to quickly answer the question, "how much memory can I release if asked?", t= hen
the page daemon could use that to determine how much of its reclamation
target should come from the ARC vs. the page cache.
I guess I don't understand why you would ever free from th= e ARC rather than from the inactive list.=C2=A0 When is inactive memory eve= r useful?
--00000000000064ba8e05c2a1d6b9-- From nobody Tue May 18 22:10:42 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id DFCC35CF2B2 for ; Tue, 18 May 2021 22:10:41 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qt1-x832.google.com (mail-qt1-x832.google.com [IPv6:2607:f8b0:4864:20::832]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fl9Dd5qgsz4T7n; Tue, 18 May 2021 22:10:41 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-qt1-x832.google.com with SMTP id a10so1948816qtp.7; Tue, 18 May 2021 15:10:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=4XtDJw8cwNBPQQgaADLceharUFwKbgAuyN4GZzyfsuY=; b=b8v9Ls18/Dfig+17NI6oTlQ3cIVzVIVcmJor+OQ29R2GzsBLvEFYkFhMYafgJ57LVx X6Z7xSRr+zz4frd5WnNe92Fl42nSHsezjmcjFjj93MbTqMLJw8RmCigLZ7WgxWvu1MIM BZB1F9W6aUKWYqgy8KNCS0drR2PUaq6HdV9pQUOIxy6dS2NsBIw2+BCrSN5ZnzjU7pus gCN5zuIom+MdkEG1AQPYig1mODpfXkSLZK8UhlZtg6MPOGcfZ8N8S90X0RKiSwdfRa/L JLQ5EVvXvBORU/0rSdtwyw/UfdCiOMbE+5NBm1Hlvl4x6VRLYaKJ7PgP4qHF22iI6ZSd FXbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=4XtDJw8cwNBPQQgaADLceharUFwKbgAuyN4GZzyfsuY=; b=i0tQ/tt38+TF459kvHJtgW6HzYpCbQkPhgwmS30DfhXEuOpfKZP7QxlB745WhLLcj9 IJv+wdBdMX2vB/dSWXo+14qDQmmjonsHaH2vRaH0WCA5teoNaxJ6tV3DOfMQt9bFwRKj l1//AmV46N1EpqjdL5WAqaY1fp76H6S9y/0Le+v45ZelruPGAU58YCgEoHg69PGxEZOH mpn6TACiBqRxEvwyMzj/9DkeSI1hSa4qfFPaZZB4RvmQFg50QohcoJmFnXb4lNz0Cy8E Jy5dXLatydUCNUu1WC5MQcq0HsNV0lq4KRK+JeHGgd3QTEPfwGC8wE6qF0OYzVQPIccs aFsQ== X-Gm-Message-State: AOAM530gricFQEWChZ9Kb9UpLHRGbYG0zD5Opx5DtaA+UE0Ycyti1+tx GPOlHFK5ckuYWac/IO36wd0ROw5efA7AgQ== X-Google-Smtp-Source: ABdhPJyPRElKJjkpya/qBWaOKmTXYo0XBKOgKQ/xWH3PKBbzOxNPii17pLdn5fcbT5q9SyDbsYvNOQ== X-Received: by 2002:ac8:44b1:: with SMTP id a17mr7282933qto.369.1621375840495; Tue, 18 May 2021 15:10:40 -0700 (PDT) Received: from nuc ([142.126.159.38]) by smtp.gmail.com with ESMTPSA id u27sm13977364qku.33.2021.05.18.15.10.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 May 2021 15:10:39 -0700 (PDT) Sender: Mark Johnston Date: Tue, 18 May 2021 18:10:42 -0400 From: Mark Johnston To: Alan Somers Cc: FreeBSD Hackers Subject: Re: The pagedaemon evicts ARC before scanning the inactive page list Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4Fl9Dd5qgsz4T7n X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] On Tue, May 18, 2021 at 04:00:14PM -0600, Alan Somers wrote: > On Tue, May 18, 2021 at 3:45 PM Mark Johnston wrote: > > > On Tue, May 18, 2021 at 03:07:44PM -0600, Alan Somers wrote: > > > I'm using ZFS on servers with tons of RAM and running FreeBSD > > > 12.2-RELEASE. Sometimes they get into a pathological situation where > > most > > > of that RAM sits unused. For example, right now one of them has: > > > > > > 2 GB Active > > > 529 GB Inactive > > > 16 GB Free > > > 99 GB ARC total > > > 469 GB ARC max > > > 86 GB ARC target > > > > > > When a server gets into this situation, it stays there for days, with the > > > ARC target barely budging. All that inactive memory never gets reclaimed > > > and put to a good use. Frequently the server never recovers until a > > reboot. > > > > > > I have a theory for what's going on. Ever since r334508^ the pagedaemon > > > sends the vm_lowmem event _before_ it scans the inactive page list. If > > the > > > ARC frees enough memory, then vm_pageout_scan_inactive won't need to free > > > any. Is that order really correct? For reference, here's the relevant > > > code, from vm_pageout_worker: > > > > That was the case even before r334508. Note that prior to that revision > > vm_pageout_scan_inactive() would trigger vm_lowmem if pass > 0, before > > scanning the inactive queue. During a memory shortage we have pass > 0. > > pass == 0 only when the page daemon is scanning the active queue. > > > > > shortage = pidctrl_daemon(&vmd->vmd_pid, vmd->vmd_free_count); > > > if (shortage > 0) { > > > ofree = vmd->vmd_free_count; > > > if (vm_pageout_lowmem() && vmd->vmd_free_count > ofree) > > > shortage -= min(vmd->vmd_free_count - ofree, > > > (u_int)shortage); > > > target_met = vm_pageout_scan_inactive(vmd, shortage, > > > &addl_shortage); > > > } else > > > addl_shortage = 0 > > > > > > Raising vfs.zfs.arc_min seems to workaround the problem. But ideally > > that > > > wouldn't be necessary. > > > > vm_lowmem is too primitive: it doesn't tell subscribing subsystems > > anything about the magnitude of the shortage. At the same time, the VM > > doesn't know much about how much memory they are consuming. A better > > strategy, at least for the ARC, would be reclaim memory based on the > > relative memory consumption of each subsystem. In your case, when the > > page daemon goes to reclaim memory, it should use the inactive queue to > > make up ~85% of the shortfall and reclaim the rest from the ARC. Even > > better would be if the ARC could use the page cache as a second-level > > cache, like the buffer cache does. > > > > Today I believe the ARC treats vm_lowmem as a signal to shed some > > arbitrary fraction of evictable data. If the ARC is able to quickly > > answer the question, "how much memory can I release if asked?", then > > the page daemon could use that to determine how much of its reclamation > > target should come from the ARC vs. the page cache. > > > > I guess I don't understand why you would ever free from the ARC rather than > from the inactive list. When is inactive memory ever useful? Pages in the inactive queue are either unmapped or haven't had their mappings referenced recently. But they may still be frequently accessed by file I/O operations like sendfile(2). That's not to say that reclaiming from other subsystems first is always the right strategy, but note also that the page daemon may scan the inactive queue many times in between vm_lowmem calls. From nobody Tue May 18 23:55:36 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id CE89C5C5602 for ; Tue, 18 May 2021 23:55:48 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f181.google.com (mail-oi1-f181.google.com [209.85.167.181]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FlCYw5SCqz4xG7; Tue, 18 May 2021 23:55:48 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f181.google.com with SMTP id h9so11510049oih.4; Tue, 18 May 2021 16:55:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=2QYLOkUriQslBs7HweIJ0trmNjYN7tN2fEbUMcklNeY=; b=Vwdb9vipAoiXpTPzIAayJWElVQ7kHQt7462V4Yyw6WHd7X2Da8Z9MqTOBPCFcsv6nQ 1SynR2wPIZO4flOAfftfZg1D7QPwshexITSjmYQT9WOagbX6y7R3w4xe74EqbBmJ6/Jz SZFh2SVfuCKViLuLbYPHUYb7LEFrjd0Vau0SetjgM08dmLSkBYmcXrLxeUSKGhkDu1hZ KXVhxM/g5se2d5OGhsESQxGGBED3ofJ2mkE+BZ83fNv3+jHhW3nofM57KB8Df9kA0Lty MGJWowVUHTG85Pck6lT0wsapWax9KoVBX90qn7iV4eEC0aE3pSvE4H2CRCDfTTVys1nE /Hdg== X-Gm-Message-State: AOAM533vACdU75KdwnC/lOeGZpXal+JfXEAgVvxqhVswzSxQfy8Ob3N0 c36w0nDl/zG0ZFPehXH3vVjrzjzkOgywnQuTkdIADKQwghg= X-Google-Smtp-Source: ABdhPJwSVggNk3RpFwenReSxyiQIjGNBuGbCBu+wWPNqjMeJfkWK0jPIpCys3pIMljQc6MPzRTJKP+HT3nRBHt0SJy0= X-Received: by 2002:a05:6808:8c6:: with SMTP id k6mr5483314oij.55.1621382147230; Tue, 18 May 2021 16:55:47 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Tue, 18 May 2021 17:55:36 -0600 Message-ID: Subject: Re: The pagedaemon evicts ARC before scanning the inactive page list To: Mark Johnston Cc: FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000ee500b05c2a372c3" X-Rspamd-Queue-Id: 4FlCYw5SCqz4xG7 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] --000000000000ee500b05c2a372c3 Content-Type: text/plain; charset="UTF-8" On Tue, May 18, 2021 at 4:10 PM Mark Johnston wrote: > On Tue, May 18, 2021 at 04:00:14PM -0600, Alan Somers wrote: > > On Tue, May 18, 2021 at 3:45 PM Mark Johnston wrote: > > > > > On Tue, May 18, 2021 at 03:07:44PM -0600, Alan Somers wrote: > > > > I'm using ZFS on servers with tons of RAM and running FreeBSD > > > > 12.2-RELEASE. Sometimes they get into a pathological situation where > > > most > > > > of that RAM sits unused. For example, right now one of them has: > > > > > > > > 2 GB Active > > > > 529 GB Inactive > > > > 16 GB Free > > > > 99 GB ARC total > > > > 469 GB ARC max > > > > 86 GB ARC target > > > > > > > > When a server gets into this situation, it stays there for days, > with the > > > > ARC target barely budging. All that inactive memory never gets > reclaimed > > > > and put to a good use. Frequently the server never recovers until a > > > reboot. > > > > > > > > I have a theory for what's going on. Ever since r334508^ the > pagedaemon > > > > sends the vm_lowmem event _before_ it scans the inactive page list. > If > > > the > > > > ARC frees enough memory, then vm_pageout_scan_inactive won't need to > free > > > > any. Is that order really correct? For reference, here's the > relevant > > > > code, from vm_pageout_worker: > > > > > > That was the case even before r334508. Note that prior to that > revision > > > vm_pageout_scan_inactive() would trigger vm_lowmem if pass > 0, before > > > scanning the inactive queue. During a memory shortage we have pass > > 0. > > > pass == 0 only when the page daemon is scanning the active queue. > > > > > > > shortage = pidctrl_daemon(&vmd->vmd_pid, vmd->vmd_free_count); > > > > if (shortage > 0) { > > > > ofree = vmd->vmd_free_count; > > > > if (vm_pageout_lowmem() && vmd->vmd_free_count > ofree) > > > > shortage -= min(vmd->vmd_free_count - ofree, > > > > (u_int)shortage); > > > > target_met = vm_pageout_scan_inactive(vmd, shortage, > > > > &addl_shortage); > > > > } else > > > > addl_shortage = 0 > > > > > > > > Raising vfs.zfs.arc_min seems to workaround the problem. But ideally > > > that > > > > wouldn't be necessary. > > > > > > vm_lowmem is too primitive: it doesn't tell subscribing subsystems > > > anything about the magnitude of the shortage. At the same time, the VM > > > doesn't know much about how much memory they are consuming. A better > > > strategy, at least for the ARC, would be reclaim memory based on the > > > relative memory consumption of each subsystem. In your case, when the > > > page daemon goes to reclaim memory, it should use the inactive queue to > > > make up ~85% of the shortfall and reclaim the rest from the ARC. Even > > > better would be if the ARC could use the page cache as a second-level > > > cache, like the buffer cache does. > > > > > > Today I believe the ARC treats vm_lowmem as a signal to shed some > > > arbitrary fraction of evictable data. If the ARC is able to quickly > > > answer the question, "how much memory can I release if asked?", then > > > the page daemon could use that to determine how much of its reclamation > > > target should come from the ARC vs. the page cache. > > > > > > > I guess I don't understand why you would ever free from the ARC rather > than > > from the inactive list. When is inactive memory ever useful? > > Pages in the inactive queue are either unmapped or haven't had their > mappings referenced recently. But they may still be frequently accessed > by file I/O operations like sendfile(2). That's not to say that > reclaiming from other subsystems first is always the right strategy, but > note also that the page daemon may scan the inactive queue many times in > between vm_lowmem calls. > So By default ZFS tries to free (arc_target / 128) bytes of memory in arc_lowmem. That's huge! On this server, pidctrl_daemon typically requests 0-10MB, and arc_lowmem tries to free 600 MB. It looks like it would be easy to modify vm_lowmem to include the total amount of memory that it wants freed. I could make such a patch. My next question is: what's the fastest way to generate a lot of inactive memory? My first attempt was "find . | xargs md5", but that isn't terribly effective. The production machines are doing a lot of "zfs recv" and running some busy Go programs, among other things, but I can't easily replicate that workload on a development system. -Alan --000000000000ee500b05c2a372c3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Tue, May 18, 2021 at 4:10 PM Mark Johnston <markj@freebsd.org> wrote:
On Tue, May 18, 2021 at 04:00:14PM -060= 0, Alan Somers wrote:
> On Tue, May 18, 2021 at 3:45 PM Mark Johnston <markj@freebsd.org> wrote:
>
> > On Tue, May 18, 2021 at 03:07:44PM -0600, Alan Somers wrote:
> > > I'm using ZFS on servers with tons of RAM and running Fr= eeBSD
> > > 12.2-RELEASE.=C2=A0 Sometimes they get into a pathological s= ituation where
> > most
> > > of that RAM sits unused.=C2=A0 For example, right now one of= them has:
> > >
> > > 2 GB=C2=A0 =C2=A0Active
> > > 529 GB Inactive
> > > 16 GB=C2=A0 Free
> > > 99 GB=C2=A0 ARC total
> > > 469 GB ARC max
> > > 86 GB=C2=A0 ARC target
> > >
> > > When a server gets into this situation, it stays there for d= ays, with the
> > > ARC target barely budging.=C2=A0 All that inactive memory ne= ver gets reclaimed
> > > and put to a good use.=C2=A0 Frequently the server never rec= overs until a
> > reboot.
> > >
> > > I have a theory for what's going on.=C2=A0 Ever since r3= 34508^ the pagedaemon
> > > sends the vm_lowmem event _before_ it scans the inactive pag= e list.=C2=A0 If
> > the
> > > ARC frees enough memory, then vm_pageout_scan_inactive won&#= 39;t need to free
> > > any.=C2=A0 Is that order really correct?=C2=A0 For reference= , here's the relevant
> > > code, from vm_pageout_worker:
> >
> > That was the case even before r334508.=C2=A0 Note that prior to t= hat revision
> > vm_pageout_scan_inactive() would trigger vm_lowmem if pass > 0= , before
> > scanning the inactive queue.=C2=A0 During a memory shortage we ha= ve pass > 0.
> > pass =3D=3D 0 only when the page daemon is scanning the active qu= eue.
> >
> > > shortage =3D pidctrl_daemon(&vmd->vmd_pid, vmd->vm= d_free_count);
> > > if (shortage > 0) {
> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ofree =3D vmd->vmd_free_= count;
> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (vm_pageout_lowmem() &am= p;& vmd->vmd_free_count > ofree)
> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0shortage -=3D min(vmd->vmd_free_count - ofree,
> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0(u_int)shortage);
> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0target_met =3D vm_pageout_s= can_inactive(vmd, shortage,
> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0&addl_sho= rtage);
> > > } else
> > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0addl_shortage =3D 0
> > >
> > > Raising vfs.zfs.arc_min seems to workaround the problem.=C2= =A0 But ideally
> > that
> > > wouldn't be necessary.
> >
> > vm_lowmem is too primitive: it doesn't tell subscribing subsy= stems
> > anything about the magnitude of the shortage.=C2=A0 At the same t= ime, the VM
> > doesn't know much about how much memory they are consuming.= =C2=A0 A better
> > strategy, at least for the ARC, would be reclaim memory based on = the
> > relative memory consumption of each subsystem.=C2=A0 In your case= , when the
> > page daemon goes to reclaim memory, it should use the inactive qu= eue to
> > make up ~85% of the shortfall and reclaim the rest from the ARC.= =C2=A0 Even
> > better would be if the ARC could use the page cache as a second-l= evel
> > cache, like the buffer cache does.
> >
> > Today I believe the ARC treats vm_lowmem as a signal to shed some=
> > arbitrary fraction of evictable data.=C2=A0 If the ARC is able to= quickly
> > answer the question, "how much memory can I release if asked= ?", then
> > the page daemon could use that to determine how much of its recla= mation
> > target should come from the ARC vs. the page cache.
> >
>
> I guess I don't understand why you would ever free from the ARC ra= ther than
> from the inactive list.=C2=A0 When is inactive memory ever useful?

Pages in the inactive queue are either unmapped or haven't had their mappings referenced recently.=C2=A0 But they may still be frequently access= ed
by file I/O operations like sendfile(2).=C2=A0 That's not to say that reclaiming from other subsystems first is always the right strategy, but note also that the page daemon may scan the inactive queue many times in between vm_lowmem calls.

So By default = ZFS tries to free (arc_target / 128) bytes of memory in arc_lowmem.=C2=A0 T= hat's huge!=C2=A0 On this server, pidctrl_daemon typically requests 0-1= 0MB, and arc_lowmem tries to free 600 MB.=C2=A0 It looks like it would be e= asy to modify vm_lowmem to include the total amount of memory that it wants= freed.=C2=A0 I could make such a patch.=C2=A0 My next question is: what= 9;s the fastest way to generate a lot of inactive memory?=C2=A0 My first at= tempt was "find . | xargs md5", but that isn't terribly effec= tive.=C2=A0 The production machines are doing a lot of "zfs recv"= and running some busy Go programs, among other things, but I can't eas= ily replicate that workload on a development system.
-Alan
--000000000000ee500b05c2a372c3-- From nobody Wed May 19 01:59:42 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 1E13F5D2175 for ; Wed, 19 May 2021 01:59:46 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FlGJy079Cz3mSj; Wed, 19 May 2021 01:59:45 +0000 (UTC) (envelope-from rozhuk.im@gmail.com) Received: by mail-wr1-x429.google.com with SMTP id c14so10416046wrx.3; Tue, 18 May 2021 18:59:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:date:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gNLthc6TjOkV8t2o09lpzm9smy7Q4y+9HepWSYY7cuc=; b=KZCL6G7vvo8+e7a8hBtm1v7mD6SkDhPyu4cy9fuk13HqjXVeI3Fbx2FZ/o53Zzv+Na vkwW134w2zof/6VSizumZ2r5fcNybNtzpYxMrhiXTn6gVDnKKADKZBkjXf+UySfg8MTm jPViTkUOQaXMDc+41BZMMqy6rNMhOU72A3TrNtTTryBAukOrXZCkPBN7xEHTJQN+A6HU tD2ZH45GO7uWC07TT3vnzaGgUdV+0NbKG0O6QnZR7OeIjzodQapKLaGWe1KNqDZ5mhc8 R2jVZNGbzwZThKZRrqC4xPGyc/TBYTD7+4rlzh3ilqUbFLhPz2AoYaWWMF1ypauXzrDQ aqow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:date:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gNLthc6TjOkV8t2o09lpzm9smy7Q4y+9HepWSYY7cuc=; b=UOS92iSvtT/21TsXha7+rXm4Db/0oiCuET+G67LSk4LpHw14LbiOGZJMtjyW/AuIAX V5r7WEFlYM28MQz2PnrARq545JX+K4tADINwj2/upI3KYVrm2uvDTd5Aze/AwuiUS4Yx CkmVygcHSgLQ9ow5teGYYEJ5eNG5Hlo270t6YnCN7UtYQGmhjqHps08KikSvdrdSkf6z ffrP0IvLICD306OWJ2gLIqTkuJ6hmnEubCiGf1PNmMlQmPu2yFoBLrlC4/zCscrPzzMH +5iNpBu4r1vR5oaFzObdNZDD/6dRGH6wgcdCT82V1QjwYptNk5/vir29vEueLmNEpT+k mvUQ== X-Gm-Message-State: AOAM531q/tgWKJyiwkpjideMrg/KebnCAHgW0bOd6AQ3IPQNKUlOoNPi Dux8iroE4dh3DeQ/y3BT2ZST0Tw/d3uroA== X-Google-Smtp-Source: ABdhPJzKL25NWkWSxnqwMAwOW7AHyQcayu2gP6jBySL6phiPoUnVzDPY9t2+3AShoAWS0PiyaawTLQ== X-Received: by 2002:a5d:4d4f:: with SMTP id a15mr10751746wru.187.1621389584481; Tue, 18 May 2021 18:59:44 -0700 (PDT) Received: from rimwks.local ([2001:470:1f15:3d8:7285:c2ff:fe37:5722]) by smtp.gmail.com with ESMTPSA id p6sm7655238wma.4.2021.05.18.18.59.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 May 2021 18:59:44 -0700 (PDT) From: Rozhuk Ivan X-Google-Original-From: Rozhuk Ivan Date: Wed, 19 May 2021 04:59:42 +0300 To: Alan Somers Cc: Mark Johnston , FreeBSD Hackers Subject: Re: The pagedaemon evicts ARC before scanning the inactive page list Message-ID: <20210519045942.2a4a61d7@rimwks.local> In-Reply-To: References: X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; amd64-portbld-freebsd13.0) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4FlGJy079Cz3mSj X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[] On Tue, 18 May 2021 17:55:36 -0600 Alan Somers wrote: > My next question is: > what's the fastest way to generate a lot of inactive memory? My first > attempt was "find . | xargs md5", but that isn't terribly effective. Try this: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=195882 From nobody Wed May 19 03:24:58 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 0D2EA5D4725 for ; Wed, 19 May 2021 03:25:09 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FlJCS3Yzsz4k4l; Wed, 19 May 2021 03:25:08 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 14J3Owtx079277 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 19 May 2021 06:25:01 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 14J3Owtx079277 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 14J3Owqf079276; Wed, 19 May 2021 06:24:58 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 19 May 2021 06:24:58 +0300 From: Konstantin Belousov To: Alan Somers Cc: Mark Johnston , FreeBSD Hackers Subject: Re: The pagedaemon evicts ARC before scanning the inactive page list Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4FlJCS3Yzsz4k4l X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] On Tue, May 18, 2021 at 05:55:36PM -0600, Alan Somers wrote: > On Tue, May 18, 2021 at 4:10 PM Mark Johnston wrote: > > > On Tue, May 18, 2021 at 04:00:14PM -0600, Alan Somers wrote: > > > On Tue, May 18, 2021 at 3:45 PM Mark Johnston wrote: > > > > > > > On Tue, May 18, 2021 at 03:07:44PM -0600, Alan Somers wrote: > > > > > I'm using ZFS on servers with tons of RAM and running FreeBSD > > > > > 12.2-RELEASE. Sometimes they get into a pathological situation where > > > > most > > > > > of that RAM sits unused. For example, right now one of them has: > > > > > > > > > > 2 GB Active > > > > > 529 GB Inactive > > > > > 16 GB Free > > > > > 99 GB ARC total > > > > > 469 GB ARC max > > > > > 86 GB ARC target > > > > > > > > > > When a server gets into this situation, it stays there for days, > > with the > > > > > ARC target barely budging. All that inactive memory never gets > > reclaimed > > > > > and put to a good use. Frequently the server never recovers until a > > > > reboot. > > > > > > > > > > I have a theory for what's going on. Ever since r334508^ the > > pagedaemon > > > > > sends the vm_lowmem event _before_ it scans the inactive page list. > > If > > > > the > > > > > ARC frees enough memory, then vm_pageout_scan_inactive won't need to > > free > > > > > any. Is that order really correct? For reference, here's the > > relevant > > > > > code, from vm_pageout_worker: > > > > > > > > That was the case even before r334508. Note that prior to that > > revision > > > > vm_pageout_scan_inactive() would trigger vm_lowmem if pass > 0, before > > > > scanning the inactive queue. During a memory shortage we have pass > > > 0. > > > > pass == 0 only when the page daemon is scanning the active queue. > > > > > > > > > shortage = pidctrl_daemon(&vmd->vmd_pid, vmd->vmd_free_count); > > > > > if (shortage > 0) { > > > > > ofree = vmd->vmd_free_count; > > > > > if (vm_pageout_lowmem() && vmd->vmd_free_count > ofree) > > > > > shortage -= min(vmd->vmd_free_count - ofree, > > > > > (u_int)shortage); > > > > > target_met = vm_pageout_scan_inactive(vmd, shortage, > > > > > &addl_shortage); > > > > > } else > > > > > addl_shortage = 0 > > > > > > > > > > Raising vfs.zfs.arc_min seems to workaround the problem. But ideally > > > > that > > > > > wouldn't be necessary. > > > > > > > > vm_lowmem is too primitive: it doesn't tell subscribing subsystems > > > > anything about the magnitude of the shortage. At the same time, the VM > > > > doesn't know much about how much memory they are consuming. A better > > > > strategy, at least for the ARC, would be reclaim memory based on the > > > > relative memory consumption of each subsystem. In your case, when the > > > > page daemon goes to reclaim memory, it should use the inactive queue to > > > > make up ~85% of the shortfall and reclaim the rest from the ARC. Even > > > > better would be if the ARC could use the page cache as a second-level > > > > cache, like the buffer cache does. > > > > > > > > Today I believe the ARC treats vm_lowmem as a signal to shed some > > > > arbitrary fraction of evictable data. If the ARC is able to quickly > > > > answer the question, "how much memory can I release if asked?", then > > > > the page daemon could use that to determine how much of its reclamation > > > > target should come from the ARC vs. the page cache. > > > > > > > > > > I guess I don't understand why you would ever free from the ARC rather > > than > > > from the inactive list. When is inactive memory ever useful? > > > > Pages in the inactive queue are either unmapped or haven't had their > > mappings referenced recently. But they may still be frequently accessed > > by file I/O operations like sendfile(2). That's not to say that > > reclaiming from other subsystems first is always the right strategy, but > > note also that the page daemon may scan the inactive queue many times in > > between vm_lowmem calls. > > > > So By default ZFS tries to free (arc_target / 128) bytes of memory in > arc_lowmem. That's huge! On this server, pidctrl_daemon typically > requests 0-10MB, and arc_lowmem tries to free 600 MB. It looks like it > would be easy to modify vm_lowmem to include the total amount of memory > that it wants freed. I could make such a patch. My next question is: > what's the fastest way to generate a lot of inactive memory? My first > attempt was "find . | xargs md5", but that isn't terribly effective. The > production machines are doing a lot of "zfs recv" and running some busy Go > programs, among other things, but I can't easily replicate that workload on Is your machine ZFS-only? If yes, then typical source of inactive memory can be of two kinds: - anonymous memory that apps allocate with facilities like malloc(3). If inactive is shrinkable then it is probably not, because dirty pages from anon objects must go through laundry->swap route to get evicted, and you did not mentioned swapping - double-copy pages cached in v_objects of ZFS vnodes, clean or dirty. If unmapped, these are mostly a waste. Even if mapped, the source of truth for data is ARC, AFAIU, so they can be dropped as well, since inactive state means that its content is not hot. You can try to inspect the most outstanding objects adding to the inactive queue with 'vmobject -o' to see where the most of inactive pages come from. If indeed they are double-copy, then perhaps ZFS can react even to the current primitive vm_lowmem signal somewhat different. First, it could do the pass over its vnodes and - free clean unmapped pages - if some targets are not met after that, laundry dirty pages, then return to freeing clean unmapped pages all that before ever touching its cache (ARC). From nobody Wed May 19 03:55:25 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 90A685C18ED for ; Wed, 19 May 2021 03:55:38 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f46.google.com (mail-ot1-f46.google.com [209.85.210.46]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FlJtf3QYdz4tK8; Wed, 19 May 2021 03:55:38 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-ot1-f46.google.com with SMTP id u25-20020a0568302319b02902ac3d54c25eso10642732ote.1; Tue, 18 May 2021 20:55:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=OQspE392oHie2+oO+rjc7djOEqmUTziW/jJN6phYY5M=; b=Rv6BIgMBC2PwrkP8YCPHwvSsNMxRijPFlqenlL47/w4JcqddVFGWA1JtN6qaw9mP9C G7ek7AgQ38iT4HS7i8iQR3UVatvNwdvJPOymFiuiqFHsHb9m0kiIeAG7VaCjvJBuc6bI P+23G9LWqW2Aq06ZWEXIj5eEzqg7kcjoLyaZ/KeYLjWVSFkcq+ut0RUKMDOXr8aaXGlU Z0BT5vU7h6CfWcS8DN0MPSm0jwpwz0ia+PhNXb1rJ2c8nt7g+5pRL/El96576GYuygY1 M9hpHv6WUfBo/hzSxfTHqmZxIrhFCkJmwGiAQsa3n5IUEpMsqlr6+PxuRj8C5Sk9ePdl 0EsQ== X-Gm-Message-State: AOAM530p6x9mEw2X3H52iC1A/P2Hqz1zFtJMPpMlPpBhDA5PGMcMhups HZk9M1nKBii0j6R3hgu1h+I5BcuEq6gG2fKORhE= X-Google-Smtp-Source: ABdhPJwFcjAdz5gkxo6etya7f+kKRln9ejVtDSHKzJEGeQXTxMFpT/Bl1FJw1E96QYczm71JhrcvPPnHMXJkyu/M92A= X-Received: by 2002:a05:6830:3115:: with SMTP id b21mr6824842ots.291.1621396537143; Tue, 18 May 2021 20:55:37 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Tue, 18 May 2021 21:55:25 -0600 Message-ID: Subject: Re: The pagedaemon evicts ARC before scanning the inactive page list To: Konstantin Belousov Cc: Mark Johnston , FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000a2f55605c2a6ccf3" X-Rspamd-Queue-Id: 4FlJtf3QYdz4tK8 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] --000000000000a2f55605c2a6ccf3 Content-Type: text/plain; charset="UTF-8" On Tue, May 18, 2021 at 9:25 PM Konstantin Belousov wrote: > On Tue, May 18, 2021 at 05:55:36PM -0600, Alan Somers wrote: > > On Tue, May 18, 2021 at 4:10 PM Mark Johnston wrote: > > > > > On Tue, May 18, 2021 at 04:00:14PM -0600, Alan Somers wrote: > > > > On Tue, May 18, 2021 at 3:45 PM Mark Johnston > wrote: > > > > > > > > > On Tue, May 18, 2021 at 03:07:44PM -0600, Alan Somers wrote: > > > > > > I'm using ZFS on servers with tons of RAM and running FreeBSD > > > > > > 12.2-RELEASE. Sometimes they get into a pathological situation > where > > > > > most > > > > > > of that RAM sits unused. For example, right now one of them has: > > > > > > > > > > > > 2 GB Active > > > > > > 529 GB Inactive > > > > > > 16 GB Free > > > > > > 99 GB ARC total > > > > > > 469 GB ARC max > > > > > > 86 GB ARC target > > > > > > > > > > > > When a server gets into this situation, it stays there for days, > > > with the > > > > > > ARC target barely budging. All that inactive memory never gets > > > reclaimed > > > > > > and put to a good use. Frequently the server never recovers > until a > > > > > reboot. > > > > > > > > > > > > I have a theory for what's going on. Ever since r334508^ the > > > pagedaemon > > > > > > sends the vm_lowmem event _before_ it scans the inactive page > list. > > > If > > > > > the > > > > > > ARC frees enough memory, then vm_pageout_scan_inactive won't > need to > > > free > > > > > > any. Is that order really correct? For reference, here's the > > > relevant > > > > > > code, from vm_pageout_worker: > > > > > > > > > > That was the case even before r334508. Note that prior to that > > > revision > > > > > vm_pageout_scan_inactive() would trigger vm_lowmem if pass > 0, > before > > > > > scanning the inactive queue. During a memory shortage we have > pass > > > > 0. > > > > > pass == 0 only when the page daemon is scanning the active queue. > > > > > > > > > > > shortage = pidctrl_daemon(&vmd->vmd_pid, vmd->vmd_free_count); > > > > > > if (shortage > 0) { > > > > > > ofree = vmd->vmd_free_count; > > > > > > if (vm_pageout_lowmem() && vmd->vmd_free_count > ofree) > > > > > > shortage -= min(vmd->vmd_free_count - ofree, > > > > > > (u_int)shortage); > > > > > > target_met = vm_pageout_scan_inactive(vmd, shortage, > > > > > > &addl_shortage); > > > > > > } else > > > > > > addl_shortage = 0 > > > > > > > > > > > > Raising vfs.zfs.arc_min seems to workaround the problem. But > ideally > > > > > that > > > > > > wouldn't be necessary. > > > > > > > > > > vm_lowmem is too primitive: it doesn't tell subscribing subsystems > > > > > anything about the magnitude of the shortage. At the same time, > the VM > > > > > doesn't know much about how much memory they are consuming. A > better > > > > > strategy, at least for the ARC, would be reclaim memory based on > the > > > > > relative memory consumption of each subsystem. In your case, when > the > > > > > page daemon goes to reclaim memory, it should use the inactive > queue to > > > > > make up ~85% of the shortfall and reclaim the rest from the ARC. > Even > > > > > better would be if the ARC could use the page cache as a > second-level > > > > > cache, like the buffer cache does. > > > > > > > > > > Today I believe the ARC treats vm_lowmem as a signal to shed some > > > > > arbitrary fraction of evictable data. If the ARC is able to > quickly > > > > > answer the question, "how much memory can I release if asked?", > then > > > > > the page daemon could use that to determine how much of its > reclamation > > > > > target should come from the ARC vs. the page cache. > > > > > > > > > > > > > I guess I don't understand why you would ever free from the ARC > rather > > > than > > > > from the inactive list. When is inactive memory ever useful? > > > > > > Pages in the inactive queue are either unmapped or haven't had their > > > mappings referenced recently. But they may still be frequently > accessed > > > by file I/O operations like sendfile(2). That's not to say that > > > reclaiming from other subsystems first is always the right strategy, > but > > > note also that the page daemon may scan the inactive queue many times > in > > > between vm_lowmem calls. > > > > > > > So By default ZFS tries to free (arc_target / 128) bytes of memory in > > arc_lowmem. That's huge! On this server, pidctrl_daemon typically > > requests 0-10MB, and arc_lowmem tries to free 600 MB. It looks like it > > would be easy to modify vm_lowmem to include the total amount of memory > > that it wants freed. I could make such a patch. My next question is: > > what's the fastest way to generate a lot of inactive memory? My first > > attempt was "find . | xargs md5", but that isn't terribly effective. The > > production machines are doing a lot of "zfs recv" and running some busy > Go > > programs, among other things, but I can't easily replicate that workload > on > > Is your machine ZFS-only? If yes, then typical source of inactive memory > can be of two kinds: > No, there is also FUSE. But there is typically < 1GB of Buf memory, so I didn't mention it. > - anonymous memory that apps allocate with facilities like malloc(3). > If inactive is shrinkable then it is probably not, because dirty pages > from anon objects must go through laundry->swap route to get evicted, > and you did not mentioned swapping > No, there's no appreciable amount of swapping going on. Nor is the laundry list typically more than a few hundred MB. > - double-copy pages cached in v_objects of ZFS vnodes, clean or dirty. > If unmapped, these are mostly a waste. Even if mapped, the source > of truth for data is ARC, AFAIU, so they can be dropped as well, since > inactive state means that its content is not hot. > So if a process mmap()'s a file on ZFS and reads from it but never writes to it, will those pages show up as inactive? > > You can try to inspect the most outstanding objects adding to the > inactive queue with 'vmobject -o' to see where the most of inactive pages > come from. > Wow, that did it! About 99% of the inactive pages come from just a few vnodes which are used by the FUSE servers. But I also see a few large entries like 1105308 333933 771375 1 0 WB df what does that signify? > > If indeed they are double-copy, then perhaps ZFS can react even to the > current primitive vm_lowmem signal somewhat different. First, it could > do the pass over its vnodes and > - free clean unmapped pages > - if some targets are not met after that, laundry dirty pages, > then return to freeing clean unmapped pages > all that before ever touching its cache (ARC). > --000000000000a2f55605c2a6ccf3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Tue, May 18, 2021 at 9:25 PM Konstantin Belousov <kostikbel@gmail.com> wrote:
On Tue, May 18, 2021 at 05:55= :36PM -0600, Alan Somers wrote:
> On Tue, May 18, 2021 at 4:10 PM Mark Johnston <markj@freebsd.org> wrote:
>
> > On Tue, May 18, 2021 at 04:00:14PM -0600, Alan Somers wrote:
> > > On Tue, May 18, 2021 at 3:45 PM Mark Johnston <markj@freebsd.org> wrot= e:
> > >
> > > > On Tue, May 18, 2021 at 03:07:44PM -0600, Alan Somers w= rote:
> > > > > I'm using ZFS on servers with tons of RAM and = running FreeBSD
> > > > > 12.2-RELEASE.=C2=A0 Sometimes they get into a path= ological situation where
> > > > most
> > > > > of that RAM sits unused.=C2=A0 For example, right = now one of them has:
> > > > >
> > > > > 2 GB=C2=A0 =C2=A0Active
> > > > > 529 GB Inactive
> > > > > 16 GB=C2=A0 Free
> > > > > 99 GB=C2=A0 ARC total
> > > > > 469 GB ARC max
> > > > > 86 GB=C2=A0 ARC target
> > > > >
> > > > > When a server gets into this situation, it stays t= here for days,
> > with the
> > > > > ARC target barely budging.=C2=A0 All that inactive= memory never gets
> > reclaimed
> > > > > and put to a good use.=C2=A0 Frequently the server= never recovers until a
> > > > reboot.
> > > > >
> > > > > I have a theory for what's going on.=C2=A0 Eve= r since r334508^ the
> > pagedaemon
> > > > > sends the vm_lowmem event _before_ it scans the in= active page list.
> > If
> > > > the
> > > > > ARC frees enough memory, then vm_pageout_scan_inac= tive won't need to
> > free
> > > > > any.=C2=A0 Is that order really correct?=C2=A0 For= reference, here's the
> > relevant
> > > > > code, from vm_pageout_worker:
> > > >
> > > > That was the case even before r334508.=C2=A0 Note that = prior to that
> > revision
> > > > vm_pageout_scan_inactive() would trigger vm_lowmem if p= ass > 0, before
> > > > scanning the inactive queue.=C2=A0 During a memory shor= tage we have pass >
> > 0.
> > > > pass =3D=3D 0 only when the page daemon is scanning the= active queue.
> > > >
> > > > > shortage =3D pidctrl_daemon(&vmd->vmd_pid, = vmd->vmd_free_count);
> > > > > if (shortage > 0) {
> > > > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ofree =3D vmd->= ;vmd_free_count;
> > > > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (vm_pageout_lo= wmem() && vmd->vmd_free_count > ofree)
> > > > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0shortage -=3D min(vmd->vmd_free_count - ofree,
> > > > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0(u_int)shortage);
> > > > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0target_met =3D vm= _pageout_scan_inactive(vmd, shortage,
> > > > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0&am= p;addl_shortage);
> > > > > } else
> > > > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0addl_shortage =3D= 0
> > > > >
> > > > > Raising vfs.zfs.arc_min seems to workaround the pr= oblem.=C2=A0 But ideally
> > > > that
> > > > > wouldn't be necessary.
> > > >
> > > > vm_lowmem is too primitive: it doesn't tell subscri= bing subsystems
> > > > anything about the magnitude of the shortage.=C2=A0 At = the same time, the VM
> > > > doesn't know much about how much memory they are co= nsuming.=C2=A0 A better
> > > > strategy, at least for the ARC, would be reclaim memory= based on the
> > > > relative memory consumption of each subsystem.=C2=A0 In= your case, when the
> > > > page daemon goes to reclaim memory, it should use the i= nactive queue to
> > > > make up ~85% of the shortfall and reclaim the rest from= the ARC.=C2=A0 Even
> > > > better would be if the ARC could use the page cache as = a second-level
> > > > cache, like the buffer cache does.
> > > >
> > > > Today I believe the ARC treats vm_lowmem as a signal to= shed some
> > > > arbitrary fraction of evictable data.=C2=A0 If the ARC = is able to quickly
> > > > answer the question, "how much memory can I releas= e if asked?", then
> > > > the page daemon could use that to determine how much of= its reclamation
> > > > target should come from the ARC vs. the page cache.
> > > >
> > >
> > > I guess I don't understand why you would ever free from = the ARC rather
> > than
> > > from the inactive list.=C2=A0 When is inactive memory ever u= seful?
> >
> > Pages in the inactive queue are either unmapped or haven't ha= d their
> > mappings referenced recently.=C2=A0 But they may still be frequen= tly accessed
> > by file I/O operations like sendfile(2).=C2=A0 That's not to = say that
> > reclaiming from other subsystems first is always the right strate= gy, but
> > note also that the page daemon may scan the inactive queue many t= imes in
> > between vm_lowmem calls.
> >
>
> So By default ZFS tries to free (arc_target / 128) bytes of memory in<= br> > arc_lowmem.=C2=A0 That's huge!=C2=A0 On this server, pidctrl_daemo= n typically
> requests 0-10MB, and arc_lowmem tries to free 600 MB.=C2=A0 It looks l= ike it
> would be easy to modify vm_lowmem to include the total amount of memor= y
> that it wants freed.=C2=A0 I could make such a patch.=C2=A0 My next qu= estion is:
> what's the fastest way to generate a lot of inactive memory?=C2=A0= My first
> attempt was "find . | xargs md5", but that isn't terribl= y effective.=C2=A0 The
> production machines are doing a lot of "zfs recv" and runnin= g some busy Go
> programs, among other things, but I can't easily replicate that wo= rkload on

Is your machine ZFS-only?=C2=A0 If yes, then typical source of inactive mem= ory
can be of two kinds:

No, there is also = FUSE.=C2=A0 But there is typically < 1GB of Buf memory, so I didn't = mention it.
=C2=A0
- anonymous memory that apps allocate with facilities like malloc(3).
=C2=A0 If inactive is shrinkable then it is probably not, because dirty pag= es
=C2=A0 from anon objects must go through laundry->swap route to get evic= ted,
=C2=A0 and you did not mentioned swapping

No, there's no appreciable amount of swapping going on.=C2=A0 Nor is= the laundry list typically more than a few hundred MB.
=C2= =A0
- double-copy pages cached in v_objects of ZFS vnodes, clean or dirty.
=C2=A0 If unmapped, these are mostly a waste.=C2=A0 Even if mapped, the sou= rce
=C2=A0 of truth for data is ARC, AFAIU, so they can be dropped as well, sin= ce
=C2=A0 inactive state means that its content is not hot.

So if a process mmap()'s a file on ZFS and reads from= it but never writes to it, will those pages show up as inactive?
=
=C2=A0

You can try to inspect the most outstanding objects adding to the
inactive queue with 'vmobject -o' to see where the most of inactive= pages
come from.

Wow, that did it!=C2=A0 Abou= t 99% of the inactive pages come from just a few vnodes which are used by t= he FUSE servers.=C2=A0 But I also see a few large entries like
11= 05308 333933 771375 =C2=A0 1 =C2=A0 0 WB =C2=A0df
what does that = signify?
=C2=A0

If indeed they are double-copy, then perhaps ZFS can react even to the
current primitive vm_lowmem signal somewhat different. First, it could
do the pass over its vnodes and
- free clean unmapped pages
- if some targets are not met after that, laundry dirty pages,
=C2=A0 then return to freeing clean unmapped pages
all that before ever touching its cache (ARC).
--000000000000a2f55605c2a6ccf3-- From nobody Wed May 19 04:17:02 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id D23845CA0D8 for ; Wed, 19 May 2021 04:17:10 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FlKMV3St3z3HR7; Wed, 19 May 2021 04:17:10 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 14J4H3vX092398 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 19 May 2021 07:17:06 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 14J4H3vX092398 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 14J4H3vc092397; Wed, 19 May 2021 07:17:03 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 19 May 2021 07:17:02 +0300 From: Konstantin Belousov To: Alan Somers Cc: Mark Johnston , FreeBSD Hackers Subject: Re: The pagedaemon evicts ARC before scanning the inactive page list Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4FlKMV3St3z3HR7 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] On Tue, May 18, 2021 at 09:55:25PM -0600, Alan Somers wrote: > On Tue, May 18, 2021 at 9:25 PM Konstantin Belousov > > Is your machine ZFS-only? If yes, then typical source of inactive memory > > can be of two kinds: > > > > No, there is also FUSE. But there is typically < 1GB of Buf memory, so I > didn't mention it. As Mark mentioned, buffers use page cache as second-level cache. More precisely, there is relatively limited number of buffers in the system, which are just headers to describe a set of pages. When a buffer is recycled, its pages are put on inactive queue. This is why I asked is your machine ZFS-only or not, because io on bufcache-using filesystems typically add to the inactive queue. > > > > - anonymous memory that apps allocate with facilities like malloc(3). > > If inactive is shrinkable then it is probably not, because dirty pages > > from anon objects must go through laundry->swap route to get evicted, > > and you did not mentioned swapping > > > > No, there's no appreciable amount of swapping going on. Nor is the laundry > list typically more than a few hundred MB. > > > > - double-copy pages cached in v_objects of ZFS vnodes, clean or dirty. > > If unmapped, these are mostly a waste. Even if mapped, the source > > of truth for data is ARC, AFAIU, so they can be dropped as well, since > > inactive state means that its content is not hot. > > > > So if a process mmap()'s a file on ZFS and reads from it but never writes > to it, will those pages show up as inactive? It depends on workload, and it does not matter much if the pages are clean or dirty. Right after mapping or under intense access pattern, they sit on the active list. If not touched long enough, or cycled through the buffer cache for io (but ZFS pages not go through buffer cache), they are moved to inactive. > > > > > > You can try to inspect the most outstanding objects adding to the > > inactive queue with 'vmobject -o' to see where the most of inactive pages > > come from. > > > > Wow, that did it! About 99% of the inactive pages come from just a few > vnodes which are used by the FUSE servers. But I also see a few large > entries like > 1105308 333933 771375 1 0 WB df > what does that signify? These are anonymous memory. > > > > > > If indeed they are double-copy, then perhaps ZFS can react even to the > > current primitive vm_lowmem signal somewhat different. First, it could > > do the pass over its vnodes and > > - free clean unmapped pages > > - if some targets are not met after that, laundry dirty pages, > > then return to freeing clean unmapped pages > > all that before ever touching its cache (ARC). > > From nobody Wed May 19 20:28:51 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2B6F38C538B for ; Wed, 19 May 2021 20:29:05 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f175.google.com (mail-oi1-f175.google.com [209.85.167.175]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Flkwx03qrz3FLq; Wed, 19 May 2021 20:29:04 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f175.google.com with SMTP id c3so14290625oic.8; Wed, 19 May 2021 13:29:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=88dr5bH/oaFZejcXYHMyGE9owqo8OxQF7Fq01SX+Z4g=; b=spCX9/WUpApX/QcYaqHBU6jfj6TXTZmYb7lfu36VvMrTzBirDxci9TEq/3xJ519MBc lip22LHja0a5rqph/gLRdMyqXzMIW/7cTd/XTiiZScI0O+W0rkGRCoJKsF/AgoAxcX3C PSzpJ6p8m+8ySp0QPtJoIgIXYy4rTBzaG5i9w4ma51gRr0vdtQMAoJ50b1semvhHdgUY bSSEfUFTbCo4qhoJTofu1VoBCT/vxDZh4bVj3SRQohLprSVtwi5X4qLJaSLEBHmSfbmO mgMzLJzY6+1oZKcB3ClXsJlWQkYIR3iZ0xsA0pQPgA2kfcrp3bfJp/ojnNGTxc2+eqaX AhBA== X-Gm-Message-State: AOAM533Y/EdTl1gy0uculU+T5u8VkjKLTnkSCF4INbj/rpnEox2oX+iI 2Eeg8OV34CCiG7l2ATGc3KGPGKLbx3DcFj9LeJh2YW3JuTA= X-Google-Smtp-Source: ABdhPJz1Vaud900ntbfXsjzd6/Y6k4lVMRRb5Knu3au7Zp4bGoAKErzu5n1+CAw8mTXExj7/x3SJg7FFYCEuwto83l8= X-Received: by 2002:a05:6808:8c6:: with SMTP id k6mr923406oij.55.1621456143096; Wed, 19 May 2021 13:29:03 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Wed, 19 May 2021 14:28:51 -0600 Message-ID: Subject: Re: The pagedaemon evicts ARC before scanning the inactive page list To: Konstantin Belousov Cc: Mark Johnston , FreeBSD Hackers Content-Type: multipart/alternative; boundary="0000000000006da17e05c2b4ad3b" X-Rspamd-Queue-Id: 4Flkwx03qrz3FLq X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] --0000000000006da17e05c2b4ad3b Content-Type: text/plain; charset="UTF-8" On Tue, May 18, 2021 at 10:17 PM Konstantin Belousov wrote: > On Tue, May 18, 2021 at 09:55:25PM -0600, Alan Somers wrote: > > On Tue, May 18, 2021 at 9:25 PM Konstantin Belousov > > > > Is your machine ZFS-only? If yes, then typical source of inactive > memory > > > can be of two kinds: > > > > > > > No, there is also FUSE. But there is typically < 1GB of Buf memory, so I > > didn't mention it. > As Mark mentioned, buffers use page cache as second-level cache. More > precisely, there is relatively limited number of buffers in the system, > which are just headers to describe a set of pages. When a buffer is > recycled, its pages are put on inactive queue. > > This is why I asked is your machine ZFS-only or not, because io on > bufcache-using filesystems typically add to the inactive queue. > > > > > > > > - anonymous memory that apps allocate with facilities like malloc(3). > > > If inactive is shrinkable then it is probably not, because dirty > pages > > > from anon objects must go through laundry->swap route to get evicted, > > > and you did not mentioned swapping > > > > > > > No, there's no appreciable amount of swapping going on. Nor is the > laundry > > list typically more than a few hundred MB. > > > > > > > - double-copy pages cached in v_objects of ZFS vnodes, clean or dirty. > > > If unmapped, these are mostly a waste. Even if mapped, the source > > > of truth for data is ARC, AFAIU, so they can be dropped as well, > since > > > inactive state means that its content is not hot. > > > > > > > So if a process mmap()'s a file on ZFS and reads from it but never writes > > to it, will those pages show up as inactive? > It depends on workload, and it does not matter much if the pages are clean > or dirty. Right after mapping or under intense access pattern, they sit > on the active list. If not touched long enough, or cycled through the > buffer cache for io (but ZFS pages not go through buffer cache), they > are moved to inactive. > > > > > > > > > > > You can try to inspect the most outstanding objects adding to the > > > inactive queue with 'vmobject -o' to see where the most of inactive > pages > > > come from. > > > > > > > Wow, that did it! About 99% of the inactive pages come from just a few > > vnodes which are used by the FUSE servers. But I also see a few large > > entries like > > 1105308 333933 771375 1 0 WB df > > what does that signify? > These are anonymous memory. > > > > > > > > > > > If indeed they are double-copy, then perhaps ZFS can react even to the > > > current primitive vm_lowmem signal somewhat different. First, it could > > > do the pass over its vnodes and > > > - free clean unmapped pages > > > - if some targets are not met after that, laundry dirty pages, > > > then return to freeing clean unmapped pages > > > all that before ever touching its cache (ARC). > > > > Follow-up: All of the big inactive-memory consumers were files on FUSE file systems that were being exported as CTL LUNs. ZFS files exported by CTL do not use any res or inactive memory. I didn't test UFS. Curiously, removing the LUN does not free the memory, but shutting down the FUSE daemon does. A valid workaround is to set the vfs.fusefs.data_cache_mode sysctl to 0. That prevents the kernel from caching any data from the FUSE file system. I've tested this on both FreeBSD 12.2 and 13.0 . Should the kernel do a better job of reclaiming inactive memory before ARC? Yes, but in my case it's better not to create so much inactive memory in the first place. Thanks for everybody's help, especially kib's tip about "vmstat -o". -Alan --0000000000006da17e05c2b4ad3b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Tue, May 18, 2021 at 10:17 PM Konstantin Belousov <kostikbel@gmail.com> wrote:
On Tue, May 18, 2021 at 09:5= 5:25PM -0600, Alan Somers wrote:
> On Tue, May 18, 2021 at 9:25 PM Konstantin Belousov <kostikbel@gmail.com>
> > Is your machine ZFS-only?=C2=A0 If yes, then typical source of in= active memory
> > can be of two kinds:
> >
>
> No, there is also FUSE.=C2=A0 But there is typically < 1GB of Buf m= emory, so I
> didn't mention it.
As Mark mentioned, buffers use page cache as second-level cache. More
precisely, there is relatively limited number of buffers in the system,
which are just headers to describe a set of pages. When a buffer is
recycled, its pages are put on inactive queue.

This is why I asked is your machine ZFS-only or not, because io on
bufcache-using filesystems typically add to the inactive queue.

>
>
> > - anonymous memory that apps allocate with facilities like malloc= (3).
> >=C2=A0 =C2=A0If inactive is shrinkable then it is probably not, be= cause dirty pages
> >=C2=A0 =C2=A0from anon objects must go through laundry->swap ro= ute to get evicted,
> >=C2=A0 =C2=A0and you did not mentioned swapping
> >
>
> No, there's no appreciable amount of swapping going on.=C2=A0 Nor = is the laundry
> list typically more than a few hundred MB.
>
>
> > - double-copy pages cached in v_objects of ZFS vnodes, clean or d= irty.
> >=C2=A0 =C2=A0If unmapped, these are mostly a waste.=C2=A0 Even if = mapped, the source
> >=C2=A0 =C2=A0of truth for data is ARC, AFAIU, so they can be dropp= ed as well, since
> >=C2=A0 =C2=A0inactive state means that its content is not hot.
> >
>
> So if a process mmap()'s a file on ZFS and reads from it but never= writes
> to it, will those pages show up as inactive?
It depends on workload, and it does not matter much if the pages are clean<= br> or dirty.=C2=A0 Right after mapping or under intense access pattern, they s= it
on the active list.=C2=A0 If not touched long enough, or cycled through the=
buffer cache for io (but ZFS pages not go through buffer cache), they
are moved to inactive.

>
>
> >
> > You can try to inspect the most outstanding objects adding to the=
> > inactive queue with 'vmobject -o' to see where the most o= f inactive pages
> > come from.
> >
>
> Wow, that did it!=C2=A0 About 99% of the inactive pages come from just= a few
> vnodes which are used by the FUSE servers.=C2=A0 But I also see a few = large
> entries like
> 1105308 333933 771375=C2=A0 =C2=A01=C2=A0 =C2=A00 WB=C2=A0 df
> what does that signify?
These are anonymous memory.

>
>
> >
> > If indeed they are double-copy, then perhaps ZFS can react even t= o the
> > current primitive vm_lowmem signal somewhat different. First, it = could
> > do the pass over its vnodes and
> > - free clean unmapped pages
> > - if some targets are not met after that, laundry dirty pages, > >=C2=A0 =C2=A0then return to freeing clean unmapped pages
> > all that before ever touching its cache (ARC).
> >

Follow-up:
All of t= he big inactive-memory consumers were files on FUSE file systems that were = being exported as CTL LUNs.=C2=A0 ZFS files exported by CTL do not use any = res or inactive memory.=C2=A0 I didn't test UFS.=C2=A0 Curiously, remov= ing the LUN does not free the memory, but shutting down the FUSE daemon doe= s.=C2=A0 A valid workaround is to set the vfs.fusefs.data_cache_mode sysctl= to 0.=C2=A0 That prevents the kernel from caching any data from the FUSE f= ile system.=C2=A0 I've tested this on both FreeBSD 12.2 and 13.0 .=C2= =A0 Should the kernel do a better job of reclaiming inactive memory before = ARC?=C2=A0 Yes, but in my case it's better not to create so much inacti= ve memory in the first place.=C2=A0 Thanks for everybody's help, especi= ally kib's tip about "vmstat -o".
-Alan
--0000000000006da17e05c2b4ad3b-- From nobody Thu May 20 06:44:20 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6E7AE8C7F27 for ; Thu, 20 May 2021 06:44:24 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fm0Zw1803z4cQM; Thu, 20 May 2021 06:44:24 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from [192.168.0.88] (unknown [195.64.148.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: avg/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 63A8026729; Thu, 20 May 2021 06:44:23 +0000 (UTC) (envelope-from avg@FreeBSD.org) Subject: Re: The pagedaemon evicts ARC before scanning the inactive page list To: Alan Somers , Konstantin Belousov Cc: Mark Johnston , FreeBSD Hackers References: From: Andriy Gapon Message-ID: <4122a6cf-2da0-0816-96af-f2058d48435e@FreeBSD.org> Date: Thu, 20 May 2021 09:44:20 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Firefox/78.0 Thunderbird/78.10.1 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit On 19/05/2021 23:28, Alan Somers wrote: > Follow-up: > All of the big inactive-memory consumers were files on FUSE file systems that > were being exported as CTL LUNs.  ZFS files exported by CTL do not use any res > or inactive memory.  I didn't test UFS.  Curiously, removing the LUN does not > free the memory, but shutting down the FUSE daemon does.  A valid workaround is > to set the vfs.fusefs.data_cache_mode sysctl to 0.  That prevents the kernel > from caching any data from the FUSE file system.  I've tested this on both > FreeBSD 12.2 and 13.0 .  Should the kernel do a better job of reclaiming > inactive memory before ARC?  Yes, but in my case it's better not to create so > much inactive memory in the first place.  Thanks for everybody's help, > especially kib's tip about "vmstat -o". Nevertheless, the larger problem still exists. I can confirm it on my system and right now I am not using fusefs at all. I do, however, use some programs that like to mmap some big data a lot. The current pageout + ARC reclaim code really oppresses the ARC. I think that Kostik and Mark made some good suggestion on how that can be fixed. -- Andriy Gapon From nobody Thu May 20 16:37:14 2021 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A25D58B3368 for ; Thu, 20 May 2021 16:37:42 +0000 (UTC) (envelope-from emblue3prd@emark4.embluejet.com) Received: from nit7.embluenitro.com (nit7.embluenitro.com [185.98.146.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FmFlR3QRcz3nKp for ; Thu, 20 May 2021 16:37:38 +0000 (UTC) (envelope-from emblue3prd@emark4.embluejet.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=epexo; d=prototype.com.ar; h=Content-Type:MIME-Version:From:To:Subject:reply-to:List-Unsubscribe: Message-ID:Date; i=info@prototype.com.ar; bh=VoV8j1nG1N7DHDN90pUZ/UsBl6DS5YiCqZCW6/P9H4I=; b=kkPxNDHlExg7HKkmOMXIgjEh3+ecKD7eqZvj2buffi+WG5TyeClqNeBzpDO7rKxeq75VzK8AeLh3 VRS5lvv0rq/nOiXMXnGWL9Lh/HtEmVrsid5H8ek50pYpoTzUSFnpwoOIFUCodMKG+Si0+nz9Orn+ dbHPo6EPqS4RtGuUU3w= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=epexo; d=embluenitro.com; h=Content-Type:MIME-Version:From:To:Subject:reply-to:List-Unsubscribe: Message-ID:Date; bh=VoV8j1nG1N7DHDN90pUZ/UsBl6DS5YiCqZCW6/P9H4I=; b=ZFeqkqE+2OfHxmTRdQciI62igrNoAWfMT1BsVTHcEvj9Yp3xRDug7jOXn/+7kMX8s16PP7/TMp1B zG45J6pNGKT/USeRjy5toEoM9h8kxi6lenohbyoVmj1WZv4E7jUN/pb7T6WAvaASMA9COPCoVQ+c JpWkxAWsXs2eML/eXJs91gOts7TxK87xHqQ2lltx+e6ulYieVcZQrwoX+X4Ludj8Fi2fDli1q/06 x2TCAZm1kXItTV3fThNq0fEGrp/U3Q02Q1zD+Xk78PbE3hVzGSSug5+pM3/A7clFSe6soleIv7ak yruvGOxsFPBhKPv70PA/MzX58nHhBBQjX/0slg== Content-Type: multipart/alternative; boundary="===============1033817644166170155==" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 To: hackers@freebsd.org Subject: =?utf-8?q?=C2=A1Ultimas_Horas!_=F0=9F=A7=A5__Abrigos_20=25_OFF_+_Env?= =?utf-8?q?=C3=ADo_Gratis?= Feedback-ID: 6c1gm:b:email_id:ENVIOSIMPLE:5d1h X-mta: emblue3prd-0 X-emmkt: E;;5h4ik64;;5d1h;;6c1gm:b;;7c7fk List-Unsubscribe: Message-ID: <94M-66962551ee-@embluemail.com> Date: Thu, 20 May 2021 13:37:14 -0300 (-03) X-Rspamd-Queue-Id: 4FmFlR3QRcz3nKp X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=prototype.com.ar header.s=epexo header.b=kkPxNDHl; dkim=pass header.d=embluenitro.com header.s=epexo header.b=ZFeqkqE+; dmarc=pass (policy=quarantine) header.from=prototype.com.ar; spf=pass (mx1.freebsd.org: domain of emblue3prd@emark4.embluejet.com designates 185.98.146.9 as permitted sender) smtp.mailfrom=emblue3prd@emark4.embluejet.com X-Spamd-Result: default: False [-0.81 / 15.00]; HAS_REPLYTO(0.00)[info@prototype.com.ar]; R_SPF_ALLOW(-0.20)[+ip4:185.98.146.0/24:c]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[prototype.com.ar:+,embluenitro.com:+]; SUBJECT_HAS_EXCLAIM(0.00)[]; DMARC_POLICY_ALLOW(-0.50)[prototype.com.ar,quarantine]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[info@prototype.com.ar,emblue3prd@emark4.embluejet.com]; RCVD_COUNT_ZERO(0.00)[0]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[185.98.146.9:from]; ASN(0.00)[asn:51050, ipnet:185.98.144.0/22, country:NL]; FROM_NEQ_ENVFROM(0.00)[info@prototype.com.ar,emblue3prd@emark4.embluejet.com]; R_PARTS_DIFFER(0.02)[51.1%]; RWL_MAILSPIKE_NEUTRAL(0.00)[185.98.146.9:from]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.83)[-0.826]; R_DKIM_ALLOW(-0.20)[prototype.com.ar:s=epexo,embluenitro.com:s=epexo]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HTML_SHORT_LINK_IMG_1(2.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[185.98.146.9:from:127.0.2.255]; MANY_INVISIBLE_PARTS(0.70)[8]; MAILMAN_DEST(0.00)[hackers] Reply-To: info@prototype.com.ar, info@prototype.com.ar From: Prototype via hackers X-Original-From: Prototype X-Spam: Yes --===============1033817644166170155== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Su Cliente de Mail NO soporta mensajes en formato HTML. Para ver correctamente el contenido del correo COPIE y PEGUE la siguiente U= RL en su Navegador Web (Chrome / Internet Explorer / FireFox / Safari)=20 https://app.embluemail.com/Online/VON.aspx?data=3DSu%2BRB1%2BU%2BHs2xMlIRYx= IVEZ7cwehoT29YbgQZ4klObc60fSqolNRcaKCEgMs%2Fz7zoCs7t8scUuPq0W%2BaQUXIMaUXtt= vLCtBu8NqJZsCDW33vSb9Z6rIIHg%2Bt8EX412y5!-!6qZcJeqz4RtsTQuLqvgs3scQNIzCHYzQ= AfXYYnde/3YN19nkgyvhxVmw/pWokZbN --===============1033817644166170155== Content-Type: text/html; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable

Ver en mi navegador

3D"Prototype=
TOPS BOTTOMS  &= nbsp; NEW= IN
= 3D"20%
¡APROVEC= HÁ LAS 3 y 6 CUOTAS SIN INTERÉS!
3D""
 
6 cuotas sin interés
3D""
 
+$8000 envíos gratis
3D""
 
Política de ca= mbios
 
Se= guinos en:
3D"Instagram"   3D"Facebook"   3D"Twitter"   3D"Pinterest"   3D"Youtube"   3D"Facebook"
 
T&eac= ute;rminos y Condiciones
=   |   Política de devolución
  |&n= bsp; Preguntas frecue= ntes
Franquicias=
&n= bsp; |   Locales<= /a>
&nbs= p; |  Contacto
 
3D"" =20 =20 =20

En virtud de lo establecido por la disposició= n de Protección de Datos Personales usted tiene derecho a solicitar = al emisor de este mensaje la rectificación, actualización, in= clusión o supresión de los datos personales incluidos en su b= ase de contactos, listas o cadenas de mensajes en los cuales usted se encue= ntre. Conozca más
Quiero desuscribirme |=20 Actualizar perfil | Términos y condi= ciones

--===============1033817644166170155==-- From nobody Thu May 20 21:53:25 2021 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4299E621444 for ; Thu, 20 May 2021 23:06:22 +0000 (UTC) (envelope-from richardmorrison0263@gmail.com) Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FmQMx0jYjz3phk for ; Thu, 20 May 2021 23:06:20 +0000 (UTC) (envelope-from richardmorrison0263@gmail.com) Received: by mail-io1-xd29.google.com with SMTP id a8so10199602ioa.12 for ; Thu, 20 May 2021 16:06:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=ZWeVAxUmGVJAISuIBZnVwWTPqC0e1MABiHW81X0ZhYU=; b=cw7sSoh/FSJtlV4joP/bDWllfYdVUEb3+TXx4Ly0/k5bh13IBa9H1uvudhFasBbOIs jL/cM6dyM6YozXPsaBcITFbO4tTp4sHX9Ov7byIXFt/SbaXC3gCbXCx5qZqQKsvmNeNd 4JnPGCdccG3SuOKycdSzMfFEtH4PYFc7pF6BAMMM5miRS4MfGGW3fjKd9fRPHBqCmORn LqqeowSPINvBSjPNSBW5dbxHejkwYucwQ3Fg+4HPOWrr9FzUORWNcPnwUUB6Dhzibn/r R4P9OhdzHKQLHsnZ2MOMAJLKlLOYtH0Ar7RabhiZzPHmYbJxC0yAXglxISNSxfKN9nzW kVng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=ZWeVAxUmGVJAISuIBZnVwWTPqC0e1MABiHW81X0ZhYU=; b=AvsamBWjpHfG6/ZgCzTcD+EcfETfatTgqs4kb7akfhLz+lztXQWLS9BChET/NMkXDO +4CYTmQ36Ru7s09LTIkalaDjBNdQ8PNKdUNidtB07lSoZ1mvQM9KBLy6dowbqyBbQwXL SFGaKEmUimKTPFsEPYDP8UMLxajsl19XTcXFuPtBPdMA0ssODyhm6TYL1/p2KfWdCupv WnA+6nRPQcQNB3s9dNLXj3bugj6jYXlpiSxafFTn7ParOjZ5SiF9Oxi3PwHkOGTdyoH+ v6fpggOQmZS2RhSVKhRpU2Cm21B5QMVl9nxqabsCFzGujz1n+ICwzeVDec2rqyy/Jrvf h0GQ== X-Gm-Message-State: AOAM533a0+q28tJ4WFkL2aaB5oWO1X7j+4u/5Gnagxm9o8vj+aUDs3t5 O4zK8MFCPlO5/UXfamk7x2ES42sj2oIBy2MM1piFOR/ZHeqCnCzE X-Google-Smtp-Source: ABdhPJxhUw5uCbFr9mF6WDv5yoJLMo8B5ZK4k5h40MJ9SVBYw5PKRVFvH2XaC+MUDdutNjYO9E+FPWgzb+1M99tYt3Y= X-Received: by 2002:a05:6602:cd:: with SMTP id z13mr8064085ioe.199.1621551979949; Thu, 20 May 2021 16:06:19 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 From: richard morrison Date: Fri, 21 May 2021 05:53:25 +0800 Message-ID: Subject: Order To: hackers@freebsd.org Content-Type: multipart/alternative; boundary="000000000000bff63c05c2cafd97" X-Rspamd-Queue-Id: 4FmQMx0jYjz3phk X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=cw7sSoh/; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of richardmorrison0263@gmail.com designates 2607:f8b0:4864:20::d29 as permitted sender) smtp.mailfrom=richardmorrison0263@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; INTRODUCTION(2.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::d29:from]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::d29:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::d29:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[hackers] --000000000000bff63c05c2cafd97 Content-Type: text/plain; charset="UTF-8" Hello My name is Richard Morrison i send this inquiry to your company in regards to order some (handlebar mirror) i will like to know the types and sizes of (handlebar mirror) you have. Immediate responds is require and advise on payment method. Morrison Warehouse Inc 170 Ayer Rd, Littleton, MA 01460 E-mail:richardmorrison0263@gmail.com Quote:"Try not to become a man of success. Rather become a man of value." --000000000000bff63c05c2cafd97 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Hello My name is Richard Morrison i send this inquiry = to your company in regards to order some (handlebar mirror) i will like to = know the types and sizes of (handlebar mirror) you have. Immediate responds= is require and advise on payment method.

Morrison Warehouse Inc
= 170 Ayer Rd, Littleton,
MA 01460
E-mail:richardmorrison0263@gmail.com
Quote:&qu= ot;Try not to become a man of success. Rather become a man of value."<= br>
--000000000000bff63c05c2cafd97-- From nobody Fri May 21 05:10:48 2021 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2DB698C12D9 for ; Fri, 21 May 2021 05:28:56 +0000 (UTC) (envelope-from 010001798d554f6e-b07e70dc-ef8e-45f9-867d-2203bed25065-000000@mail.businessdiscoverydatabase.com) Received: from a48-183.smtp-out.amazonses.com (a48-183.smtp-out.amazonses.com [54.240.48.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FmZsL6lCmz3sfP for ; Fri, 21 May 2021 05:28:54 +0000 (UTC) (envelope-from 010001798d554f6e-b07e70dc-ef8e-45f9-867d-2203bed25065-000000@mail.businessdiscoverydatabase.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=flodtspall4krzaydk3v62i3rbjiuuoj; d=businessdiscoverydatabase.com; t=1621573849; h=Subject:From:To:Date:Mime-Version:Content-Type:In-Reply-To:References:Message-Id; bh=4ebeE1JQwS2sNSHA+cRP7Kh5ImDLC8hEWiW1FONaPQM=; b=gDBO2kgn7gNqrdTfwEaNlMJJX3N7M1lwkdAjDZfJYOBY+Ig0goqpauuOc2P6MFaW JjJilB9bfQNfkJq27IK1iSEPsqKMl+KkZOTDlv3zihb3Vs7gDgD6GQ/YAqrxrRWZsEH Ba30hiagddQFjjlsF/GY9DVZ2yc034iJIXcNITIk= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=ug7nbtf4gccmlpwj322ax3p6ow6yfsug; d=amazonses.com; t=1621573849; h=Subject:From:To:Date:Mime-Version:Content-Type:In-Reply-To:References:Message-Id:Feedback-ID; bh=4ebeE1JQwS2sNSHA+cRP7Kh5ImDLC8hEWiW1FONaPQM=; b=WTQLQypm8frBvLy3QNUoFySuApDMV7/Nsm4VPhb1vPhpoixXXR/9kBIZ73ngV9aP NnylyszR91zEMQdVJN3xT3ykS/KWtSpK/e+LFBWVEcmM+Bqx9kKorfKcZ2/DkN5RuJb LL4sEJkVvi44u1p/HK0j6EEIKWtk1sgne3GsWjoU= Subject: RE: Medical Practice Management Software Users To: =?UTF-8?Q?hackers=40freebsd=2Eorg?= Date: Fri, 21 May 2021 05:10:48 +0000 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="=_o3QF2duAe8MTyk-+LI17FGaf0R3W1jj3DHC0Oe6KvIjCW07-" In-Reply-To: References: X-Priority: 3 (Normal) X-Mailer: Amazon WorkMail Thread-Index: AdbHJVeCN+ZJT7k+RbSulrVcenKY6CG2i/1w Thread-Topic: Medical Practice Management Software Users Disposition-Notification-To: =?UTF-8?Q?Daisy_Fitzgerald?= X-Wm-Sent-Timestamp: 1621573848 Message-ID: <010001798d554f6e-b07e70dc-ef8e-45f9-867d-2203bed25065-000000@email.amazonses.com> Feedback-ID: 1.us-east-1.LF00NED762KFuBsfzrtoqw+Brn/qlF9OYdxWukAhsl8=:AmazonSES X-SES-Outgoing: 2021.05.21-54.240.48.183 X-Rspamd-Queue-Id: 4FmZsL6lCmz3sfP X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=pass header.d=businessdiscoverydatabase.com header.s=flodtspall4krzaydk3v62i3rbjiuuoj header.b=gDBO2kgn; dkim=pass header.d=amazonses.com header.s=ug7nbtf4gccmlpwj322ax3p6ow6yfsug header.b=WTQLQypm; dmarc=pass (policy=quarantine) header.from=businessdiscoverydatabase.com; spf=pass (mx1.freebsd.org: domain of 010001798d554f6e-b07e70dc-ef8e-45f9-867d-2203bed25065-000000@mail.businessdiscoverydatabase.com designates 54.240.48.183 as permitted sender) smtp.mailfrom=010001798d554f6e-b07e70dc-ef8e-45f9-867d-2203bed25065-000000@mail.businessdiscoverydatabase.com X-Spamd-Result: default: False [1.54 / 15.00]; XM_UA_NO_VERSION(0.01)[]; RWL_MAILSPIKE_GOOD(0.00)[54.240.48.183:from]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:54.240.0.0/18]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DKIM_TRACE(0.00)[businessdiscoverydatabase.com:+,amazonses.com:+]; DMARC_POLICY_ALLOW(-0.50)[businessdiscoverydatabase.com,quarantine]; HAS_X_PRIO_THREE(0.00)[3]; FROM_EXCESS_QP(1.20)[]; FORGED_SENDER(0.30)[daisy.fitzgerald@businessdiscoverydatabase.com,010001798d554f6e-b07e70dc-ef8e-45f9-867d-2203bed25065-000000@mail.businessdiscoverydatabase.com]; RCVD_COUNT_ZERO(0.00)[0]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[54.240.48.183:from]; ASN(0.00)[asn:14618, ipnet:54.240.48.0/23, country:US]; FROM_NEQ_ENVFROM(0.00)[daisy.fitzgerald@businessdiscoverydatabase.com,010001798d554f6e-b07e70dc-ef8e-45f9-867d-2203bed25065-000000@mail.businessdiscoverydatabase.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.67)[-0.669]; R_DKIM_ALLOW(-0.20)[businessdiscoverydatabase.com:s=flodtspall4krzaydk3v62i3rbjiuuoj,amazonses.com:s=ug7nbtf4gccmlpwj322ax3p6ow6yfsug]; FROM_HAS_DN(0.00)[]; TO_EXCESS_QP(1.20)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HEADER_FORGED_MDN(2.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[54.240.48.183:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[54.240.48.183:from]; TO_DN_EQ_ADDR_ALL(0.00)[]; GREYLIST(0.00)[pass,body]; MAILMAN_DEST(0.00)[hackers] Reply-To: daisy.fitzgerald@businessdiscoverydatabase.com From: Daisy Fitzgerald via hackers X-Original-From: Daisy Fitzgerald X-Spam: Yes This is a multi-part message in MIME format. Your mail reader does not understand MIME message format. --=_o3QF2duAe8MTyk-+LI17FGaf0R3W1jj3DHC0Oe6KvIjCW07- Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi, =C2=A0 Apologies for interrupting your busy schedule, I'm following up to check = if you had a chance to review my previous email. Do let me know your inte= rest and we shall provide you further details for your perusal. =C2=A0 Waiting for your response. =C2=A0 Best Regards, =C2=A0 Daisy Fitzgerald =C2=A0| Direct marketing manager =C2=A0 =C2=A0 From: Daisy Fitzgerald=20 Sent: Monday, November 30, 2020 9:40 AM To: hackers@freebsd.org Subject: Medical Practice Management Software Users =C2=A0 Hi, =C2=A0 I am writing to you in regards to our recent lists released and check if = you would be interested in acquiring our recently verified Medical Practi= ce Management Software Users to promote your product/services or for any = marketing initiatives=3F =C2=A0 =C2=A0The list has been based on the following client=E2=80=99s requireme= nts:=C2=A0 =C2=A0 =C3=BC=C2=A0 Electronic Medical Records Software =C3=BC=C2=A0 Medical Billing Software =C3=BC=C2=A0 Medical Scheduling Software =C3=BC=C2=A0 Hospital Management Software =C3=BC=C2=A0 Chiropractic Software =C3=BC=C2=A0 Medical Transcription Software =C3=BC=C2=A0 Mental Health Software =C3=BC=C2=A0 Patient Case Management Software =C3=BC=C2=A0 Pediatric Software =C3=BC=C2=A0 Patient Management Software =C3=BC=C2=A0 Waiver Software, End Users and many more across the globe. =C2=A0 Let me know your thoughts on your target requirement and I can assist you= with more info and sample file.=C2=A0Or can we have a quick call schedul= ed on your time=3F =C2=A0 Looking forward to your response =C2=A0 Best Regards, =C2=A0 Daisy Fitzgerald | Business Development Manager Appending | De-Duping |SEO| Content Management As this is not an auto generated email, to discontinue receiving email fr= om us reply as "Exclude" _________________________________________________________________________= ___________ Confidentiality Notice: This email message and any files transmitted with= it may contain confidential information intended only for the person(s) = to whom this email is addressed. If you have received this email in error= , please notify the sender immediately by phone or email and destroy the = original message without making a copy. Thank you. --=_o3QF2duAe8MTyk-+LI17FGaf0R3W1jj3DHC0Oe6KvIjCW07- Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: quoted-printable

Hi,

 =

Apologies for interrupting your bu= sy schedule, I'm following up to check if you had a chance to review my p= revious email. Do let me know your interest and we shall provide you furt= her details for your perusal.

<= span style=3D'font-size:14.0pt;color:black'> 

<= p class=3DMsoNormal>Waiting for your response= =2E

 

Best Regards,

 

Daisy Fitzgerald  | D= irect marketing manager

 

 

=

From: Daisy Fitzgerald
Sent= : Monday, November 30, 2020 9:40 AM
To: hackers@freebsd.org=
Subject: Medical Practice Management Software Users=

 

Hi,

 <= /p>

I am writing to you in regards to our recent lists released and c= heck if you would be interested in acquiring our recently verified Med= ical Practice Management Software Users to promote your produc= t/services or for any marketing initiatives=3F

 

 = The list has been based on the following cli= ent’s requirements: 

&= nbsp;

ü = Electronic Medical Records Software=

= ü  <= /span>Medical Billing Software

ü  Medical S= cheduling Software

üHospital Management Software

= ü  <= /span>Chiropractic Software

ü  Medical Transcri= ption Software

ü&nbs= p; Mental Health Software

ü  Patient Case Management Software

ü  Pediatric S= oftware

<= span style=3D'font-size:12.0pt;font-family:Wingdings'>ü  Patient Management Software

ü  Waiver Software, End Users and many more across the globe.

 

Let me know your thoughts on your target requirement and I can as= sist you with more info and sample file. Or can we have a quick call= scheduled on your time=3F

 = ;

Looking forward to your response

 

B= est Regards,

 =

Daisy Fitzgerald | Business Development Manager

App= ending | De-Duping |SEO| Content Management

As this is not= an auto generated email, to discontinue receiving email from us reply as= "Exclude"

= ________________________________________________________________________= ____________=

Confidentiality Notice: This email me= ssage and any files transmitted with it may contain confidential informat= ion intended only for the person(s) to whom this email is addressed. If y= ou have received this email in error, please notify the sender immediatel= y by phone or email and destroy the original message without making a cop= y. Thank you.

--=_o3QF2duAe8MTyk-+LI17FGaf0R3W1jj3DHC0Oe6KvIjCW07--- From nobody Fri May 21 07:35:20 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id B28DC8B2EAD for ; Fri, 21 May 2021 07:35:23 +0000 (UTC) (envelope-from 010b01798dd99fac-b91c5520-eba3-4412-8e78-bce1f3982b2c-000000@eu-west-2.amazonses.com) Received: from d218-11.smtp-out.eu-west-2.amazonses.com (d218-11.smtp-out.eu-west-2.amazonses.com [23.249.218.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FmdgG0BWDz3JbS for ; Fri, 21 May 2021 07:35:21 +0000 (UTC) (envelope-from 010b01798dd99fac-b91c5520-eba3-4412-8e78-bce1f3982b2c-000000@eu-west-2.amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=4yvrckfjx4ar4pcfrcu7cigcpdet2ekr; d=espertiseo.eu; t=1621582520; h=Message-Id:Mime-Version:From:To:Subject:Date:Content-Type; bh=rrF41sSbf4PffXuM4VNcuOwD9lgxTV8moZ3LaiEif90=; b=ERGumP8qoAhkd4o1hCMhlRDSJk/6lVaRuTYonMCiusgGw5jUSdqP9nxzyNilPafL L/iHdGzYLYy8m7EApA1g88CfXwXUlENMtNLbRea2Nniq72/iljmN8feFNRFVWwlMTu4 io8MsTqdciCawJgLK7ptBLJ5EySTLzEJygUIofh4= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=pgxy5mtxzx6eoyytua4nvvg26jbuf6lj; d=amazonses.com; t=1621582520; h=Message-Id:Mime-Version:From:To:Subject:Date:Content-Type:Feedback-ID; bh=rrF41sSbf4PffXuM4VNcuOwD9lgxTV8moZ3LaiEif90=; b=K5RNzMz0WLO/ouKNfes+xHjb97naaLPUKGj/HuhdEWJB65yAT2yezg2tBU1ptqro YzMcaBTlPNoRayZi5rr9bCSY/yumtxNX8JuP16MLUH6w9Wxem3xSahs2bfEgDRwkhSy nTj5+OPBgsWb70w+CsKqHHh5juoE43oMRh08a0hs= Message-ID: <010b01798dd99fac-b91c5520-eba3-4412-8e78-bce1f3982b2c-000000@eu-west-2.amazonses.com> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 From: Mailtarget To: freebsd-hackers Subject: =?utf-8?Q?Clienti_interessati_ai_tuoi_prodotti_o_servizi_a_0.009=E2=82=AC?= Date: Fri, 21 May 2021 07:35:20 +0000 Content-Type: multipart/alternative; Boundary="--=BOUNDARY_521935_CGTN_SUOG_PVSA_VSRG" X-Antivirus: Avast (VPS 210520-4, 20/05/2021), Outbound message X-Antivirus-Status: Clean Feedback-ID: 1.eu-west-2.ggSdn/Xz5D9+Ba7b0nv9xm3SNaIUrE7jaogVatTX5Fg=:AmazonSES X-SES-Outgoing: 2021.05.21-23.249.218.11 X-Rspamd-Queue-Id: 4FmdgG0BWDz3JbS X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=espertiseo.eu header.s=4yvrckfjx4ar4pcfrcu7cigcpdet2ekr header.b=ERGumP8q; dkim=pass header.d=amazonses.com header.s=pgxy5mtxzx6eoyytua4nvvg26jbuf6lj header.b=K5RNzMz0; dmarc=none; spf=pass (mx1.freebsd.org: domain of 010b01798dd99fac-b91c5520-eba3-4412-8e78-bce1f3982b2c-000000@eu-west-2.amazonses.com designates 23.249.218.11 as permitted sender) smtp.mailfrom=010b01798dd99fac-b91c5520-eba3-4412-8e78-bce1f3982b2c-000000@eu-west-2.amazonses.com X-Spamd-Result: default: False [0.30 / 15.00]; RWL_MAILSPIKE_GOOD(0.00)[23.249.218.11:from]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:23.249.208.0/20]; SUBJECT_HAS_CURRENCY(1.00)[]; URI_COUNT_ODD(1.00)[7]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[espertiseo.eu:+,amazonses.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[contatti@espertiseo.eu,010b01798dd99fac-b91c5520-eba3-4412-8e78-bce1f3982b2c-000000@eu-west-2.amazonses.com]; RCVD_COUNT_ZERO(0.00)[0]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[23.249.218.11:from]; ASN(0.00)[asn:16509, ipnet:23.249.218.0/23, country:US]; FROM_NEQ_ENVFROM(0.00)[contatti@espertiseo.eu,010b01798dd99fac-b91c5520-eba3-4412-8e78-bce1f3982b2c-000000@eu-west-2.amazonses.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[espertiseo.eu:s=4yvrckfjx4ar4pcfrcu7cigcpdet2ekr,amazonses.com:s=pgxy5mtxzx6eoyytua4nvvg26jbuf6lj]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[espertiseo.eu]; HTML_SHORT_LINK_IMG_2(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[23.249.218.11:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[23.249.218.11:from]; MAILMAN_DEST(0.00)[freebsd-hackers] X-Spam: Yes Questo messaggio è in formato MIME. Poichè il tuo programma di posta non comprende questo formato, una porta o tutto il messaggio potrebbe non essere leggibile. ----=BOUNDARY_521935_CGTN_SUOG_PVSA_VSRG Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Buongiorno, se avete un prodotto o servizio da vendere, noi consegniamo la = vostra offerta solo a contatti qualificati, profilati e geolocalizzati in b= ase al vostro target. Vi garantiamo il 100% di consegna del messaggio e garantiamo che il messagg= io venga letto. Pagate appena 0,009=E2=82=AC per ogni cliente contattato e = se acquistate piani da 1 Milione di clienti in su, avrete sconti fino all= =E2=80=9980%. Noi suggeriamo di acquistare un piano pi=C3=B9 alto per pagare molto meno e= suddividere gli invii in pi=C3=B9 lanci. Aumenterete cos=C3=AC il numero d= ei clienti che vi contatteranno ad un costo-cliente molto pi=C3=B9 basso. Se mi indica il target e la GEO che Le interessa Le quantifico quanti conta= tti possiamo estrapolare. Facciamo campagne per oltre 1.500 Clienti e 500 A= gency e siamo certificati ISO9001 e ISO27001. Resto a disposizione sia tramite Whatsapp al numero 3939316379 sia tramite = il fisso 0287168036 che tramite la Live Chat sul nostro sito Mailtarget. Antonio Morlino Tel. (+39) 393-9316379 Tel. (+39) 02 87168036 Mailtarget by Seometrics P.IVA IT04074570716 Ai sensi degli artt.15 diritto di accesso, 16 diritto di rettifica, 17 diri= tto alla cancellazione, 18 diritto alla limitazione del trattamento, 20 dir= itto alla portabilit=C3=A0, 21-22 diritto all=E2=80=99opposizione e 22 diri= tto di opposizione del GDPR 679/16 cliccando http://eoaclk.com/ttNGnujoSf/ = cancelleremo definitivamente i suoi dati -- Questa e-mail =C3=A8 stata controllata per individuare virus con Avast anti= virus. https://www.avast.com/antivirus ----=BOUNDARY_521935_CGTN_SUOG_PVSA_VSRG Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable HTML Message
Buongiorno, se avete un prodo= tto o servizio da vendere, noi consegniamo la vostra offerta solo a contatt= i qualificati, profilati e geolocalizzati in base al vostro target. =

Vi garantiamo il 100% di cons= egna del messaggio e garantiamo che il messaggio venga letto. Pagate appena= 0,009€ per ogni cliente contattato e se acquistate piani da 1 Milione= di clienti in su, avrete sconti fino all’80%.

Noi suggeriamo di acquistare = un piano pi=C3=B9 alto per pagare molto meno e suddividere gli invii in pi= =C3=B9 lanci. Aumenterete cos=C3=AC il numero dei clienti che vi contattera= nno ad un costo-cliente molto pi=C3=B9 basso.

Se mi indica il target e la G= EO che Le interessa Le quantifico quanti contatti possiamo estrapolare. Fac= ciamo campagne per oltre 1.500 Clienti e 500 Agency e siamo certificati ISO= 9001 e ISO27001.

Resto a disposizione sia tram= ite Whatsapp al numero 3939316379 sia tramite il fisso 0287168036 che trami= te la Live Chat sul nostro sito Mailtarget.

Antonio Morlino
Tel. (+39) 393-9316379=
Tel. (+39) 02 87168036=
Mailtarget by Seometrics
P.IVA IT04074570716

Ai sensi degli artt.15 diri= tto di accesso, 16 diritto di rettifica, 17 diritto alla cancellazione, 18 = diritto alla limitazione del trattamento, 20 diritto alla portabilit=C3=A0,= 21-22 diritto all=E2=80=99opposizione e 22 diritto di opposizione del GDPR= 679/16 cliccando qui cancell= eremo definitivamente i suoi dati

3D"" Mail p= riva di virus. www.avast.com
=
----=BOUNDARY_521935_CGTN_SUOG_PVSA_VSRG-- From nobody Fri May 21 15:56:37 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A92068CB7EB for ; Fri, 21 May 2021 15:56:41 +0000 (UTC) (envelope-from rollingbits@gmail.com) Received: from mail-qv1-xf2e.google.com (mail-qv1-xf2e.google.com [IPv6:2607:f8b0:4864:20::f2e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fmrnj004Jz3FDl for ; Fri, 21 May 2021 15:56:40 +0000 (UTC) (envelope-from rollingbits@gmail.com) Received: by mail-qv1-xf2e.google.com with SMTP id o59so10646700qva.1 for ; Fri, 21 May 2021 08:56:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=content-transfer-encoding:from:mime-version:date:subject:message-id :to; bh=2OSkP7cNasXmWyIpf0DQLNw9soo4m6FlylngBx5e5a4=; b=uw+6EnkD2J7/E5hsYY+6/IdifGcAe0cnFbWJNlcCCorxkwvdSkDSHNxQvLW9JfZxad s1y3vK8Dj3Pe3MWr33Ys96RfI4JR46EALOb1PamMj+bBKtz02rdtT/Z3LdIhDjpQorOR n/3udiR/81cVOdm5M+67E2ET0mzdIXrArdGEoNkjBNtNmBVO03sPJv1hus8en1fA1k8J xkZzzZh2odm7muGvVUU0to+iAnF2o9V5TJD+zyTJxdBre6d6ajJC32g3+cW9at+3eOFm F2t5luMg2FYNPcj8NooLAB8E4U4SZIkWKzeTW41DXMj65ulYdhRELDFmRkA5EVYq4bQw iXCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version:date :subject:message-id:to; bh=2OSkP7cNasXmWyIpf0DQLNw9soo4m6FlylngBx5e5a4=; b=AJIO07s1Wu5JCPcKv/LEvXeNiqhqMp57hJWqwoh2UeiMFXM9kQLLyF2NszPun3GGvD ESMvGr6CaUVpsgcFJnpkyqKbwQwRg0qbW3eEnbtNABapcCALC0wygea31QMXGcNeoco6 JzrOh8yZpfjzw+fdMbilb11uM77T+dwhOP+pDLhKwzVsLcmkKlO1XKNoACUGTj9TOfu6 zCrd8Zm9nBpXxy3MRAzQVgBFiqWaW++Bk19QENzN9rP0P0q/OOZv4l+bSXq6BvK2epxB DRzGmbKhZTfApl7aqfYyfGXp27ZukfpuwoYC27qnTs+y8ngkrG5ptS90e/zEhfF7kmqS Lz8g== X-Gm-Message-State: AOAM530sZNxPfOg/Sl6AOFtSIq66zynq4SzV1SfgRIp8rdicCNuB+4ZA Uzta4+RgcMOLZ1U8Tiat/YRVEcvcYSNuvg== X-Google-Smtp-Source: ABdhPJzOo6TCaqALkDlrZ6mO1ERIavFCvDmH7fyUAmWdKuSfGrEu7C+35mm/ywxTw27uE+TthLbiRA== X-Received: by 2002:a0c:b292:: with SMTP id r18mr13640273qve.57.1621612600005; Fri, 21 May 2021 08:56:40 -0700 (PDT) Received: from smtpclient.apple ([2804:389:202c:e01f:442:b535:5d1c:ac4e]) by smtp.gmail.com with ESMTPSA id p14sm5123771qki.27.2021.05.21.08.56.39 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 21 May 2021 08:56:39 -0700 (PDT) Content-Type: multipart/alternative; boundary=Apple-Mail-ED5FF75F-5F8A-4984-A1BC-2E3B412D3FD3 Content-Transfer-Encoding: 7bit From: =?utf-8?Q?Lucas_Nali_de_Magalh=C3=A3es?= List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (1.0) Date: Fri, 21 May 2021 12:56:37 -0300 Subject: New? MTA problems Message-Id: To: FreeBSD Hackers X-Mailer: iPhone Mail (18E212) X-Rspamd-Queue-Id: 4Fmrnj004Jz3FDl X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=uw+6EnkD; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rollingbits@gmail.com designates 2607:f8b0:4864:20::f2e as permitted sender) smtp.mailfrom=rollingbits@gmail.com X-Spamd-Result: default: False [-2.74 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_MIXED_CHARSET(0.71)[subject]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; SUBJECT_HAS_QUESTION(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::f2e:from]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.96)[-0.956]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::f2e:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::f2e:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] --Apple-Mail-ED5FF75F-5F8A-4984-A1BC-2E3B412D3FD3 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello everyone, I hope that by writing here someone will be able to help me. Recently I star= ted receiving mail bounces from FreeBSD servers from mails I didn't send. I t= hink the servers changed the behavior and started not just filtering the spa= m messages but also bouncing them back. It happens the information in the he= ader of those messages are all fake and the actual messages end in the wrong= mail box. I hope someone can fix the problem. On my side, everything is up t= o date and I'd no need to change any configuration recently. TIA, --=20 rollingbits =E2=80=94 =F0=9F=93=A7 rollingbits@icloud.com =F0=9F=93=A7 rolli= ngbits@gmail.com =F0=9F=93=A7 rollingbits@yahoo.com =F0=9F=93=A7 rollingbits= @terra.com.br =F0=9F=93=A7 rollingbits@globo.com= --Apple-Mail-ED5FF75F-5F8A-4984-A1BC-2E3B412D3FD3 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello everyone,

I hope t= hat by writing here someone will be able to help me. Recently I started rece= iving mail bounces from FreeBSD servers from mails I didn't send. I think th= e servers changed the behavior and started not just filtering the spam messa= ges but also bouncing them back. It happens the information in the header of= those messages are all fake and the actual messages end in the wrong mail b= ox. I hope someone can fix the problem. On my side, everything is up to date= and I'd no need to change any configuration recently.

TIA,

-- 
rollingbits =E2=80= =94 =F0=9F=93= =A7 rollingbits@icloud.com =F0=9F=93=A7 roll= ingbits@gmail.com =F0=9F=93=A7 rollingbits@yahoo.com =F0=9F=93=A7 rollingbits@terra.com.br =F0=9F=93=A7 rollingbits@globo.com
= --Apple-Mail-ED5FF75F-5F8A-4984-A1BC-2E3B412D3FD3-- From nobody Fri May 21 17:07:24 2021 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 22C698B56F4 for ; Fri, 21 May 2021 17:07:51 +0000 (UTC) (envelope-from emblue3prd@emark4.embluejet.com) Received: from nit7.embluenitro.com (nit7.embluenitro.com [185.98.146.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FmtMf1vJ4z4W0s for ; Fri, 21 May 2021 17:07:41 +0000 (UTC) (envelope-from emblue3prd@emark4.embluejet.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=epexo; d=prototype.com.ar; h=Content-Type:MIME-Version:From:To:Subject:reply-to:List-Unsubscribe: Message-ID:Date; i=info@prototype.com.ar; bh=qYoHdqjR9kAbO+sX2/Ns50kiwZ0BHkuxPyxPUCCpu7k=; b=eRmYvwNepVAAzlHBSH/H1fyN85yIWXeo587XNDBduy+LikVk+ZCiG5ksvFgRAu+0f5O/2B1J8eco RsA4ehZygXbXXIHkIvkHH7yMR+6V1uvnrgfq3YHuAUQnSKSkqko9ZP7wVOAZumo3VSq+j4aisCx/ UY6Or0tllf1bf8amwu4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=epexo; d=embluenitro.com; h=Content-Type:MIME-Version:From:To:Subject:reply-to:List-Unsubscribe: Message-ID:Date; bh=qYoHdqjR9kAbO+sX2/Ns50kiwZ0BHkuxPyxPUCCpu7k=; b=O48AgdL428ef7QnETmjKbkoeNUHwXW1MiuayPu6WbRqv3krX0MH5f/HBbZBNe429DzQt0DLK/aQt cCkg37UBs70i2Dj1uq/PsHkbpLA+OYHIngyPRExtFvp6ZCPBghz1/WunWds/h2uyoZPO2c/09FSC k7qNyhOEn8VIX196FeYJLErHnwULXbrXZ3XdO68gOfz8y58czDlAKlCVF76J6YMAaU3NQjSyKvM+ YqvABNtJ7O3SYAnBmbW+dgw47Jqx5aRu7SJY0CQql2ZGIepUZY+ho3D8/wg+0Wz0RzyUFHvquRz5 cPDYhVGaMhExJlFAlMpIEcgQ7r7fx2f+FNeIMQ== Content-Type: multipart/alternative; boundary="===============5204116918946091560==" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 To: hackers@freebsd.org Subject: =?utf-8?q?Weekend_Sale!_=F0=9F=98=B1__30=25_OFF_en_Camisas_=C2=A1y_Env?= =?utf-8?q?=C3=ADo_Gratis!?= Feedback-ID: 6c2,kb6:email_id:ENVIOSIMPLE:5d1h X-mta: emblue3prd-5 X-emmkt: E;;5h5am:6;;5d1h;;6c2,kb6;;7c7fk List-Unsubscribe: Message-ID: <94M-a369cf6c5c-@embluemail.com> Date: Fri, 21 May 2021 14:07:24 -0300 (-03) X-Rspamd-Queue-Id: 4FmtMf1vJ4z4W0s X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=prototype.com.ar header.s=epexo header.b=eRmYvwNe; dkim=pass header.d=embluenitro.com header.s=epexo header.b=O48AgdL4; dmarc=pass (policy=quarantine) header.from=prototype.com.ar; spf=pass (mx1.freebsd.org: domain of emblue3prd@emark4.embluejet.com designates 185.98.146.9 as permitted sender) smtp.mailfrom=emblue3prd@emark4.embluejet.com X-Spamd-Result: default: False [0.00 / 15.00]; HAS_REPLYTO(0.00)[info@prototype.com.ar]; R_SPF_ALLOW(-0.20)[+ip4:185.98.146.0/24:c]; ZERO_FONT(0.50)[5]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[prototype.com.ar:+,embluenitro.com:+]; DMARC_POLICY_ALLOW(-0.50)[prototype.com.ar,quarantine]; NEURAL_HAM_SHORT(-1.00)[-1.000]; SUBJECT_ENDS_EXCLAIM(0.00)[]; FORGED_SENDER(0.30)[info@prototype.com.ar,emblue3prd@emark4.embluejet.com]; RCVD_COUNT_ZERO(0.00)[0]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[185.98.146.9:from]; ASN(0.00)[asn:51050, ipnet:185.98.144.0/22, country:NL]; FROM_NEQ_ENVFROM(0.00)[info@prototype.com.ar,emblue3prd@emark4.embluejet.com]; R_PARTS_DIFFER(0.15)[57.3%]; RWL_MAILSPIKE_NEUTRAL(0.00)[185.98.146.9:from]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.93)[-0.933]; R_DKIM_ALLOW(-0.20)[prototype.com.ar:s=epexo,embluenitro.com:s=epexo]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HTML_SHORT_LINK_IMG_1(2.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[185.98.146.9:from:127.0.2.255]; MANY_INVISIBLE_PARTS(1.00)[10]; MAILMAN_DEST(0.00)[hackers] Reply-To: info@prototype.com.ar, info@prototype.com.ar From: Prototype via hackers X-Original-From: Prototype X-Spam: Yes --===============5204116918946091560== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Su Cliente de Mail NO soporta mensajes en formato HTML. Para ver correctamente el contenido del correo COPIE y PEGUE la siguiente U= RL en su Navegador Web (Chrome / Internet Explorer / FireFox / Safari)=20 https://app.embluemail.com/Online/VON.aspx?data=3D7ff9bCco0d2ylTjlTvBMVJDZ9= POA%2FtX4vM7D5gfH%2BfoWzI9Kr4n6GxM6FgFrb%2FrP5m8EP8JHGwwZq1xW7yfS53LlBlAMsN= LEw8Vnm2ujODYHUNjvMA0zXMlay%2FkIU0yd!-!3QFK0+yJxkYahrWQ1GzHpc18Lh/wNl3cc2aJ= NqDb+miBhuNczViZiO6e+s8L8kfH --===============5204116918946091560== Content-Type: text/html; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable
3 y 6 cuotas sin inter= 33;s

Ver en mi navegador

3D"Prototype"
BOT= TOMS  &= nbsp; NEW IN<= a>
=3D"30%
3 y 6 CUOTAS S= IN INTERÉS
y ¡ENVíO GRATIS!
3D"CA=
CAMISA FRANCIS
SHOP NOW
 
3D"CAMISA
CAMISA EVENING
SHOP NOW
 
3D"CAMISA
CAMISA GENTLE
SHOP NOW
 
3D"CAMISA
CAMISA NENDAZ
SHOP NOW
 
3D"CAMISA
CAMISA NOODLE
= SHOP NOW
 
3D"CAMISA
CAMISA STEER
<= span style=3D"color: #000;"> SHOP NOW
 
 
3D""
 
6 cuotas sin interés
3D""
 
+$8000 envíos gratis
3D""
 
= Política de camb= ios
 
Se= guinos en:
3D"Instagram"   3D"Facebook"   3D"Twitter"   3D"Pinterest"   3D"Youtube"   3D"Facebook"
 
Términos y = Condiciones
  |   Política de devolución
  |  Preguntas frecuentes
Franquicias
 = ; |   Locales
  |&nb= sp; Contacto
 
3D"" =20 =20 =20

En virtud de lo establecido por la disposició= n de Protección de Datos Personales usted tiene derecho a solicitar = al emisor de este mensaje la rectificación, actualización, in= clusión o supresión de los datos personales incluidos en su b= ase de contactos, listas o cadenas de mensajes en los cuales usted se encue= ntre. Conozca más
Quiero desuscribirme |=20 Actualizar perfil | Términos y condiciones

--===============5204116918946091560==-- From nobody Sat May 22 01:29:25 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id BBD5D8B77AE for ; Sat, 22 May 2021 01:29:26 +0000 (UTC) (envelope-from lojas@arroway.org) Received: from hobbes.arroway.org (hobbes.arroway.org [173.199.118.77]) by mx1.freebsd.org (Postfix) with ESMTP id 4Fn5VZ0PqPz4THD for ; Sat, 22 May 2021 01:29:25 +0000 (UTC) (envelope-from lojas@arroway.org) Received: from [10.1.7.11] (unknown [187.61.214.250]) by hobbes.arroway.org (Postfix) with ESMTPA id 63D8C14D20F for ; Fri, 21 May 2021 22:29:25 -0300 (-03) Received: from 10.1.4.100 (SquirrelMail authenticated user matheus) by 10.1.7.11 with HTTP; Fri, 21 May 2021 22:29:25 -0300 Message-ID: Date: Fri, 21 May 2021 22:29:25 -0300 Subject: Re: amdgpu.ko crashes on Radeon Vega From: "none" To: freebsd-hackers@freebsd.org User-Agent: SquirrelMail/1.4.23 [SVN] List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Rspamd-Queue-Id: 4Fn5VZ0PqPz4THD X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of lojas@arroway.org designates 173.199.118.77 as permitted sender) smtp.mailfrom=lojas@arroway.org X-Spamd-Result: default: False [0.19 / 15.00]; ARC_NA(0.00)[]; FAKE_REPLY(1.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[173.199.118.77:from]; R_SPF_ALLOW(-0.20)[+a:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[173.199.118.77:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[arroway.org]; NEURAL_HAM_LONG(-0.80)[-0.805]; NEURAL_HAM_SHORT(-1.00)[-1.000]; HAS_X_PRIO_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-0.81)[-0.809]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:20473, ipnet:173.199.116.0/22, country:US]; MID_BARE_IP(2.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-Spam: Yes On Sat, May 15, 2021 14:50, George Mitchell wrote: > On 5/15/21 1:29 PM, Nenhum_de_Nos wrote: >>[...] >> George, >> >> I'm about to try this on Ryzen 3400G here, where can I find this step by step to try to repeat what you did? My box has just basic FreeBSD install, >> cause I couldn't use Xorg at all. When I finish something on another OS > I >> will try this. >> >> Thanks, >> >> matheus >> >> > 1. Install FreeBSD 12. > 2. Install drm-fbsd12.0-kmod. > 3. Boot into single-user mode. > 4. Type "kldload amdgpu". Hi, I did just that and no crash here. The system is: FreeBSD xxx 12.2-RELEASE-p6 FreeBSD 12.2-RELEASE-p6 GENERIC amd64 % kldstat Id Refs Address Size Name 1 63 0xffffffff80200000 227ae98 kernel 2 1 0xffffffff82a11000 24f9e4 amdgpu.ko 3 2 0xffffffff82c61000 75e80 drm.ko 4 5 0xffffffff82cd7000 12d30 linuxkpi.ko 5 4 0xffffffff82cea000 13f30 linuxkpi_gplv2.ko 6 2 0xffffffff82cfe000 6d0 debugfs.ko 7 1 0xffffffff82cff000 f0e1 ttm.ko 8 1 0xffffffff82d0f000 16bf0 if_iwm.ko 9 1 0xffffffff82d26000 2698 intpm.ko 10 1 0xffffffff82d29000 b40 smbus.ko 11 1 0xffffffff82d2a000 28debf iwm9260fw.ko 12 1 0xffffffff82fb8000 4260 ng_ubt.ko 13 6 0xffffffff82fbd000 9bd0 netgraph.ko 14 2 0xffffffff82fc7000 9128 ng_hci.ko 15 3 0xffffffff82fd1000 9b0 ng_bluetooth.ko 16 1 0xffffffff82fd2000 1860 uhid.ko 17 1 0xffffffff82fd4000 2908 ums.ko 18 1 0xffffffff82fd7000 1a40 wmt.ko 19 1 0xffffffff82fd9000 caf0 ng_l2cap.ko 20 1 0xffffffff82fe6000 1af20 ng_btsocket.ko 21 1 0xffffffff83001000 2150 ng_socket.ko > When you say you can't use Xorg at all, does that mean VESA mode is not working? -- George Yep, but I had quite no time to research for the right xorg.conf to use it. I tried startx here zero xorg.conf fiddling, and server error. Its so many time since I last did this, I am here quite a first timer. thanks, matheus -- "We will call you Cygnus, the God of balance you shall be." From nobody Sat May 22 02:18:18 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9F6B18BF40E for ; Sat, 22 May 2021 02:18:19 +0000 (UTC) (envelope-from lojas@arroway.org) Received: from hobbes.arroway.org (hobbes.arroway.org [173.199.118.77]) by mx1.freebsd.org (Postfix) with ESMTP id 4Fn6Zz07dDz4tP6 for ; Sat, 22 May 2021 02:18:18 +0000 (UTC) (envelope-from lojas@arroway.org) Received: from [10.1.7.11] (unknown [187.61.214.250]) by hobbes.arroway.org (Postfix) with ESMTPA id 4E9D014D20F for ; Fri, 21 May 2021 23:18:17 -0300 (-03) Received: from 10.1.6.55 (SquirrelMail authenticated user matheus) by 10.1.7.11 with HTTP; Fri, 21 May 2021 23:18:18 -0300 Message-ID: In-Reply-To: References: Date: Fri, 21 May 2021 23:18:18 -0300 Subject: Re: amdgpu.ko crashes on Radeon Vega From: "none" To: freebsd-hackers@freebsd.org User-Agent: SquirrelMail/1.4.23 [SVN] List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Rspamd-Queue-Id: 4Fn6Zz07dDz4tP6 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of lojas@arroway.org designates 173.199.118.77 as permitted sender) smtp.mailfrom=lojas@arroway.org X-Spamd-Result: default: False [-0.78 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+a:c]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_SHORT(-1.00)[-1.000]; HAS_X_PRIO_THREE(0.00)[3]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[173.199.118.77:from]; ASN(0.00)[asn:20473, ipnet:173.199.116.0/22, country:US]; R_DKIM_NA(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.58)[-0.583]; FROM_HAS_DN(0.00)[]; SH_EMAIL_DBL_DONT_QUERY_IPS(0.00)[0.0.0.0:email]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.996]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[arroway.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[173.199.118.77:from:127.0.2.255]; DBL_PROHIBIT(0.00)[0.0.0.0:email]; MID_BARE_IP(2.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-Spam: Yes On Fri, May 21, 2021 22:29, none wrote: > On Sat, May 15, 2021 14:50, George Mitchell wrote: >> On 5/15/21 1:29 PM, Nenhum_de_Nos wrote: >>>[...] >>> George, >>> >>> I'm about to try this on Ryzen 3400G here, where can I find this step > by step to try to repeat what you did? My box has just basic FreeBSD > install, >>> cause I couldn't use Xorg at all. When I finish something on another OS >> I >>> will try this. >>> >>> Thanks, >>> >>> matheus >>> >>> >> 1. Install FreeBSD 12. >> 2. Install drm-fbsd12.0-kmod. >> 3. Boot into single-user mode. >> 4. Type "kldload amdgpu". > > Hi, > > I did just that and no crash here. The system is: > > FreeBSD xxx 12.2-RELEASE-p6 FreeBSD 12.2-RELEASE-p6 GENERIC amd64 > > % kldstat > Id Refs Address Size Name > 1 63 0xffffffff80200000 227ae98 kernel > 2 1 0xffffffff82a11000 24f9e4 amdgpu.ko > 3 2 0xffffffff82c61000 75e80 drm.ko > 4 5 0xffffffff82cd7000 12d30 linuxkpi.ko > 5 4 0xffffffff82cea000 13f30 linuxkpi_gplv2.ko > 6 2 0xffffffff82cfe000 6d0 debugfs.ko > 7 1 0xffffffff82cff000 f0e1 ttm.ko > 8 1 0xffffffff82d0f000 16bf0 if_iwm.ko > 9 1 0xffffffff82d26000 2698 intpm.ko > 10 1 0xffffffff82d29000 b40 smbus.ko > 11 1 0xffffffff82d2a000 28debf iwm9260fw.ko > 12 1 0xffffffff82fb8000 4260 ng_ubt.ko > 13 6 0xffffffff82fbd000 9bd0 netgraph.ko > 14 2 0xffffffff82fc7000 9128 ng_hci.ko > 15 3 0xffffffff82fd1000 9b0 ng_bluetooth.ko > 16 1 0xffffffff82fd2000 1860 uhid.ko > 17 1 0xffffffff82fd4000 2908 ums.ko > 18 1 0xffffffff82fd7000 1a40 wmt.ko > 19 1 0xffffffff82fd9000 caf0 ng_l2cap.ko > 20 1 0xffffffff82fe6000 1af20 ng_btsocket.ko > 21 1 0xffffffff83001000 2150 ng_socket.ko > > >> When you say you can't use Xorg at all, does that mean VESA mode is not > working? -- George > > Yep, but I had quite no time to research for the right xorg.conf to use > it. I tried startx here zero xorg.conf fiddling, and server error. Its so > many time since I last did this, I am here quite a first timer. > > thanks, > > matheus I tried it on rc.conf for loading, no crashes on two reboots. The console won't change resolution though, as it does on another box here that uses some old radeon gpu. startx dies in front of me: % cat /var/log/Xorg.0.log [ 36.115] X.Org X Server 1.20.9 X Protocol Version 11, Revision 0 [ 36.115] Build Operating System: FreeBSD 12.1-RELEASE-p12 amd64 [ 36.115] Current Operating System: FreeBSD elita 12.2-RELEASE-p6 FreeBSD 12.2-RELEASE-p6 GENERIC amd64 [ 36.115] Build Date: 05 January 2021 02:48:32PM [ 36.115] [ 36.115] Current version of pixman: 0.40.0 [ 36.115] Before reporting problems, check http://wiki.x.org to make sure that you have the latest version. [ 36.115] Markers: (--) probed, (**) from config file, (==) default setting, (++) from command line, (!!) notice, (II) informational, (WW) warning, (EE) error, (NI) not implemented, (??) unknown. [ 36.116] (==) Log file: "/var/log/Xorg.0.log", Time: Fri May 21 23:06:24 2021 [ 36.118] (==) Using system config directory "/usr/local/share/X11/xorg.conf.d" [ 36.119] (==) No Layout section. Using the first Screen section. [ 36.120] (==) No screen section available. Using defaults. [ 36.120] (**) |-->Screen "Default Screen Section" (0) [ 36.120] (**) | |-->Monitor "" [ 36.120] (==) No monitor specified for screen "Default Screen Section". Using a default monitor configuration. [ 36.120] (==) Automatically adding devices [ 36.120] (==) Automatically enabling devices [ 36.120] (==) Not automatically adding GPU devices [ 36.121] (==) Max clients allowed: 256, resource mask: 0x1fffff [ 36.126] (==) FontPath set to: /usr/local/share/fonts/misc/, /usr/local/share/fonts/TTF/, /usr/local/share/fonts/OTF/, /usr/local/share/fonts/Type1/, /usr/local/share/fonts/100dpi/, /usr/local/share/fonts/75dpi/, catalogue:/usr/local/etc/X11/fontpath.d [ 36.126] (==) ModulePath set to "/usr/local/lib/xorg/modules" [ 36.126] (II) The server relies on udev to provide the list of input devices. If no devices become available, reconfigure udev or disable AutoAddDevices. [ 36.126] (II) Loader magic: 0x42f020 [ 36.126] (II) Module ABI versions: [ 36.126] X.Org ANSI C Emulation: 0.4 [ 36.126] X.Org Video Driver: 24.1 [ 36.126] X.Org XInput driver : 24.1 [ 36.126] X.Org Server Extension : 10.0 [ 36.126] (--) PCI:*(10@0:0:0) 1002:15d8:1002:15d8 rev 200, Mem @ 0xe0000000/268435456, 0xf0000000/2097152, 0xfca00000/524288, I/O @ 0x0000e000/256, BIOS @ 0x????????/65536 [ 36.126] (II) LoadModule: "glx" [ 36.127] (II) Loading /usr/local/lib/xorg/modules/extensions/libglx.so [ 36.138] (II) Module glx: vendor="X.Org Foundation" [ 36.138] compiled for 1.20.9, module version = 1.0.0 [ 36.139] ABI class: X.Org Server Extension, version 10.0 [ 36.139] (==) Matched ati as autoconfigured driver 0 [ 36.139] (==) Matched modesetting as autoconfigured driver 1 [ 36.139] (==) Matched scfb as autoconfigured driver 2 [ 36.139] (==) Matched vesa as autoconfigured driver 3 [ 36.139] (==) Assigned the driver to the xf86ConfigLayout [ 36.139] (II) LoadModule: "ati" [ 36.139] (WW) Warning, couldn't open module ati [ 36.139] (EE) Failed to load module "ati" (module does not exist, 0) [ 36.139] (II) LoadModule: "modesetting" [ 36.140] (II) Loading /usr/local/lib/xorg/modules/drivers/modesetting_drv.so [ 36.141] (II) Module modesetting: vendor="X.Org Foundation" [ 36.141] compiled for 1.20.9, module version = 1.20.9 [ 36.141] Module class: X.Org Video Driver [ 36.141] ABI class: X.Org Video Driver, version 24.1 [ 36.141] (II) LoadModule: "scfb" [ 36.141] (II) Loading /usr/local/lib/xorg/modules/drivers/scfb_drv.so [ 36.141] (II) Module scfb: vendor="X.Org Foundation" [ 36.141] compiled for 1.20.9, module version = 0.0.5 [ 36.141] ABI class: X.Org Video Driver, version 24.1 [ 36.141] (II) LoadModule: "vesa" [ 36.141] (II) Loading /usr/local/lib/xorg/modules/drivers/vesa_drv.so [ 36.142] (II) Module vesa: vendor="X.Org Foundation" [ 36.142] compiled for 1.20.9, module version = 2.5.0 [ 36.142] Module class: X.Org Video Driver [ 36.142] ABI class: X.Org Video Driver, version 24.1 [ 36.142] (II) modesetting: Driver for Modesetting Kernel Drivers: kms [ 36.142] (II) scfb: driver for wsdisplay framebuffer: scfb [ 36.142] (II) VESA: driver for VESA chipsets: vesa [ 36.142] (--) Using syscons driver with X support (version 2.0) [ 36.142] (--) using VT number 9 [ 36.142] (EE) open /dev/dri/card0: No such file or directory [ 36.142] (WW) Falling back to old probe method for modesetting [ 36.142] (EE) open /dev/dri/card0: No such file or directory [ 36.142] (WW) Falling back to old probe method for scfb [ 36.142] scfb trace: probe start [ 36.142] (II) scfb(1): using default device [ 36.142] scfb trace: probe done [ 36.142] (WW) VGA arbiter: cannot open kernel arbiter, no multi-card support [ 36.142] (EE) Screen 0 deleted because of no matching config section. [ 36.142] (II) UnloadModule: "modesetting" [ 36.142] (EE) Fatal server error: [ 36.142] (EE) Cannot run in framebuffer mode. Please specify busIDs for all framebuffer devices [ 36.142] (EE) [ 36.142] (EE) Please consult the The X.Org Foundation support at http://wiki.x.org for help. [ 36.142] (EE) Please also check the log file at "/var/log/Xorg.0.log" for additional information. [ 36.142] (EE) [ 36.142] (EE) Server terminated with error (1). Closing log file. I tried it without any pre-config, this is the error. I tried Xorg -configure and the file created by it. Error no screens found. Am I missing something? The handbook told me not much more. Thanks, matheus PS: I couldn't answer that first mail using my other email address, as the mail list software told me several times there was an access error: The message from with subject "Re: amdgpu.ko crashes on Radeon Vega" was unable to be delivered to the list because of an access rule set up by the list administrator. (The denied message is below.) The mail is subscribed, is receiving messages fine (got that one from new address). I didn't get this error at all. -- "We will call you Cygnus, the God of balance you shall be." From nobody Sat May 22 10:54:34 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id EFA8F8B888E for ; Sat, 22 May 2021 10:54:37 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com [IPv6:2a00:1450:4864:20::22c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FnL2j6HVXz4l51 for ; Sat, 22 May 2021 10:54:37 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: by mail-lj1-x22c.google.com with SMTP id f12so27025984ljp.2 for ; Sat, 22 May 2021 03:54:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=MpPIHDBkQnTEbbsgE8j2cgSs8gs4IgthJS8kRjhjuGQ=; b=WjxEFIKFn4PRCxno7NT35p1k+ngSbFiuoj+95trq3hPcZ4pyB/5sLFW5mZBnNgJQhK vZ2U/9yRUIsFjPinbPFYWntXfb3nWKN4N/CqoZTZe1EecKXubkyFbps2qSqpjTPYjDw6 +s6+IJRxT+IOVgd5WY+al//Jok9lQX2HbVuYZJ9VbL9ZFZO0/h9Gemjj4CK2Pz3H4a3t h7apth563NRkCuIF1gH0nO//ydxgl3pV+QU6wdQ/tgI+fVJQEXlYU3VVpHbDvv614Nj9 D778cQWMZSD39/yL4EZFjODOABbQU+Ug4Ap6vGKH6Nd6bZgHl2fk6rTuBvRI6zXgqFoI 9Iuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=MpPIHDBkQnTEbbsgE8j2cgSs8gs4IgthJS8kRjhjuGQ=; b=tod0MPiIP/pzGZ76sFDe2SHeQcLGtm0/eT87zyJ5yIekvTMuSx/gWJZnH0kc1kbVO5 h5L2djiMfEv2VK/dGnsbP61+kgJD7us4rlhcmB8I1m2tl8YO9td6jhBYDPl6OsprbRIw CAjVJOm2WL0Gj2wP4CIy3O3YWXgrlWX654+OuBTdKWXx+yY2BLEGxlcU1SokEtO2NyUG sKQwlEhExro3KE9wz+jjnZBR8v7qkUljZQ+ELikPDO/L5Z0BemVJ4xi28jqbfQpx/bDs y2wSWj8tB+FfkeP8AucqBjHsKjtmaYbZFm2MZUKB4mQ5TUaYFMi8Em8PYTBYTYu+v9e/ 5SWg== X-Gm-Message-State: AOAM533C6+DOr/tDP3jmD+lzszMenCbFYNIn9RQJ4cLUbb2Mm/LgKPBG QqF3NKkVtzEWgxNr31NCvtzGLbb25YjHiI7pxGRa0B6D X-Google-Smtp-Source: ABdhPJwsvHLRaaInhuaAVD9ih1jIgzX/U0iDz+xKMoVtWUY8MRoql7h20KRhPGt8b7PooDH5AbB0sW2OsF22FrisnkU= X-Received: by 2002:a05:651c:b1f:: with SMTP id b31mr10321273ljr.0.1621680876045; Sat, 22 May 2021 03:54:36 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Received: by 2002:a05:651c:115:0:0:0:0 with HTTP; Sat, 22 May 2021 03:54:34 -0700 (PDT) In-Reply-To: References: From: Stefan Blachmann Date: Sat, 22 May 2021 12:54:34 +0200 Message-ID: Subject: Re: amdgpu.ko crashes on Radeon Vega To: none Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4FnL2j6HVXz4l51 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] May I point you at the fact that FreeBSD 12's drm-kmod is taken from Linux 4.16. Linux 4.16 does afaik not yet support Vega graphics. Btw, this information is shown only on the Intel graphics wiki, not in the AMD graphics wiki, which might be the reason why you missed out that. But regarding hardware support, this applies to AMD graphics as well: https://wiki.freebsd.org/Graphics/Intel-GPU-Matrix If this is correct, the only way to make Vega graphics work is to use FreeBSD 13, whose drm-kmod is taken from Linux 5.4. P.S.: I usually don't post anymore on the freebsd lists, as after I criticized the inclusion of Wayland's libinput, pointing into the mouse wheel issues it introduces (and which are since then a long-running issue on FreeBSD also), I got moderated and it is hit and miss whether my posts are let through or not. On 5/22/21, none wrote: > On Fri, May 21, 2021 22:29, none wrote: >> On Sat, May 15, 2021 14:50, George Mitchell wrote: >>> On 5/15/21 1:29 PM, Nenhum_de_Nos wrote: >>>>[...] >>>> George, >>>> >>>> I'm about to try this on Ryzen 3400G here, where can I find this step >> by step to try to repeat what you did? My box has just basic FreeBSD >> install, >>>> cause I couldn't use Xorg at all. When I finish something on another OS >>> I >>>> will try this. >>>> >>>> Thanks, >>>> >>>> matheus >>>> >>>> >>> 1. Install FreeBSD 12. >>> 2. Install drm-fbsd12.0-kmod. >>> 3. Boot into single-user mode. >>> 4. Type "kldload amdgpu". >> >> Hi, >> >> I did just that and no crash here. The system is: >> >> FreeBSD xxx 12.2-RELEASE-p6 FreeBSD 12.2-RELEASE-p6 GENERIC amd64 >> >> % kldstat >> Id Refs Address Size Name >> 1 63 0xffffffff80200000 227ae98 kernel >> 2 1 0xffffffff82a11000 24f9e4 amdgpu.ko >> 3 2 0xffffffff82c61000 75e80 drm.ko >> 4 5 0xffffffff82cd7000 12d30 linuxkpi.ko >> 5 4 0xffffffff82cea000 13f30 linuxkpi_gplv2.ko >> 6 2 0xffffffff82cfe000 6d0 debugfs.ko >> 7 1 0xffffffff82cff000 f0e1 ttm.ko >> 8 1 0xffffffff82d0f000 16bf0 if_iwm.ko >> 9 1 0xffffffff82d26000 2698 intpm.ko >> 10 1 0xffffffff82d29000 b40 smbus.ko >> 11 1 0xffffffff82d2a000 28debf iwm9260fw.ko >> 12 1 0xffffffff82fb8000 4260 ng_ubt.ko >> 13 6 0xffffffff82fbd000 9bd0 netgraph.ko >> 14 2 0xffffffff82fc7000 9128 ng_hci.ko >> 15 3 0xffffffff82fd1000 9b0 ng_bluetooth.ko >> 16 1 0xffffffff82fd2000 1860 uhid.ko >> 17 1 0xffffffff82fd4000 2908 ums.ko >> 18 1 0xffffffff82fd7000 1a40 wmt.ko >> 19 1 0xffffffff82fd9000 caf0 ng_l2cap.ko >> 20 1 0xffffffff82fe6000 1af20 ng_btsocket.ko >> 21 1 0xffffffff83001000 2150 ng_socket.ko >> >> >>> When you say you can't use Xorg at all, does that mean VESA mode is not >> working? -- George >> >> Yep, but I had quite no time to research for the right xorg.conf to use >> it. I tried startx here zero xorg.conf fiddling, and server error. Its so >> many time since I last did this, I am here quite a first timer. >> >> thanks, >> >> matheus > > I tried it on rc.conf for loading, no crashes on two reboots. The console > won't change resolution though, as it does on another box here that uses > some old radeon gpu. > > startx dies in front of me: > > % cat /var/log/Xorg.0.log > [ 36.115] > X.Org X Server 1.20.9 > X Protocol Version 11, Revision 0 > [ 36.115] Build Operating System: FreeBSD 12.1-RELEASE-p12 amd64 > [ 36.115] Current Operating System: FreeBSD elita 12.2-RELEASE-p6 > FreeBSD 12.2-RELEASE-p6 GENERIC amd64 > [ 36.115] Build Date: 05 January 2021 02:48:32PM > [ 36.115] > [ 36.115] Current version of pixman: 0.40.0 > [ 36.115] Before reporting problems, check http://wiki.x.org > to make sure that you have the latest version. > [ 36.115] Markers: (--) probed, (**) from config file, (==) default > setting, > (++) from command line, (!!) notice, (II) informational, > (WW) warning, (EE) error, (NI) not implemented, (??) unknown. > [ 36.116] (==) Log file: "/var/log/Xorg.0.log", Time: Fri May 21 > 23:06:24 2021 > [ 36.118] (==) Using system config directory > "/usr/local/share/X11/xorg.conf.d" > [ 36.119] (==) No Layout section. Using the first Screen section. > [ 36.120] (==) No screen section available. Using defaults. > [ 36.120] (**) |-->Screen "Default Screen Section" (0) > [ 36.120] (**) | |-->Monitor "" > [ 36.120] (==) No monitor specified for screen "Default Screen Section". > Using a default monitor configuration. > [ 36.120] (==) Automatically adding devices > [ 36.120] (==) Automatically enabling devices > [ 36.120] (==) Not automatically adding GPU devices > [ 36.121] (==) Max clients allowed: 256, resource mask: 0x1fffff > [ 36.126] (==) FontPath set to: > /usr/local/share/fonts/misc/, > /usr/local/share/fonts/TTF/, > /usr/local/share/fonts/OTF/, > /usr/local/share/fonts/Type1/, > /usr/local/share/fonts/100dpi/, > /usr/local/share/fonts/75dpi/, > catalogue:/usr/local/etc/X11/fontpath.d > [ 36.126] (==) ModulePath set to "/usr/local/lib/xorg/modules" > [ 36.126] (II) The server relies on udev to provide the list of input > devices. > If no devices become available, reconfigure udev or disable > AutoAddDevices. > [ 36.126] (II) Loader magic: 0x42f020 > [ 36.126] (II) Module ABI versions: > [ 36.126] X.Org ANSI C Emulation: 0.4 > [ 36.126] X.Org Video Driver: 24.1 > [ 36.126] X.Org XInput driver : 24.1 > [ 36.126] X.Org Server Extension : 10.0 > [ 36.126] (--) PCI:*(10@0:0:0) 1002:15d8:1002:15d8 rev 200, Mem @ > 0xe0000000/268435456, 0xf0000000/2097152, 0xfca00000/524288, I/O @ > 0x0000e000/256, BIOS @ 0x????????/65536 > [ 36.126] (II) LoadModule: "glx" > [ 36.127] (II) Loading /usr/local/lib/xorg/modules/extensions/libglx.so > [ 36.138] (II) Module glx: vendor="X.Org Foundation" > [ 36.138] compiled for 1.20.9, module version = 1.0.0 > [ 36.139] ABI class: X.Org Server Extension, version 10.0 > [ 36.139] (==) Matched ati as autoconfigured driver 0 > [ 36.139] (==) Matched modesetting as autoconfigured driver 1 > [ 36.139] (==) Matched scfb as autoconfigured driver 2 > [ 36.139] (==) Matched vesa as autoconfigured driver 3 > [ 36.139] (==) Assigned the driver to the xf86ConfigLayout > [ 36.139] (II) LoadModule: "ati" > [ 36.139] (WW) Warning, couldn't open module ati > [ 36.139] (EE) Failed to load module "ati" (module does not exist, 0) > [ 36.139] (II) LoadModule: "modesetting" > [ 36.140] (II) Loading > /usr/local/lib/xorg/modules/drivers/modesetting_drv.so > [ 36.141] (II) Module modesetting: vendor="X.Org Foundation" > [ 36.141] compiled for 1.20.9, module version = 1.20.9 > [ 36.141] Module class: X.Org Video Driver > [ 36.141] ABI class: X.Org Video Driver, version 24.1 > [ 36.141] (II) LoadModule: "scfb" > [ 36.141] (II) Loading /usr/local/lib/xorg/modules/drivers/scfb_drv.so > [ 36.141] (II) Module scfb: vendor="X.Org Foundation" > [ 36.141] compiled for 1.20.9, module version = 0.0.5 > [ 36.141] ABI class: X.Org Video Driver, version 24.1 > [ 36.141] (II) LoadModule: "vesa" > [ 36.141] (II) Loading /usr/local/lib/xorg/modules/drivers/vesa_drv.so > [ 36.142] (II) Module vesa: vendor="X.Org Foundation" > [ 36.142] compiled for 1.20.9, module version = 2.5.0 > [ 36.142] Module class: X.Org Video Driver > [ 36.142] ABI class: X.Org Video Driver, version 24.1 > [ 36.142] (II) modesetting: Driver for Modesetting Kernel Drivers: kms > [ 36.142] (II) scfb: driver for wsdisplay framebuffer: scfb > [ 36.142] (II) VESA: driver for VESA chipsets: vesa > [ 36.142] (--) Using syscons driver with X support (version 2.0) > [ 36.142] (--) using VT number 9 > > [ 36.142] (EE) open /dev/dri/card0: No such file or directory > [ 36.142] (WW) Falling back to old probe method for modesetting > [ 36.142] (EE) open /dev/dri/card0: No such file or directory > [ 36.142] (WW) Falling back to old probe method for scfb > [ 36.142] scfb trace: probe start > [ 36.142] (II) scfb(1): using default device > [ 36.142] scfb trace: probe done > [ 36.142] (WW) VGA arbiter: cannot open kernel arbiter, no multi-card > support > [ 36.142] (EE) Screen 0 deleted because of no matching config section. > [ 36.142] (II) UnloadModule: "modesetting" > [ 36.142] (EE) > Fatal server error: > [ 36.142] (EE) Cannot run in framebuffer mode. Please specify busIDs > for all framebuffer devices > [ 36.142] (EE) > [ 36.142] (EE) > Please consult the The X.Org Foundation support > at http://wiki.x.org > for help. > [ 36.142] (EE) Please also check the log file at "/var/log/Xorg.0.log" > for additional information. > [ 36.142] (EE) > [ 36.142] (EE) Server terminated with error (1). Closing log file. > > I tried it without any pre-config, this is the error. > > I tried Xorg -configure and the file created by it. Error no screens found. > > Am I missing something? > > The handbook told me not much more. > > Thanks, > > matheus > > PS: I couldn't answer that first mail using my other email address, as the > mail list software told me several times there was an access error: > The message from with subject "Re: amdgpu.ko > crashes on Radeon Vega" was unable to be delivered to the list because of > an access rule set up by the list administrator. > > (The denied message is below.) > > The mail is subscribed, is receiving messages fine (got that one from new > address). I didn't get this error at all. > > > -- > "We will call you Cygnus, > the God of balance you shall be." > > > From nobody Sun May 23 01:25:41 2021 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E36FF8EED79 for ; Sun, 23 May 2021 01:25:57 +0000 (UTC) (envelope-from lojas@arroway.org) Received: from hobbes.arroway.org (hobbes.arroway.org [173.199.118.77]) by mx1.freebsd.org (Postfix) with ESMTP id 4FnjMy42ySz4qDf for ; Sun, 23 May 2021 01:25:50 +0000 (UTC) (envelope-from lojas@arroway.org) Received: from [10.1.7.11] (unknown [187.61.214.250]) by hobbes.arroway.org (Postfix) with ESMTPA id 487D614D20F for ; Sat, 22 May 2021 22:25:40 -0300 (-03) Received: from 10.1.6.55 (SquirrelMail authenticated user matheus) by 10.1.7.11 with HTTP; Sat, 22 May 2021 22:25:41 -0300 Message-ID: <6c0f174ad8085be0be0e6a8dad0c474b.squirrel@10.1.7.11> In-Reply-To: References: Date: Sat, 22 May 2021 22:25:41 -0300 Subject: Re: amdgpu.ko crashes on Radeon Vega From: "none" To: hackers@freebsd.org User-Agent: SquirrelMail/1.4.23 [SVN] List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Rspamd-Queue-Id: 4FnjMy42ySz4qDf X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of lojas@arroway.org designates 173.199.118.77 as permitted sender) smtp.mailfrom=lojas@arroway.org X-Spamd-Result: default: False [-1.04 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.84)[-0.844]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[173.199.118.77:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[173.199.118.77:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[arroway.org]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; HAS_X_PRIO_THREE(0.00)[3]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:20473, ipnet:173.199.116.0/22, country:US]; MID_BARE_IP(2.00)[]; MAILMAN_DEST(0.00)[hackers] X-Spam: Yes On Sat, May 22, 2021 07:54, Stefan Blachmann wrote: > May I point you at the fact that FreeBSD 12's drm-kmod is taken from Linux > 4.16. > Linux 4.16 does afaik not yet support Vega graphics. > > Btw, this information is shown only on the Intel graphics wiki, not in > the AMD graphics wiki, which might be the reason why you missed out > that. > But regarding hardware support, this applies to AMD graphics as well: > https://wiki.freebsd.org/Graphics/Intel-GPU-Matrix > > If this is correct, the only way to make Vega graphics work is to use > FreeBSD 13, whose drm-kmod is taken from Linux 5.4. > > P.S.: > I usually don't post anymore on the freebsd lists, as after I > criticized the inclusion of Wayland's libinput, pointing into the > mouse wheel issues it introduces (and which are since then a > long-running issue on FreeBSD also), I got moderated and it is hit and > miss whether my posts are let through or not. I tried running Xorg on FreeBSD 13.0R, and it just worked. I found about this on the FreeBSD Forum. matheus -- "We will call you Cygnus, the God of balance you shall be."