From ml@ft-c.de Tue Sep 7 08:26:31 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 1FAC117A2934 for ; Tue, 7 Sep 2021 08:26:41 +0000 (UTC) (envelope-from ml@ft-c.de) Received: from einhorn-mail-out.in-berlin.de (einhorn.in-berlin.de [192.109.42.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail.in-berlin.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H3df80b4Bz4RRf for ; Tue, 7 Sep 2021 08:26:39 +0000 (UTC) (envelope-from ml@ft-c.de) X-Envelope-From: ml@ft-c.de X-Envelope-To: Received: from authenticated.user (localhost [127.0.0.1]) by einhorn.in-berlin.de with ESMTPSA id 1878QVVd018456 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT) for ; Tue, 7 Sep 2021 10:26:32 +0200 Message-ID: Subject: Re: ERROR:scsi-cam.c:173:brasero_device_handle_open -- SOLVED From: ml@ft-c.de Reply-To: ml@ft-c.de To: FreeBSD-STABLE Mailing List Date: Tue, 07 Sep 2021 10:26:31 +0200 In-Reply-To: <2d78a06d5f0b75b0fa7ee3e6e09b3afe8db2ab50.camel@ft-c.de> References: <2d78a06d5f0b75b0fa7ee3e6e09b3afe8db2ab50.camel@ft-c.de> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.40.2 FreeBSD GNOME Team List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4H3df80b4Bz4RRf X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of ml@ft-c.de has no SPF policy when checking 192.109.42.8) smtp.mailfrom=ml@ft-c.de X-Spamd-Result: default: False [-0.62 / 15.00]; HAS_REPLYTO(0.00)[ml@ft-c.de]; RCVD_VIA_SMTP_AUTH(0.00)[]; REPLYTO_EQ_FROM(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[192.109.42.8:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_SPAM_MEDIUM(0.54)[0.536]; DMARC_NA(0.00)[ft-c.de]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.95)[-0.954]; FROM_NO_DN(0.00)[]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:29670, ipnet:192.109.42.0/24, country:DE]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[192.109.42.8:from] X-ThisMailContainsUnwantedMimeParts: N This problem is solved. since the last 'pkg upgrade' there is no error. On Sun, 2021-09-05 at 11:55 +0200, ml@ft-c.de wrote: > Hallo, > > yesterday I had updated to 13.0-RELEASE-p4  > > when I start rhythmbox as User, I get the message: >  > rhythmbox > ** > ERROR:scsi-cam.c:173:brasero_device_handle_open: assertion failed: > (path != NULL) > Bail out! ERROR:scsi-cam.c:173:brasero_device_handle_open: assertion > failed: (path != NULL) > Abort trap > > When I start it as root, there are some warning-message, but the > program is running (and music is playing). > > Franz From nobody Thu Sep 9 17:17:13 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id DC0FC17AA9A7 for ; Thu, 9 Sep 2021 17:17:26 +0000 (UTC) (envelope-from eugene@zhegan.in) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [128.127.146.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.norma.perm.ru", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H55Kd4X95z4ccW for ; Thu, 9 Sep 2021 17:17:25 +0000 (UTC) (envelope-from eugene@zhegan.in) Received: from [192.168.243.7] ([192.168.243.7]) by elf.hq.norma.perm.ru (8.16.1/8.15.2) with ESMTP id 189HErpV051183 for ; Thu, 9 Sep 2021 22:14:53 +0500 (+05) (envelope-from eugene@zhegan.in) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=vivat-retail.ru; s=key; t=1631207693; bh=IVV/x8u8eXPCWSz7jHO8+iafsgAhKAOHujtsOqR46mY=; h=To:From:Subject:Date; b=nUEvanUCWhJ3yPW1xUs34PJxrBviAA82VVp8Btx4iirtggFc7KnqFQ3Nduc9Bu3/O wPtt2U/uLiSHJORhjgv2YV4lgqRdU4snPWiuNyfkGtOFwPD2i235jhWBsck8j6AEV3 2HA2Yi5XehuoS4U4ogBiojtvIqEZeNQ78OKGfWKU= To: freebsd-stable@freebsd.org From: "Eugene M. Zheganin" Subject: kern.kq_calloutmax - when to increase Message-ID: Date: Thu, 9 Sep 2021 22:17:13 +0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: ru X-Rspamd-Queue-Id: 4H55Kd4X95z4ccW X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=fail ("headers rsa verify failed") header.d=vivat-retail.ru header.s=key header.b=nUEvanUC; dmarc=none; spf=pass (mx1.freebsd.org: domain of eugene@zhegan.in designates 128.127.146.8 as permitted sender) smtp.mailfrom=eugene@zhegan.in X-Spamd-Result: default: False [-0.39 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_NA(0.00)[zhegan.in]; R_DKIM_REJECT(1.00)[vivat-retail.ru:s=key]; DKIM_TRACE(0.00)[vivat-retail.ru:-]; NEURAL_SPAM_SHORT(0.91)[0.910]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:212494, ipnet:128.127.146.0/24, country:RU]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Hello, so imagine I have a web balancer using nginx with kqueue which handles numerous connections to a set of kqueue-enabled custom applications/backends, so my question is - should I ever touch the default kern.kq_calloutmax of 4K or should I left it there ? And if it should be tuned - where is the TCP connection treshold ? Thanks. From nobody Thu Sep 9 21:38:49 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id AF03117B2D8E for ; Thu, 9 Sep 2021 21:38:50 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [IPv6:2607:f3e0:0:3::19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pyroxene.sentex.ca", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H5C7F5DBtz3F2c; Thu, 9 Sep 2021 21:38:49 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [IPv6:2607:f3e0:0:4:143c:9b3f:6fc9:8e0b] ([IPv6:2607:f3e0:0:4:143c:9b3f:6fc9:8e0b]) by pyroxene2a.sentex.ca (8.16.1/8.15.2) with ESMTPS id 189LcmgA046551 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Thu, 9 Sep 2021 17:38:48 -0400 (EDT) (envelope-from mike@sentex.net) To: FreeBSD-STABLE Mailing List From: mike tancsa Subject: routing changes from Sept 7th and possible ppp breakage (RELENG_13) Message-ID: Date: Thu, 9 Sep 2021 17:38:49 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 Content-Language: en-US X-Rspamd-Queue-Id: 4H5C7F5DBtz3F2c X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:3::19 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-0.02 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_SPAM_SHORT(0.97)[0.974]; FREEFALL_USER(0.00)[mike]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; MIME_GOOD(-0.10)[text/plain]; HFILTER_HELO_IP_A(1.00)[pyroxene2a.sentex.ca]; HFILTER_HELO_NORES_A_OR_MX(0.30)[pyroxene2a.sentex.ca]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_NA(0.00)[sentex.net]; TO_DN_ALL(0.00)[]; MIME_BASE64_TEXT(0.10)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[2607:f3e0:0:3::19:from] X-ThisMailContainsUnwantedMimeParts: N T24gb3VyIHN0YW5kYXJkIG5hbm9ic2Qgcm91dGVyIGltYWdlIHdlIHVzZSwgSSBub3RpY2Vk IGZhaWxvdmVyIHJvdXRpbmcNCmJyb2tlIGFsbCBvZiBhIHN1ZGRlbi7CoCBXZSBhcmUgbm90 IHVzaW5nIG11bHRpcGxlIEZJQnMsIGJ1dCBzb21laG93IHdlDQpub3cgaGF2ZSB0d28gZGVm YXVsdCByb3V0ZXMgSSBhbSBndWVzc2luZyBiZWNhdXNlIG9mIG11bHRpcGF0aC7CoCAoTm90 ZSwNCndlIGp1c3Qgc3RhcnRlZCB0ZXN0aW5nIFJFTEVOR18xMyBzbyBpdHMgcG9zc2libGUg SSBhbSBtYWtpbmcgYmFkDQphc3N1bXB0aW9ucyBzb21ld2hlcmUuKQ0KDQpkZWZhdWx0wqDC oMKgwqDCoMKgwqDCoMKgwqDCoCAxMC4yNTUuMjU1LjHCoMKgwqDCoMKgwqAgVUdTwqDCoMKg wqDCoMKgwqDCoCA2wqDCoCAxNTAwwqDCoMKgwqDCoMKgIGlnYjANCmRlZmF1bHTCoMKgwqDC oMKgwqDCoMKgwqDCoMKgIDEwLjEuMC4xwqDCoMKgwqDCoMKgwqDCoMKgwqAgVUdTwqDCoMKg wqDCoMKgwqAgMTfCoMKgIDE1MDDCoMKgwqDCoMKgIHR1bjEwDQoNCkkgYW0gcHJldHR5IHN1 cmUgZnJvbSB0aGUgY29kZSBiYXNlIGZyb20gMysgZGF5cyBhZ28sIEkgd291bGQgbm90IHNl ZQ0KdGhpcyBjb25kaXRpb24uDQoNCkluIHRoZSBwYXN0LCBteSBuYW5vIGltYWdlIHdvdWxk IGdldCBhIGRlZmF1bHQgcm91dGUgdmlhIERIQ1AuwqAgSWYgdGhhdA0KcGF0aCB3YXMgbm90 IGZ1bmN0aW9uYWwgb3IgYmxvY2tlZCwgbXkgbW9uaXRvcmluZyBzb2Z0d2FyZSB3b3VsZCBk ZXRlY3QNCnRoZSBmYWlsdXJlIGFuZCBmaXJlIHVwIFBQUCB3aGljaCBoYXMNCg0KwqBhZGQh IGRlZmF1bHQgSElTQUREUg0KDQpUaGF0IG5vdyBzZWVtcyB0byBhZGQgYSBzZWNvbmQgZGVm YXVsdCByb3V0ZSBpbnN0ZWFkIG9mIGdldHRpbmcgcmlkIG9mDQp0aGUgb2xkIG9uZS4NCg0K aWYgSSBhZGQgbmV0LnJvdXRlLm11bHRpcGF0aD0wDQoNCkkgZ2V0IGJhY2sgdGhlIG9sZCBi ZWhhdmlvdXINCg0KQWNjb3JkaW5nIHRvIHRoZSBwcHAgZG9jcw0KDQrCoMKgwqDCoMKgwqDC oMKgIElmIHRoZSBhZGQhIGNvbW1hbmQgaXMgdXNlZCAobm90ZSB0aGUgdHJhaWxpbmcgIiEi KSwgdGhlbiBpZiB0aGUNCsKgwqDCoMKgwqDCoMKgwqAgcm91dGUgYWxyZWFkeSBleGlzdHMs IGl0IHdpbGwgYmUgdXBkYXRlZCBhcyB3aXRoIHRoZSBgcm91dGUgY2hhbmdlJw0KwqDCoMKg wqDCoMKgwqDCoCBjb21tYW5kIChzZWUgcm91dGUoOCkgZm9yIGZ1cnRoZXIgZGV0YWlscyku DQoNCldoZXJlIGFzIG5vdywgaXQgZG9lc250IHNlZW0gdG8gcmVwbGFjZSBpdCwgaXQgc2Vl bXMgdG8ganVzdCBhZGQgYW5vdGhlcg0KZGVmYXVsdCByb3V0ZS4NCg0KRm9yIG15IHB1cnBv c2VzLCBzZXR0aW5nIG5ldC5yb3V0ZS5tdWx0aXBhdGg9MCBzZWVtcyB0byB3b3JrIGp1c3Qg ZmluZSwNCmJ1dCBJIHdvbmRlciBpZiB0aGlzIHdhcnJhbnRzIGEgbm90ZSBpbiBwcHAncyBk b2NzIHNpbmNlIGl0IG5vIGxvbmdlcg0Kc2VlbXMgdG8gd29yayB0aGF0IHdheSBpbiBSRUxF TkdfMTMgPw0KDQrCoMKgwqAgLS0tTWlrZQ0KDQoNCg== From nobody Fri Sep 10 08:34:20 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id B7DDB17AF626 for ; Fri, 10 Sep 2021 08:55:31 +0000 (UTC) (envelope-from zlei.huang@gmail.com) Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H5V8349ffz4rQ2 for ; Fri, 10 Sep 2021 08:55:31 +0000 (UTC) (envelope-from zlei.huang@gmail.com) Received: by mail-pf1-x436.google.com with SMTP id g14so1307802pfm.1 for ; Fri, 10 Sep 2021 01:55:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=KaRvlRa0RMgGYikPs8+tS9Y9FXOZYmlqPxzZoBTiLpE=; b=VzoqHJRD1Uw13W11Dux/f3UnA998Xk58Qc/CgZVmzPbktHUJJh6uLVFyjEx5JJiNrT qQmva+l+l5/SU1Mr3LCNciU76BEvmijrBRsU1Czs5TUHwkPeLz5Gc2Sv0/LX8wUvd3HP Jfv4mjVJev9OP9928aplXcov6VIcgvulxiPRKt6YQeIf9OxJbDVfA5LFvIM3O08+Ajcf FZ6fmS+EfAWL0Z2wBpeMacgzmnOhGo5iuiJ0467CEUN8tF6iVifgmc6d/hsX1VebKbB+ R29oIfgVWY6M3KTMRWOC7TykQab+M88q3YCmVG8XuUU34+9AvsayQaQwt/VKq6icmkaG UDpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=KaRvlRa0RMgGYikPs8+tS9Y9FXOZYmlqPxzZoBTiLpE=; b=16uf0t1Vg79C7fzZX1kDF9DUCZBV3v2XA6LY5PprAv8vyEQSZkvFINKTj4pruY8UNl 3bd/FdcZBRKb8L9ktDh38m392Q7/IVq7K6FgQDszn3hwluE9znh3cWFbGpLkFE2uo8tG zXu1Rg7cybKO1dc+Ugt1mL74vJzHE7+Sa5Z1qPoCt8+KymZlz948CBQnfIre2HGQRqyN JMAVuAlYpaJBOjeJ52/zycEF1B15DRe2P71od+2KW1bE9qSXHwwVIIrT9jPUmoQ7WEx3 dv7zhgeJpyjUyq7fqBDFtyAdS6AGa5am34j6ity3GcO0HEOYwuJC5/OdMO+RW94q0Zc9 NBvQ== X-Gm-Message-State: AOAM533TorD08nGFA7ADS+KA/no1C66dX4hrKjBX1MF7Ina7ciX0mSau 2DPxJD7X2wFzx0ZCrpcdvGC+73NsGOb6oQ== X-Google-Smtp-Source: ABdhPJyOAGyLfSiBBhwdQycbVFVNkCs8kq2YaFjWjI+5pTm6omBgYezZkHLIL9xO0QHl4TknKsvt+g== X-Received: by 2002:a63:a311:: with SMTP id s17mr6325572pge.359.1631264130626; Fri, 10 Sep 2021 01:55:30 -0700 (PDT) Received: from [172.17.252.129] (ns1.oxydns.net. [45.32.91.63]) by smtp.gmail.com with ESMTPSA id v17sm4234356pff.6.2021.09.10.01.55.29 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 10 Sep 2021 01:55:30 -0700 (PDT) Content-Type: text/plain; charset=us-ascii List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.7\)) Subject: Re: routing changes from Sept 7th and possible ppp breakage (RELENG_13) From: Zhenlei Huang In-Reply-To: Date: Fri, 10 Sep 2021 16:34:20 +0800 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: To: mike tancsa X-Mailer: Apple Mail (2.3608.120.23.2.7) X-Rspamd-Queue-Id: 4H5V8349ffz4rQ2 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N > On Sep 10, 2021, at 5:38 AM, mike tancsa wrote: >=20 > On our standard nanobsd router image we use, I noticed failover = routing > broke all of a sudden. We are not using multiple FIBs, but somehow we > now have two default routes I am guessing because of multipath. = (Note, > we just started testing RELENG_13 so its possible I am making bad > assumptions somewhere.) >=20 > default 10.255.255.1 UGS 6 1500 igb0 > default 10.1.0.1 UGS 17 1500 tun10 >=20 > I am pretty sure from the code base from 3+ days ago, I would not see > this condition. >=20 > In the past, my nano image would get a default route via DHCP. If = that > path was not functional or blocked, my monitoring software would = detect > the failure and fire up PPP which has >=20 > add! default HISADDR >=20 > That now seems to add a second default route instead of getting rid of > the old one. >=20 > if I add net.route.multipath=3D0 >=20 > I get back the old behaviour FreeBSD 13 enabled multipath and default is on. See the routing section of release notes. = https://www.freebsd.org/releases/13.0R/relnotes/#network-routing >=20 > According to the ppp docs >=20 > If the add! command is used (note the trailing "!"), then if = the > route already exists, it will be updated as with the `route = change' > command (see route(8) for further details). >=20 > Where as now, it doesnt seem to replace it, it seems to just add = another > default route. At first glance it seems that ppp does not check existing default route = before replacing it with a new one. >=20 > For my purposes, setting net.route.multipath=3D0 seems to work just = fine, > but I wonder if this warrants a note in ppp's docs since it no longer > seems to work that way in RELENG_13 ? Not sure if we need errata. I think we should fix ppp :) >=20 > ---Mike >=20 >=20 From nobody Fri Sep 10 15:34:36 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A49C517AD14F for ; Fri, 10 Sep 2021 15:34:47 +0000 (UTC) (envelope-from freebsd@oldach.net) Received: from nuc.oldach.net (hmo.in-vpn.de [IPv6:2001:67c:1407:60::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "nuc.oldach.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H5g0k2Nmdz4rbj for ; Fri, 10 Sep 2021 15:34:46 +0000 (UTC) (envelope-from freebsd@oldach.net) Received: from nuc.oldach.net (localhost [127.0.0.1]) by nuc.oldach.net (8.17.1/8.17.1/hmo17dec20) with ESMTPS id 18AFYaGD001703 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 10 Sep 2021 17:34:37 +0200 (CEST) (envelope-from freebsd@oldach.net) Received: (from hmo@localhost) by nuc.oldach.net (8.17.1/8.17.1/Submit) id 18AFYarp001702; Fri, 10 Sep 2021 17:34:36 +0200 (CEST) (envelope-from freebsd@oldach.net) Message-Id: <202109101534.18AFYarp001702@nuc.oldach.net> Subject: Re: routing changes from Sept 7th and possible ppp breakage (RELENG_13) In-Reply-To: from Zhenlei Huang at "10 Sep 2021 16:34:20" To: zlei.huang@gmail.com (Zhenlei Huang) Date: Fri, 10 Sep 2021 17:34:36 +0200 (CEST) Cc: freebsd-stable@freebsd.org, mike@sentex.net From: freebsd@oldach.net (Helge Oldach) X-No-Archive: Yes List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Greylist: inspected by milter-greylist-4.6.4 (nuc.oldach.net [0.0.0.0]); Fri, 10 Sep 2021 17:34:37 +0200 (CEST) for IP:127.0.0.1 DOMAIN:localhost HELO:nuc.oldach.net FROM:freebsd@oldach.net RCPT: X-Rspamd-Queue-Id: 4H5g0k2Nmdz4rbj X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd@oldach.net designates 2001:67c:1407:60::1 as permitted sender) smtp.mailfrom=freebsd@oldach.net X-Spamd-Result: default: False [-3.30 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[oldach.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_NO_DN(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:29670, ipnet:2001:67c:1400::/45, country:DE]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Zhenlei Huang wrote on Fri, 10 Sep 2021 10:34:20 +0200 (CEST): > > On Sep 10, 2021, at 5:38 AM, mike tancsa wrote: > > default 10.255.255.1 UGS 6 1500 igb0 > > default 10.1.0.1 UGS 17 1500 tun10 > > > > Where as now, it doesnt seem to replace it, it seems to just add > > another default route. > At first glance it seems that ppp does not check existing default > route before replacing it with a new one. My thinking as well. Similar thing (creating kind-of an overlay network) with openvpn results in a single default route - apparently the original one is properly removed before adding openvpn's own default route. Kind regads Helge From nobody Fri Sep 10 15:45:54 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A02FC17B165B for ; Fri, 10 Sep 2021 15:46:12 +0000 (UTC) (envelope-from mike@jellydonut.org) Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H5gFv6nl9z4vVh for ; Fri, 10 Sep 2021 15:46:11 +0000 (UTC) (envelope-from mike@jellydonut.org) Received: by mail-ej1-x62a.google.com with SMTP id t19so5090018ejr.8 for ; Fri, 10 Sep 2021 08:46:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jellydonut-org.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=rrpXRXY7xHpayEpOBCUaHEr8la5d0nVKNCogy/KtORQ=; b=Db7ZXsdZMm1awGuqiji7mYdD1uP7uCm3pLDd7GmKIV3z0vH19Vx5h9u+zBMPT2PAU8 LpHROwXLvX+CrjgYyqPN6mW1UKdxlibEi9RyjJWXbxWS3FC8dJbWQFuu3brbvjOH+cB9 Ttx4D4uHpqb/170SN6V101ngFE2Ep3il7WeEWcubWaff3O4lDd5pcHwm62V6Y9sA0utZ mmjY7IYuTiPN2vuNN96azzJ/mtMt/KTkMXJ0JlFfDVFAOY5yAmUeC4BSZm2qLp7UDS/I 6pFN0a7LqSWpxLLdP2ww9Dsr4AO5064hweZViuo6wLkSecsfrqygTk+ZzNn0vw9lbQRe lBVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=rrpXRXY7xHpayEpOBCUaHEr8la5d0nVKNCogy/KtORQ=; b=Wbn5YiIjTyHyW131+wtXjywauAuLtDktyt+46SIN65B6aMz8420PQwogLcCgd9Klgt kof9+7aIYvDenFM4Vl3MOwgllazCDFzroF5yrRxNSVHnvSeXL1K/WeyfW0H77twLWiIr bgiTLSsus2qzHDtgA13V/VtgrrjjIUxunWGnuCo4rS6IbYc5gx28KZbIoTZ5a6T4QVfm d4PoWHPjrWXZZhysLP0eSpZPnRmJTeSefwDS1jO8/HzVM4M41ALWiJyd3gibHT/pbLF4 qOd/dYXIHsYCBo+AegU4HdzFuy6B2FtADPk7N0PtTgKR5ksnIjCQW9Xbir15PhHESw4M Rxvg== X-Gm-Message-State: AOAM530B4Omf6CpUiiDcmClQVSYokLfXqhgQXZBBuV33UtxwGzX0Pqag Vc4BLGleodkO4FeeWqsQNLqUfV8qZATuTIFAgoZGS0d7GQKEbw== X-Google-Smtp-Source: ABdhPJzaDc6XQg9OFU8UguIv67cFtTTwqAlHzm+/b1Vk8hfeVZefMGWcQehqpr+NXDakGHqEgd6LIQ5kUPrbQld6buU= X-Received: by 2002:a17:906:a0c9:: with SMTP id bh9mr10131802ejb.51.1631288764892; Fri, 10 Sep 2021 08:46:04 -0700 (PDT) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 From: Michael Proto Date: Fri, 10 Sep 2021 11:45:54 -0400 Message-ID: Subject: Constant state-changes in a ZFS array To: FreeBSD-STABLE Mailing List , freebsd-hardware@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4H5gFv6nl9z4vVh X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=jellydonut-org.20150623.gappssmtp.com header.s=20150623 header.b=Db7ZXsdZ; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@jellydonut.org designates 2a00:1450:4864:20::62a as permitted sender) smtp.mailfrom=mike@jellydonut.org X-Spamd-Result: default: False [-1.50 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[jellydonut-org.20150623.gappssmtp.com:s=20150623]; FREEFALL_USER(0.00)[mike]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[jellydonut.org]; NEURAL_SPAM_SHORT(1.00)[1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[jellydonut-org.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62a:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Hey all, I had a server go dark earlier this week and after several hardware swaps I'm left scratching my head. The server is a HP DL380p Gen8 with a D3600 shelf attached, using 2 EF0600FARNA HP 600G disks in a ZFS mirror (da0 and da1) and another 22 8TB Ultrastar disks in a ZFS RAID10 for data (da3 through da23, though da23 has been removed in this situation). They're all attached to a LSI SAS2308 operating in HBA mode. The large array threw a disk shortly before which we would normally handle online like we've done dozens of times before. In this case there's a bigger problem I'm struggling with. In addition to the disk being thrown I'm now unable to bring the larger ZFS array online. Commands issued to check array status or bring it online during boot are stalling. The 2-disk zroot mirror is recognized on boot and is loading, so I can get into the OS as normal with the larger tank array failing to come online. Looking at syslog I'm seeing a regular stream of messages coming from devd regarding media and state-change events from both ZFS, GEOM and DEVFS. Sample below: Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS subsystem=CDEV type=MEDIACHANGE cdev=da2' Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM subsystem=DEV type=MEDIACHANGE cdev=da2' Sep 10 04:28:33 backup11 devd: Processing event '!system=ZFS subsystem=ZFS type=resource.fs.zfs.statechange version=0 class=resource.fs.zfs.statechange pool_guid=9328454021323814501 vdev_guid=8915574321583737794' Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS subsystem=CDEV type=MEDIACHANGE cdev=da2p1' Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM subsystem=DEV type=MEDIACHANGE cdev=da2p1' Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS subsystem=CDEV type=MEDIACHANGE cdev=da6' Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS subsystem=CDEV type=MEDIACHANGE cdev=da2' Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM subsystem=DEV type=MEDIACHANGE cdev=da2' Sep 10 04:28:33 backup11 devd: Processing event '!system=ZFS subsystem=ZFS type=resource.fs.zfs.statechange version=0 class=resource.fs.zfs.statechange pool_guid=9328454021323814501 vdev_guid=8915574321583737794' Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS subsystem=CDEV type=MEDIACHANGE cdev=da2p1' Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM subsystem=DEV type=MEDIACHANGE cdev=da2p1' Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS subsystem=CDEV type=MEDIACHANGE cdev=da6' Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM subsystem=DEV type=MEDIACHANGE cdev=da6' Sep 10 04:28:33 backup11 devd: Processing event '!system=ZFS subsystem=ZFS type=resource.fs.zfs.statechange version=0 class=resource.fs.zfs.statechange pool_guid=9328454021323814501 vdev_guid=7024987654522270730' Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS subsystem=CDEV type=MEDIACHANGE cdev=da6p1' Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM subsystem=DEV type=MEDIACHANGE cdev=da6p1' Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS subsystem=CDEV type=MEDIACHANGE cdev=da9' Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM subsystem=DEV type=MEDIACHANGE cdev=da9' Sep 10 04:28:33 backup11 devd: Processing event '!system=ZFS subsystem=ZFS type=resource.fs.zfs.statechange version=0 class=resource.fs.zfs.statechange pool_guid=9328454021323814501 vdev_guid=4207599288564790488' Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS subsystem=CDEV type=MEDIACHANGE cdev=da9p1' Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM subsystem=DEV type=MEDIACHANGE cdev=da9p1' The disk devices appearing in these messages are all disks in the RAID10 array. They appear as a group, every 5 seconds. Furthermore the state changes seem to be happening evenly across all affected devices with the exception of da15 which is precisely half the volume. Here's a count from /var/log/messages piped to sort and uniq (count, then message): 32100 cdev=da10' 32100 cdev=da10p1' 32100 cdev=da11' 32100 cdev=da11p1' 32100 cdev=da12' 32100 cdev=da12p1' 32100 cdev=da13' 32100 cdev=da13p1' 32100 cdev=da14' 32100 cdev=da14p1' 16050 cdev=da15' 16050 cdev=da15p1' 32100 cdev=da16' 32100 cdev=da16p1' 32100 cdev=da17' 32100 cdev=da17p1' 32100 cdev=da18' 32100 cdev=da18p1' 32100 cdev=da19' 32100 cdev=da19p1' 32100 cdev=da2' 32100 cdev=da20' 32100 cdev=da20p1' 32100 cdev=da21' 32100 cdev=da21p1' 32100 cdev=da22' 32100 cdev=da22p1' 32100 cdev=da2p1' 32100 cdev=da3' 32100 cdev=da3p1' 32100 cdev=da4' 32100 cdev=da4p1' 32100 cdev=da5' 32100 cdev=da5p1' 32100 cdev=da6' 32100 cdev=da6p1' 32100 cdev=da7' 32100 cdev=da7p1' 32100 cdev=da8' 32100 cdev=da8p1' 32100 cdev=da9' 32100 cdev=da9p1' I can run diskinfo against all the listed disks no problem and I see them via camcontrol. I can also issue a reset via camcontrol to both the chassis and the D3600 with no issues. sesutil-map sees the chassis, shelf, and all disk devices. So far I've swapped the LSI controller, the D3600 shelf (twice) and the cabling, same behavior. Previously when a collection of disks go problematic like this we've swapped the D3600 shelf or occasionally just reseated the external cabling and everything came back normal. Not this time. I'm scheduling a chassis swap for next week but figured I'd throw this out here to see if anyone has seen this before. Thanks! Mike Proto From nobody Fri Sep 10 15:52:15 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id CFFFC17B596F for ; Fri, 10 Sep 2021 15:52:16 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [IPv6:2607:f3e0:0:3::19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pyroxene.sentex.ca", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H5gNw5MSbz3FF5 for ; Fri, 10 Sep 2021 15:52:16 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [IPv6:2607:f3e0:0:4:e130:b43:acb5:d4fe] ([IPv6:2607:f3e0:0:4:e130:b43:acb5:d4fe]) by pyroxene2a.sentex.ca (8.16.1/8.15.2) with ESMTPS id 18AFqFMd010092 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Fri, 10 Sep 2021 11:52:15 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: routing changes from Sept 7th and possible ppp breakage (RELENG_13) To: Helge Oldach , Zhenlei Huang Cc: freebsd-stable@freebsd.org References: <202109101534.18AFYarp001702@nuc.oldach.net> From: mike tancsa Message-ID: <24c853a3-bd84-58ac-e835-919f31b4d9ad@sentex.net> Date: Fri, 10 Sep 2021 11:52:15 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 In-Reply-To: <202109101534.18AFYarp001702@nuc.oldach.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4H5gNw5MSbz3FF5 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On 9/10/2021 11:34 AM, Helge Oldach wrote: > Zhenlei Huang wrote on Fri, 10 Sep 2021 10:34:20 +0200 (CEST): >>> On Sep 10, 2021, at 5:38 AM, mike tancsa wrote: >>> default 10.255.255.1 UGS 6 1500 igb0 >>> default 10.1.0.1 UGS 17 1500 tun10 >>> >>> Where as now, it doesnt seem to replace it, it seems to just add >>> another default route. >> At first glance it seems that ppp does not check existing default >> route before replacing it with a new one. > My thinking as well. Similar thing (creating kind-of an overlay network) > with openvpn results in a single default route - apparently the original > one is properly removed before adding openvpn's own default route. > One other thing I noticed is that net.route.multipath is RO on i386 but RW on AMD64.  Is that intentional for some reason ?     ---Mike From nobody Fri Sep 10 16:01:47 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 815D117B9E74 for ; Fri, 10 Sep 2021 16:01:49 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [IPv6:2607:f3e0:0:3::19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pyroxene.sentex.ca", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H5gbv6ltdz3JSm for ; Fri, 10 Sep 2021 16:01:47 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [IPv6:2607:f3e0:0:4:e130:b43:acb5:d4fe] ([IPv6:2607:f3e0:0:4:e130:b43:acb5:d4fe]) by pyroxene2a.sentex.ca (8.16.1/8.15.2) with ESMTPS id 18AG1lo5013069 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Fri, 10 Sep 2021 12:01:47 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: routing changes from Sept 7th and possible ppp breakage (RELENG_13) From: mike tancsa To: Helge Oldach , Zhenlei Huang Cc: freebsd-stable@freebsd.org References: <202109101534.18AFYarp001702@nuc.oldach.net> <24c853a3-bd84-58ac-e835-919f31b4d9ad@sentex.net> Message-ID: <376c0ad2-39b2-2d20-78a6-dbc122bf0605@sentex.net> Date: Fri, 10 Sep 2021 12:01:47 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 In-Reply-To: <24c853a3-bd84-58ac-e835-919f31b4d9ad@sentex.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4H5gbv6ltdz3JSm X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:3::19 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-0.10 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[mike]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HFILTER_HELO_IP_A(1.00)[pyroxene2a.sentex.ca]; DMARC_NA(0.00)[sentex.net]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_SHORT(1.00)[1.000]; HFILTER_HELO_NORES_A_OR_MX(0.30)[pyroxene2a.sentex.ca]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FREEMAIL_TO(0.00)[oldach.net,gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[2607:f3e0:0:3::19:from] X-ThisMailContainsUnwantedMimeParts: N On 9/10/2021 11:52 AM, mike tancsa wrote: > On 9/10/2021 11:34 AM, Helge Oldach wrote: >> Zhenlei Huang wrote on Fri, 10 Sep 2021 10:34:20 +0200 (CEST): >>>> On Sep 10, 2021, at 5:38 AM, mike tancsa wrote: >>>> default 10.255.255.1 UGS 6 1500 igb0 >>>> default 10.1.0.1 UGS 17 1500 tun10 >>>> >>>> Where as now, it doesnt seem to replace it, it seems to just add >>>> another default route. >>> At first glance it seems that ppp does not check existing default >>> route before replacing it with a new one. >> My thinking as well. Similar thing (creating kind-of an overlay network) >> with openvpn results in a single default route - apparently the original >> one is properly removed before adding openvpn's own default route. >> > One other thing I noticed is that net.route.multipath is RO on i386 but > RW on AMD64.  Is that intentional for some reason ? > Actually, its neither. On my i386 image I was using an old kernel def that didnt have options         ROUTE_MPATH  yet the sysctl still shows up with a value of 1 # sysctl net.route.multipath net.route.multipath: 1 # sysctl -w net.route.multipath=0 sysctl: oid 'net.route.multipath' is read only #     ---Mike From nobody Fri Sep 10 16:10:05 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 78E9B17BE017 for ; Fri, 10 Sep 2021 16:10:24 +0000 (UTC) (envelope-from mike@jellydonut.org) Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H5gnq5Q42z3Ls5 for ; Fri, 10 Sep 2021 16:10:23 +0000 (UTC) (envelope-from mike@jellydonut.org) Received: by mail-ej1-x630.google.com with SMTP id h9so5295431ejs.4 for ; Fri, 10 Sep 2021 09:10:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jellydonut-org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=Wvf1Ija0NxbcFuEB/z/eESMX1+9QwOv0/+26AGT1vGY=; b=qWyai5164BROexPhv6z9CbeAdSDUualXISxZHhhqouvKrk1fuhdOTJEwvnbdk85gv6 5qBjIm3NBu1XoTGjQleMQOZMFVe67v7aXsHcLk7AtZw/OVXhyYV7hFG359tEWd9JZCUk fo3ajs64C9tGK7RW2SXl+SAWLoycWj0GaIhwsdVeoH7whnhFHZhrtslIQkc11n8cESLQ 5piqR0sVb9dqzpRokCMQrhd5Kgz+dT24Co1vFGwA9pL7tS7DtvtychsaxYpQ+o75Y+XS 5KxTWXI0/K+hWFer+J9+jtPjQrkQ/ge/uto90dv8c/4dg//oeT1m2nenIp3TUZLOHEoA Zs1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=Wvf1Ija0NxbcFuEB/z/eESMX1+9QwOv0/+26AGT1vGY=; b=fYCNdBInRaXH9x94PP3GIM+GeRl9aQzCmzsRuDMvx9eGdKDLybLsy4IpFcFmhX70// SX4/1Q2IIZTYGuvMWWDkrk5YWyz0ZqAw6yaBBpxK1pFIW6BTHCE0cEwKKVxOveAiQ8ob X9Zkt7zRiWsWfzsU6rVp/R9SqWmlO9IyLrgXBKNecHi/9tu1dCkFNQWW33YgnMtdbcc0 9ylL/YwweOOmQvskSajKooDzXGaZ2NafEmF8AsXJbtTksa1ka1+TBWs79dHBb3xXWY1e K2vkzpu1KL95PgoBvCeLm9fTTMUi3bi0AVv9Wvv+hS/my7dDzACUoYMgdsAUAX3tL9oN meRA== X-Gm-Message-State: AOAM532zIwU2M741mBOy8Ywc0ozmGWazqz+M4rmdkQvVM9Nn9B6ruGib rDW0b5deS0dwIwLFGhd7P30t79ZKkqBo71WOk0tvo0mcezI= X-Google-Smtp-Source: ABdhPJxPwKJjMM2OYpNnDieJHsxsrf4x3zcPhWzyC3eSEki6MfjlUZiZ43SBeoG9kJnaM/QSgfPdtTK0dR9n880YQy4= X-Received: by 2002:a17:906:7802:: with SMTP id u2mr10210201ejm.325.1631290216484; Fri, 10 Sep 2021 09:10:16 -0700 (PDT) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Michael Proto Date: Fri, 10 Sep 2021 12:10:05 -0400 Message-ID: Subject: Re: Constant state-changes in a ZFS array To: FreeBSD-STABLE Mailing List , freebsd-hardware@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4H5gnq5Q42z3Ls5 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=jellydonut-org.20150623.gappssmtp.com header.s=20150623 header.b=qWyai516; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@jellydonut.org designates 2a00:1450:4864:20::630 as permitted sender) smtp.mailfrom=mike@jellydonut.org X-Spamd-Result: default: False [-1.50 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[jellydonut-org.20150623.gappssmtp.com:s=20150623]; FREEFALL_USER(0.00)[mike]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[jellydonut.org]; NEURAL_SPAM_SHORT(1.00)[1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[jellydonut-org.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::630:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Just realized I neglected version info. This is on FreeBSD 11.4-RELEASE-p3 On Fri, Sep 10, 2021 at 11:45 AM Michael Proto wrote: > > Hey all, > > I had a server go dark earlier this week and after several hardware > swaps I'm left scratching my head. The server is a HP DL380p Gen8 with > a D3600 shelf attached, using 2 EF0600FARNA HP 600G disks in a ZFS > mirror (da0 and da1) and another 22 8TB Ultrastar disks in a ZFS > RAID10 for data (da3 through da23, though da23 has been removed in > this situation). They're all attached to a LSI SAS2308 operating in > HBA mode. > > The large array threw a disk shortly before which we would normally > handle online like we've done dozens of times before. In this case > there's a bigger problem I'm struggling with. In addition to the disk > being thrown I'm now unable to bring the larger ZFS array online. > Commands issued to check array status or bring it online during boot > are stalling. The 2-disk zroot mirror is recognized on boot and is > loading, so I can get into the OS as normal with the larger tank array > failing to come online. > > Looking at syslog I'm seeing a regular stream of messages coming from > devd regarding media and state-change events from both ZFS, GEOM and > DEVFS. Sample below: > > Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS > subsystem=CDEV type=MEDIACHANGE cdev=da2' > Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM > subsystem=DEV type=MEDIACHANGE cdev=da2' > Sep 10 04:28:33 backup11 devd: Processing event '!system=ZFS > subsystem=ZFS type=resource.fs.zfs.statechange version=0 > class=resource.fs.zfs.statechange pool_guid=9328454021323814501 > vdev_guid=8915574321583737794' > Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS > subsystem=CDEV type=MEDIACHANGE cdev=da2p1' > Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM > subsystem=DEV type=MEDIACHANGE cdev=da2p1' > Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS > subsystem=CDEV type=MEDIACHANGE cdev=da6' > Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS > subsystem=CDEV type=MEDIACHANGE cdev=da2' > Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM > subsystem=DEV type=MEDIACHANGE cdev=da2' > Sep 10 04:28:33 backup11 devd: Processing event '!system=ZFS > subsystem=ZFS type=resource.fs.zfs.statechange version=0 > class=resource.fs.zfs.statechange pool_guid=9328454021323814501 > vdev_guid=8915574321583737794' > Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS > subsystem=CDEV type=MEDIACHANGE cdev=da2p1' > Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM > subsystem=DEV type=MEDIACHANGE cdev=da2p1' > Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS > subsystem=CDEV type=MEDIACHANGE cdev=da6' > Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM > subsystem=DEV type=MEDIACHANGE cdev=da6' > Sep 10 04:28:33 backup11 devd: Processing event '!system=ZFS > subsystem=ZFS type=resource.fs.zfs.statechange version=0 > class=resource.fs.zfs.statechange pool_guid=9328454021323814501 > vdev_guid=7024987654522270730' > Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS > subsystem=CDEV type=MEDIACHANGE cdev=da6p1' > Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM > subsystem=DEV type=MEDIACHANGE cdev=da6p1' > Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS > subsystem=CDEV type=MEDIACHANGE cdev=da9' > Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM > subsystem=DEV type=MEDIACHANGE cdev=da9' > Sep 10 04:28:33 backup11 devd: Processing event '!system=ZFS > subsystem=ZFS type=resource.fs.zfs.statechange version=0 > class=resource.fs.zfs.statechange pool_guid=9328454021323814501 > vdev_guid=4207599288564790488' > Sep 10 04:28:33 backup11 devd: Processing event '!system=DEVFS > subsystem=CDEV type=MEDIACHANGE cdev=da9p1' > Sep 10 04:28:33 backup11 devd: Processing event '!system=GEOM > subsystem=DEV type=MEDIACHANGE cdev=da9p1' > > > > The disk devices appearing in these messages are all disks in the > RAID10 array. They appear as a group, every 5 seconds. Furthermore the > state changes seem to be happening evenly across all affected devices > with the exception of da15 which is precisely half the volume. Here's > a count from /var/log/messages piped to sort and uniq (count, then > message): > > 32100 cdev=da10' > 32100 cdev=da10p1' > 32100 cdev=da11' > 32100 cdev=da11p1' > 32100 cdev=da12' > 32100 cdev=da12p1' > 32100 cdev=da13' > 32100 cdev=da13p1' > 32100 cdev=da14' > 32100 cdev=da14p1' > 16050 cdev=da15' > 16050 cdev=da15p1' > 32100 cdev=da16' > 32100 cdev=da16p1' > 32100 cdev=da17' > 32100 cdev=da17p1' > 32100 cdev=da18' > 32100 cdev=da18p1' > 32100 cdev=da19' > 32100 cdev=da19p1' > 32100 cdev=da2' > 32100 cdev=da20' > 32100 cdev=da20p1' > 32100 cdev=da21' > 32100 cdev=da21p1' > 32100 cdev=da22' > 32100 cdev=da22p1' > 32100 cdev=da2p1' > 32100 cdev=da3' > 32100 cdev=da3p1' > 32100 cdev=da4' > 32100 cdev=da4p1' > 32100 cdev=da5' > 32100 cdev=da5p1' > 32100 cdev=da6' > 32100 cdev=da6p1' > 32100 cdev=da7' > 32100 cdev=da7p1' > 32100 cdev=da8' > 32100 cdev=da8p1' > 32100 cdev=da9' > 32100 cdev=da9p1' > > > I can run diskinfo against all the listed disks no problem and I see > them via camcontrol. I can also issue a reset via camcontrol to both > the chassis and the D3600 with no issues. sesutil-map sees the > chassis, shelf, and all disk devices. > > So far I've swapped the LSI controller, the D3600 shelf (twice) and > the cabling, same behavior. Previously when a collection of disks go > problematic like this we've swapped the D3600 shelf or occasionally > just reseated the external cabling and everything came back normal. > Not this time. I'm scheduling a chassis swap for next week but figured > I'd throw this out here to see if anyone has seen this before. > > > > Thanks! > Mike Proto From nobody Sun Sep 12 09:41:52 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2630617BCD42 for ; Sun, 12 Sep 2021 09:41:54 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H6l4f0kDhz3pqK; Sun, 12 Sep 2021 09:41:54 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from [192.168.0.88] (unknown [195.64.148.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: avg/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 975D02763C; Sun, 12 Sep 2021 09:41:53 +0000 (UTC) (envelope-from avg@FreeBSD.org) To: mike tancsa , FreeBSD-STABLE Mailing List References: From: Andriy Gapon Subject: Re: routing changes from Sept 7th and possible ppp breakage (RELENG_13) Message-ID: <67010baa-ecb4-7b9b-a797-efb3e2940fbb@FreeBSD.org> Date: Sun, 12 Sep 2021 12:41:52 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Firefox/78.0 Thunderbird/78.14.0 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-ThisMailContainsUnwantedMimeParts: N On 10/09/2021 00:38, mike tancsa wrote: > On our standard nanobsd router image we use, I noticed failover routing > broke all of a sudden.  We are not using multiple FIBs, but somehow we > now have two default routes I am guessing because of multipath.  (Note, > we just started testing RELENG_13 so its possible I am making bad > assumptions somewhere.) > > default            10.255.255.1       UGS         6   1500       igb0 > default            10.1.0.1           UGS        17   1500      tun10 > > I am pretty sure from the code base from 3+ days ago, I would not see > this condition. I just want to add that I've been seeing the same behavior on CURRENT for a few months now. I think that ppp probably needs some sort of an update to be aware of multipath. BTW, net@ may be a good mailing list to discuss this as well. > In the past, my nano image would get a default route via DHCP.  If that > path was not functional or blocked, my monitoring software would detect > the failure and fire up PPP which has > >  add! default HISADDR > > That now seems to add a second default route instead of getting rid of > the old one. > > if I add net.route.multipath=0 > > I get back the old behaviour > > According to the ppp docs > >          If the add! command is used (note the trailing "!"), then if the >          route already exists, it will be updated as with the `route change' >          command (see route(8) for further details). > > Where as now, it doesnt seem to replace it, it seems to just add another > default route. > > For my purposes, setting net.route.multipath=0 seems to work just fine, > but I wonder if this warrants a note in ppp's docs since it no longer > seems to work that way in RELENG_13 ? -- Andriy Gapon From nobody Sun Sep 12 11:55:23 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id F296617AED5E for ; Sun, 12 Sep 2021 11:55:24 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [IPv6:2607:f3e0:0:3::19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pyroxene.sentex.ca", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H6p2h5vJyz3Lvb; Sun, 12 Sep 2021 11:55:24 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [IPv6:2607:f3e0:0:4:7d4a:ebfa:d7d9:2a31] ([IPv6:2607:f3e0:0:4:7d4a:ebfa:d7d9:2a31]) by pyroxene2a.sentex.ca (8.16.1/8.15.2) with ESMTPS id 18CBtNGo019191 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Sun, 12 Sep 2021 07:55:23 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: routing changes from Sept 7th and possible ppp breakage (RELENG_13) To: Andriy Gapon , FreeBSD-STABLE Mailing List References: <67010baa-ecb4-7b9b-a797-efb3e2940fbb@FreeBSD.org> From: mike tancsa Message-ID: <704f17bf-0a23-56c3-3bca-70e48417ad1c@sentex.net> Date: Sun, 12 Sep 2021 07:55:23 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 In-Reply-To: <67010baa-ecb4-7b9b-a797-efb3e2940fbb@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4H6p2h5vJyz3Lvb X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On 9/12/2021 5:41 AM, Andriy Gapon wrote: > On 10/09/2021 00:38, mike tancsa wrote: >> On our standard nanobsd router image we use, I noticed failover routing >> broke all of a sudden.  We are not using multiple FIBs, but somehow we >> now have two default routes I am guessing because of multipath.  (Note, >> we just started testing RELENG_13 so its possible I am making bad >> assumptions somewhere.) >> >> default            10.255.255.1       UGS         6   1500       igb0 >> default            10.1.0.1           UGS        17   1500      tun10 >> >> I am pretty sure from the code base from 3+ days ago, I would not see >> this condition. > > I just want to add that I've been seeing the same behavior on CURRENT > for a few months now. > I think that ppp probably needs some sort of an update to be aware of > multipath. > I added https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258447     ---Mike From nobody Sun Sep 12 12:53:58 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2A40517A5911 for ; Sun, 12 Sep 2021 12:54:12 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from land.berklix.org (land.berklix.org [144.76.10.75]) by mx1.freebsd.org (Postfix) with ESMTP id 4H6qLV6Strz4V1K for ; Sun, 12 Sep 2021 12:54:10 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from mart.js.berklix.net (p4fe6dae8.dip0.t-ipconnect.de [79.230.218.232]) (authenticated bits=128) by land.berklix.org (8.15.2/8.15.2) with ESMTPA id 18CCs4qf008779 for ; Sun, 12 Sep 2021 12:54:04 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by mart.js.berklix.net (8.14.3/8.14.3) with ESMTP id 18CCrwqh021785 for ; Sun, 12 Sep 2021 14:53:59 +0200 (CEST) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.16.1/8.16.1) with ESMTP id 18CCrw1O000484 for ; Sun, 12 Sep 2021 14:53:58 +0200 (CEST) (envelope-from jhs@berklix.com) Message-Id: <202109121253.18CCrw1O000484@fire.js.berklix.net> To: FreeBSD-STABLE Mailing List Subject: src/lib/libgcc_s needs mv /usr/obj/usr/src/amd64.amd64/lib/libc/libc.a From: "Julian H. Stacey" Organization: http://berklix.com/jhs/ User-agent: EXMH on FreeBSD http://www.berklix.eu/free/ X-From: http://www.berklix.eu/~jhs/ List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <482.1631451238.1@fire.js.berklix.net> Content-Transfer-Encoding: quoted-printable Date: Sun, 12 Sep 2021 14:53:58 +0200 X-Rspamd-Queue-Id: 4H6qLV6Strz4V1K X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jhs@berklix.com has no SPF policy when checking 144.76.10.75) smtp.mailfrom=jhs@berklix.com X-Spamd-Result: default: False [-0.33 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[jhs]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; URI_HIDDEN_PATH(1.00)[http://berklix.com/~jhs/bin/.csh/unsetenv.csh]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[berklix.com]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; TO_DN_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.33)[-0.332]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:144.76.0.0/16, country:DE]; RECEIVED_SPAMHAUS_PBL(0.00)[79.230.218.232:received] X-ThisMailContainsUnwantedMimeParts: N Hi all, Anyone else seen this ? After cd /usr/src ; make install this fails cd /usr/src/lib/libgcc_s ; make until a manual mv /usr/obj/usr/src/amd64.amd64/lib/libc/libc.a \ /usr/obj/usr/src/amd64.amd64/lib/libc/libc.a.MV when = cd /usr/src ; make all ; make install can run OK, but then again needs another 'mv'. Identical: md5 /usr/obj/usr/src/amd64.amd64/lib/libc/libc.a* /usr/lib/libc= .a Maybe its changing where linking from ? ** I've had this on & off for months I think, on multiple hardware, but possi= bly just 12.2-STABLE, = it's still so with a stable src/ fetched a day or ago with git pull --ff-o= nly To help simplify debug, I have no /etc/src.conf /etc/make.conf I also stripped my env with http://berklix.com/~jhs/bin/.csh/unsetenv.csh source `which unsetenv.csh` Remaining: DESTDIR DISPLAY HOSTDISPLAY PATH TERM TERMCAP TERMPATH None of that helps. I am running a 12.2 self built system, uncustomised src/, no kernel module= s, uname -a FreeBSD fire.js.berklix.net 12.2-RELEASE FreeBSD 12.2-RELEASE #0: Sat May 22 20:41:18 CEST 2021 jhs@fire.js.berklix.net:/1s4/ftp/pri/FreeBSD/releases/12.2-RELEASE/generi= c/src/sys/amd64/compile/GENERIC amd64 with the exception of /boot/kernel which is 12.2-RELEASE as for some unkno= wn reason I cant boot a local compiled or cross compiled 12.2-STABLE kernel (I'm still investigating that presumed un-associated issue) ** Here's a script log : {------- =3D=3D=3D> etc (install) =3D=3D=3D> etc/sendmail (install) cd /usr1/src/share/man; make makedb makewhatis /usr/share/man makewhatis /usr/share/openssl/man .... cd /usr/src/lib/libgcc_s ; make cat /usr1/src/lib/libgcc_s/Symbol.map /usr1/src/lib/libgcc_s/SymbolDefault= .map | cpp - - | awk -v vfile=3D/usr1/src/lib/libgcc_s/Versions.def -f /u= sr/share/mk/version_gen.awk > Version.map building shared library libgcc_s.so.1 cc -nodefaultlibs -Wl,--version-script=3DVersion.map -shared -Wl,-x -Wl= ,--fatal-warnings -Wl,--warn-shared-textrel -o libgcc_s.so.1.full -Wl,-so= name,libgcc_s.so.1 `NM=3D'nm' NMFLAGS=3D'' lorder i386/fp_mode.pico absvd= i2.pico absvsi2.pico absvti2.pico addvdi3.pico addvsi3.pico addvti3.pico a= pple_versioning.pico ashldi3.pico ashlti3.pico ashrdi3.pico ashrti3.pico b= swapdi2.pico bswapsi2.pico clear_cache.pico clzdi2.pico clzsi2.pico clzti2= .pico cmpdi2.pico cmpti2.pico ctzdi2.pico ctzsi2.pico ctzti2.pico divdc3.p= ico divdi3.pico divmoddi4.pico divmodsi4.pico divsc3.pico divsi3.pico divt= c3.pico divti3.pico divxc3.pico enable_execute_stack.pico eprintf.pico ext= endhfsf2.pico ffsdi2.pico ffssi2.pico ffsti2.pico fixdfdi.pico fixdfti.pic= o fixsfdi.pico fixsfti.pico fixunsdfdi.pico fixunsdfsi.pico fixunsdfti.pic= o fixunssfdi.pico fixunssfsi.pico fixunssfti.pico fixunsxfdi.pico fixunsxf= si.pico fixunsxfti.pico fixxfdi.pico fixxfti.pico floatditf.pico floattidf= .pico floattisf.pico floattixf.pico floatunditf.pico floatunsidf.pico floa= tunsisf.pico floatuntidf.pico floatuntisf.pico floatuntixf.pico gcc_person= ality_v0.pico int_util.pico lshrdi3.pico lshrti3.pico moddi3.pico modsi3.p= ico modti3.pico muldc3.pico muldi3.pico mulodi4.pico mulosi4.pico muloti4.= pico mulsc3.pico multc3.pico multi3.pico mulvdi3.pico mulvsi3.pico mulvti3= .pico mulxc3.pico negdf2.pico negdi2.pico negsf2.pico negti2.pico negvdi2.= pico negvsi2.pico negvti2.pico paritydi2.pico paritysi2.pico parityti2.pic= o popcountdi2.pico popcountsi2.pico popcountti2.pico powidf2.pico powisf2.= pico powitf2.pico powixf2.pico subvdi3.pico subvsi3.pico subvti3.pico tram= poline_setup.pico truncdfhf2.pico truncsfhf2.pico ucmpdi2.pico ucmpti2.pic= o udivdi3.pico udivmoddi4.pico udivmodsi4.pico udivmodti4.pico udivsi3.pic= o udivti3.pico umoddi3.pico umodsi3.pico umodti3.pico floatdidf.pico float= disf.pico floatdixf.pico floatundidf.pico floatundisf.pico floatundixf.pic= o cpu_model.pico adddf3.pico addsf3.pico divdf3.pico divsf3.pico extendsfd= f2.pico fixdfsi.pico fixsfsi.pico floatsidf.pico floatsisf.pico muldf3.pic= o mulsf3.pico subdf3.pico subsf3.pico truncdfsf2.pico comparedf2.pico comp= aresf2.pico gcc_personality_v0.pico int_util.pico Unwind-EHABI.pico Unwind= -sjlj.pico UnwindLevel1-gcc-ext.pico UnwindLevel1.pico UnwindRegistersRest= ore.pico UnwindRegistersSave.pico libunwind.pico s_fabs.pico s_fabsf.pico = s_fabsl.pico s_fmax.pico s_fmaxf.pico s_logb.pico s_logbf.pico s_scalbn.pi= co s_scalbnf.pico s_fmaxl.pico s_logbl.pico s_scalbnl.pico | tsort -q` -L= /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc -lc ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32S aga= inst symbol: __je_sz_size2index_tab in readonly segment; recompile object = files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the = output >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_sz.o) >>> referenced by sz.h:158 (/usr1/src/contrib/jemalloc/include/jemalloc/in= ternal/sz.h:158) >>> jemalloc_jemalloc.o:(a0ialloc) in archive /1s4/release/1= 2.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32S aga= inst symbol: __je_arenas in readonly segment; recompile object files with = -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_jemalloc.o) >>> referenced by atomic.h:55 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/atomic.h:55) >>> jemalloc_jemalloc.o:(a0ialloc) in archive /1s4/release/1= 2.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32S aga= inst symbol: __je_sz_index2size_tab in readonly segment; recompile object = files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the = output >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_sz.o) >>> referenced by sz.h:201 (/usr1/src/contrib/jemalloc/include/jemalloc/in= ternal/sz.h:201) >>> jemalloc_jemalloc.o:(a0ialloc) in archive /1s4/release/1= 2.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32 agai= nst local symbol in readonly segment; recompile object files with -fPIC or= pass '-Wl,-z,notext' to allow text relocations in the output >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_jemalloc.o) >>> referenced by mutex.h:144 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/mutex.h:144) >>> jemalloc_jemalloc.o:(a0ialloc) in archive /1s4/release/1= 2.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32 agai= nst local symbol in readonly segment; recompile object files with -fPIC or= pass '-Wl,-z,notext' to allow text relocations in the output >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_jemalloc.o) >>> referenced by mutex.h:203 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/mutex.h:203) >>> jemalloc_jemalloc.o:(a0ialloc) in archive /1s4/release/1= 2.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32 agai= nst local symbol in readonly segment; recompile object files with -fPIC or= pass '-Wl,-z,notext' to allow text relocations in the output >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_jemalloc.o) >>> referenced by mutex.h:214 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/mutex.h:214) >>> jemalloc_jemalloc.o:(a0ialloc) in archive /1s4/release/1= 2.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32 agai= nst symbol: __je_extent_hooks_default in readonly segment; recompile objec= t files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in th= e output >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_extent.o) >>> referenced by jemalloc_internal_inlines_a.h:91 (/usr1/src/contrib/jema= lloc/include/jemalloc/internal/jemalloc_internal_inlines_a.h:91) >>> jemalloc_jemalloc.o:(a0ialloc) in archive /1s4/release/1= 2.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32 agai= nst symbol: __je_extents_rtree in readonly segment; recompile object files= with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the outpu= t >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_extent.o) >>> referenced by rtree.h:381 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/rtree.h:381) >>> jemalloc_jemalloc.o:(a0ialloc) in archive /1s4/release/1= 2.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32 agai= nst symbol: __je_extents_rtree in readonly segment; recompile object files= with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the outpu= t >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_extent.o) >>> referenced by rtree.h:381 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/rtree.h:381) >>> jemalloc_jemalloc.o:(a0ialloc) in archive /1s4/release/1= 2.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32S aga= inst symbol: __je_arenas in readonly segment; recompile object files with = -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_jemalloc.o) >>> referenced by atomic.h:55 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/atomic.h:55) >>> jemalloc_jemalloc.o:(a0idalloc) in archive /1s4/release/= 12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32S aga= inst symbol: __je_sz_index2size_tab in readonly segment; recompile object = files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the = output >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_sz.o) >>> referenced by sz.h:201 (/usr1/src/contrib/jemalloc/include/jemalloc/in= ternal/sz.h:201) >>> jemalloc_jemalloc.o:(a0idalloc) in archive /1s4/release/= 12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32 agai= nst symbol: __je_extents_rtree in readonly segment; recompile object files= with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the outpu= t >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_extent.o) >>> referenced by rtree.h:381 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/rtree.h:381) >>> jemalloc_jemalloc.o:(a0idalloc) in archive /1s4/release/= 12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32 agai= nst symbol: __je_extents_rtree in readonly segment; recompile object files= with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the outpu= t >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_extent.o) >>> referenced by rtree.h:381 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/rtree.h:381) >>> jemalloc_jemalloc.o:(a0idalloc) in archive /1s4/release/= 12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32 agai= nst symbol: __je_extents_rtree in readonly segment; recompile object files= with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the outpu= t >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_extent.o) >>> referenced by rtree.h:381 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/rtree.h:381) >>> jemalloc_jemalloc.o:(a0idalloc) in archive /1s4/release/= 12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32 agai= nst symbol: __je_extents_rtree in readonly segment; recompile object files= with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the outpu= t >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_extent.o) >>> referenced by rtree.h:381 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/rtree.h:381) >>> jemalloc_jemalloc.o:(a0idalloc) in archive /1s4/release/= 12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32S aga= inst symbol: __je_arenas in readonly segment; recompile object files with = -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_jemalloc.o) >>> referenced by atomic.h:55 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/atomic.h:55) >>> jemalloc_jemalloc.o:(__je_arena_set) in archive /1s4/rel= ease/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32 agai= nst symbol: __je_arenas_lock in readonly segment; recompile object files w= ith -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_jemalloc.o) >>> referenced by mutex.h:144 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/mutex.h:144) >>> jemalloc_jemalloc.o:(__je_arena_init) in archive /1s4/re= lease/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32 agai= nst symbol: __je_arenas_lock in readonly segment; recompile object files w= ith -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_jemalloc.o) >>> referenced by mutex.h:203 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/mutex.h:203) >>> jemalloc_jemalloc.o:(__je_arena_init) in archive /1s4/re= lease/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32S aga= inst symbol: __je_arenas in readonly segment; recompile object files with = -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_jemalloc.o) >>> referenced by atomic.h:55 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/atomic.h:55) >>> jemalloc_jemalloc.o:(__je_arena_init) in archive /1s4/re= lease/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mcan't create dynamic relocation R_X86_64_32 agai= nst symbol: __je_arenas_lock in readonly segment; recompile object files w= ith -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output >>> defined in /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/li= bc/libc.a(jemalloc_jemalloc.o) >>> referenced by mutex.h:214 (/usr1/src/contrib/jemalloc/include/jemalloc= /internal/mutex.h:214) >>> jemalloc_jemalloc.o:(__je_arena_init) in archive /1s4/re= lease/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc/libc.a ld: =1B[0;31merror: =1B[0mtoo many errors emitted, stopping now (use -erro= r-limit=3D0 to see all errors) cc: =1B[0;1;31merror: =1B[0mlinker command failed with exit code 1 (use -v= to see invocation)=1B[0m *** Error code 1 Stop. make: stopped in /usr1/src/lib/libgcc_s 12.2-RELEASE /dev/pts/2 jhs 1 fire/usr1/src/lib/libgcc_s ls -l /usr/obj/u= sr/src/amd64.amd64/lib/libc/libc.a -rw-r--r-- 1 jhs staff 17050712 Sep 12 13:28 /usr/obj/usr/src/amd64.amd= 64/lib/libc/libc.a 12.2-RELEASE /dev/pts/2 jhs 2 fire/usr1/src/lib/libgcc_s mv /usr/obj/usr/s= rc/amd64.amd64/lib/libc/libc.a /usr/obj/usr/src/amd64.amd64/lib/libc/libc.= a.MV 12.2-RELEASE /dev/pts/2 jhs 3 fire/usr1/src/lib/libgcc_s make building shared library libgcc_s.so.1 cc -nodefaultlibs -Wl,--version-script=3DVersion.map -shared -Wl,-x -Wl= ,--fatal-warnings -Wl,--warn-shared-textrel -o libgcc_s.so.1.full -Wl,-so= name,libgcc_s.so.1 `NM=3D'nm' NMFLAGS=3D'' lorder i386/fp_mode.pico absvd= i2.pico absvsi2.pico absvti2.pico addvdi3.pico addvsi3.pico addvti3.pico a= pple_versioning.pico ashldi3.pico ashlti3.pico ashrdi3.pico ashrti3.pico b= swapdi2.pico bswapsi2.pico clear_cache.pico clzdi2.pico clzsi2.pico clzti2= .pico cmpdi2.pico cmpti2.pico ctzdi2.pico ctzsi2.pico ctzti2.pico divdc3.p= ico divdi3.pico divmoddi4.pico divmodsi4.pico divsc3.pico divsi3.pico divt= c3.pico divti3.pico divxc3.pico enable_execute_stack.pico eprintf.pico ext= endhfsf2.pico ffsdi2.pico ffssi2.pico ffsti2.pico fixdfdi.pico fixdfti.pic= o fixsfdi.pico fixsfti.pico fixunsdfdi.pico fixunsdfsi.pico fixunsdfti.pic= o fixunssfdi.pico fixunssfsi.pico fixunssfti.pico fixunsxfdi.pico fixunsxf= si.pico fixunsxfti.pico fixxfdi.pico fixxfti.pico floatditf.pico floattidf= .pico floattisf.pico floattixf.pico floatunditf.pico floatunsidf.pico floa= tunsisf.pico floatuntidf.pico floatuntisf.pico floatuntixf.pico gcc_person= ality_v0.pico int_util.pico lshrdi3.pico lshrti3.pico moddi3.pico modsi3.p= ico modti3.pico muldc3.pico muldi3.pico mulodi4.pico mulosi4.pico muloti4.= pico mulsc3.pico multc3.pico multi3.pico mulvdi3.pico mulvsi3.pico mulvti3= .pico mulxc3.pico negdf2.pico negdi2.pico negsf2.pico negti2.pico negvdi2.= pico negvsi2.pico negvti2.pico paritydi2.pico paritysi2.pico parityti2.pic= o popcountdi2.pico popcountsi2.pico popcountti2.pico powidf2.pico powisf2.= pico powitf2.pico powixf2.pico subvdi3.pico subvsi3.pico subvti3.pico tram= poline_setup.pico truncdfhf2.pico truncsfhf2.pico ucmpdi2.pico ucmpti2.pic= o udivdi3.pico udivmoddi4.pico udivmodsi4.pico udivmodti4.pico udivsi3.pic= o udivti3.pico umoddi3.pico umodsi3.pico umodti3.pico floatdidf.pico float= disf.pico floatdixf.pico floatundidf.pico floatundisf.pico floatundixf.pic= o cpu_model.pico adddf3.pico addsf3.pico divdf3.pico divsf3.pico extendsfd= f2.pico fixdfsi.pico fixsfsi.pico floatsidf.pico floatsisf.pico muldf3.pic= o mulsf3.pico subdf3.pico subsf3.pico truncdfsf2.pico comparedf2.pico comp= aresf2.pico gcc_personality_v0.pico int_util.pico Unwind-EHABI.pico Unwind= -sjlj.pico UnwindLevel1-gcc-ext.pico UnwindLevel1.pico UnwindRegistersRest= ore.pico UnwindRegistersSave.pico libunwind.pico s_fabs.pico s_fabsf.pico = s_fabsl.pico s_fmax.pico s_fmaxf.pico s_logb.pico s_logbf.pico s_scalbn.pi= co s_scalbnf.pico s_fmaxl.pico s_logbl.pico s_scalbnl.pico | tsort -q` -L= /1s4/release/12.2-STABLE/usr/obj/usr/src/amd64.amd64/lib/libc -lc objcopy --only-keep-debug libgcc_s.so.1.full libgcc_s.so.1.debug objcopy --strip-debug --add-gnu-debuglink=3Dlibgcc_s.so.1.debug libgcc_s.= so.1.full libgcc_s.so.1 12.2-RELEASE /dev/pts/2 jhs 4 fire/usr1/src/lib/libgcc_s = -------} Cheers, -- = Julian Stacey http://berklix.com/jhs/ http://stolenvotes.uk