From nobody Tue May 16 23:06:51 2023 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QLX1R3Ss7z4BZ5t for ; Tue, 16 May 2023 23:06:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QLX1R2D8yz4W5R for ; Tue, 16 May 2023 23:06:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1684278411; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=4B+jyreqbKqM+CKZ8tlbNks9aeOmIihKB7bq2oupAm8=; b=DbKWKsY/Lkf8jJzIkEmBLPFKsDqUVaxo8QBcXubNWd7dxO+QJPIhStFMNeDGSmkGOCAOF1 /tVDMq6b9sIMOj/OQeIh8PCrGG0CMhYyTAGGNohQOdhcUvV7k8f81tkbybgXGbblGl9OHr k32f/hyQ5GML+iFVR+Hdp+oIA7651UrJQoQfO13dxdifGKTNN59YXHnWubwTUyej2db7UR O8No4vxWccyB+TgCL5Ev8VVJU34hNpkXSEwrDB0Wr9abKT5jFsvfCQlBrOxFseZubVh7ij V+t6ENI/gMZf7o49wBWS5/22bCDdKai9lSOHTEuDObEzsX0USz4ytTttZCrIeg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1684278411; a=rsa-sha256; cv=none; b=d3xPPQSgBJ2O1Z9IjTdQpVSVhhtjP+79O2HL20eLScwtQyjQs8QDj/b4/g1FBIzAqGy+Ym MQDmcjVYMz1S43U1rWE5rP1P6xw+4Kgx4Rrlp4EfkqcHwUCPUs41NR55JmlfFB6rxwmnYi teESA7Lqhe5OLpmdFsLSQnWaW+tX8Gvut63GFccPf4Qw1KWigZVSzCNqZJkgGXusKeyzBt wEy4WJIYiyhaY29rbipn0xfk/a2NWu8j75/b714QvhzXGtXZ3I01DqXGo98rPHxAOAhzfh XhQ2/Lje80G51dzAPYisEZcRz0C1yD5v0TWloJjb7a0QLuunAEt7qW5q0NvS1w== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4QLX1R19gxzgPm for ; Tue, 16 May 2023 23:06:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 34GN6pj5058183 for ; Tue, 16 May 2023 23:06:51 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 34GN6p35058182 for bugs@FreeBSD.org; Tue, 16 May 2023 23:06:51 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 271460] ctld ports become inaccessible due to concurrent service restarts Date: Tue, 16 May 2023 23:06:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: asomers@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter attachments.created Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D271460 Bug ID: 271460 Summary: ctld ports become inaccessible due to concurrent service restarts Product: Base System Version: CURRENT Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: asomers@FreeBSD.org Created attachment 242225 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D242225&action= =3Dedit Example ctl configuration file If two separate processes do "service ctld restart", then they can race. T= he result is ctl ports that are inaccessible (clients can't connect), and the ports don't get torn down after ctld exits. Attempting to start ctld again fails to fix the stuck ports (though new ports can be added). The only rem= edy is to restart. Steps to reproduce =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1) Create about 32 zvols (i've also observed this bug with file-backed LUNs) 2) Configure /etc/ctl.conf as shown in the attached file 3) Run the following in two separate terminals: for ((i=3D0; i<10000; i=3D$i+1)); do service ctld onerestart|| break; done After some time, usually < 1 second, one terminal will fail with an error l= ike this: ctld: LUN modification error: LUN 31 is not managed by the block backend ctld: failed to modify lun "disk31", CTL lun 31 ctld: CTL_LUN_MAP ioctl failed: Device not configured ctld: failed to apply configuration; exiting /etc/rc.d/ctld: WARNING: failed to start ctld Then, kill the loop in the other terminal. Then ensure that no ctld proces= s is running, and do "ctladm portlist". All 32 ports will be shown. Attempting= to start ctld one more time will result in an error like this: ctld: error returned from port creation request: target "iqn.2018-10.myhost:disk0" for portal group tag 257 already exists ctld: failed to update port pg0-iqn.2018-10.myhost:disk0 --=20 You are receiving this mail because: You are the assignee for the bug.=