From nobody Thu Jun 15 00:00:16 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QhMqj4nRWz4dLC9; Thu, 15 Jun 2023 00:00:17 +0000 (UTC) (envelope-from salvadore@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QhMqj0XYWz3Fr8; Thu, 15 Jun 2023 00:00:17 +0000 (UTC) (envelope-from salvadore@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1686787217; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc; bh=BZDboLs55RS+2CA6Wp814zfKjcwV1ra3lerb9u3JLZI=; b=in8Nv/atlpnQW76D4Ajlwx1ZvSeqLX/cM3UdwyCggTGZfCoCiFtSvA8k14d4eYW7h/QYbK MCNxEP3TQbkedV8sa1xuQpj2WFAQQgl3e7w7bpEHYahMBIkTxnCsq17Um7dTniH3daoJ/e UDI015peQz8joo/N8gRrRF7Zjw4d9JBFMPQqNa6NRig05f3brJhhTWagLIQ7B5E0xmo96X kt9J1Rh0gKz7tvFw2iv59hrQUmehue2fYS8lQUSDQ29tc7LvSFN2I+qquVUSR0aJOFidIx hTrRh8V9Td+3a/wgCqE7yEC2qPdl7PjsHYcfN5KF67sjnb06c3bIAK+sjPy5gg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1686787217; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc; bh=BZDboLs55RS+2CA6Wp814zfKjcwV1ra3lerb9u3JLZI=; b=HZjNJf8R7WDZC/HAD30gOUwF8N/U+Z3LIxcxodToU5hSeZQvnwnVWfz6P00rwfmZ82vIrf MjFhIxXFBjit97/grpRwqDaiDjqMWFZM7nm2hwSZ+L41BXFDn4QZDD5yw3haGu8Bksz9pH 3Ai/mDPPsdbFoz9DpdjtAKIlxnHq3vIn3VicsN6mIzLlkHa650d35dtCO0CuUIe9Nxzvak ijzt8KtaxbcPLlTvKO1NuyoXHGTTh5+G64E2MdaQ3vxQaYBap6MqrKfQBrk44/ST85xrfa DHYSeATu5SJpfx9xT2CtYnoyf3/H6zLLTJYJNKPVykArOsc1O6ELdORNqrhIwg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1686787217; a=rsa-sha256; cv=none; b=UUjN5V3ECKJOjs5nmmqjLjJlsf26X67vF5/VXCVyEkHG5qhLWKLjbM5J8wxhizkoDhJewU lc8LBFqotR+Gnboy7Q/T7PBGrdBy65TZRRRxhL1Kfpay9mdbXRMX3OdwU70L5rXYoft86l pqqXEYAZ1JVz12iuD7q/LAka2UkQubzY/q8HeJiZXl2zVmNVEbsLQX+SkenQjZ3F9PunvY 4ssuEauLH7nkiXsVsh4vVeqTJ6I4eK5a0mnNDeh7btV0rcjPr7E9M7VpdSJE2e1lS2+m8N KA36VWqscvM1TOkW9Y4o50qLXPMSEtJn1f2g2O/ohvzcKWfQ00LeZhyuo28bdA== Received: by freefall.freebsd.org (Postfix, from userid 1472) id D1B731477B; Thu, 15 Jun 2023 00:00:16 +0000 (UTC) To: freebsd-status-calls@FreeBSD.org Subject: [2 WEEKS LEFT REMINDER] Call for 2023Q2 status reports Cc: freebsd-current@FreeBSD.org,freebsd-hackers@FreeBSD.org,devsummit@FreeBSD.org,info@bsdcan.org,soc-students@FreeBSD.org,soc-mentors@FreeBSD.org Message-Id: <20230615000016.D1B731477B@freefall.freebsd.org> Date: Thu, 15 Jun 2023 00:00:16 +0000 (UTC) From: Lorenzo Salvadore X-ThisMailContainsUnwantedMimeParts: N List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Dear FreeBSD Community, The deadline for the next FreeBSD Status Report update is June, 30th 2023 for work done since the last round of quarterly reports: April 2023 - June 2023. I would like to remind you that reports are published on a quarterly basis and are usually collected during the last month of each quarter, You are also welcome to submit them even earlier if you want, and the earlier you submit them, the more time we have for reviewing. Status report submissions do not need to be very long. They may be about anything happening in the FreeBSD project and community, and they provide a great way to inform FreeBSD users and developers about work that is underway or has been completed. Report submissions are not limited to committers; anyone doing anything interesting and FreeBSD related can -- and should -- write one! The following methods are available to submit your reports: * submit a review on Phabricator and add the group "status" to the reviewers list. You should put your reports in the directory doc/website/content/en/status/report-2023-04-2023-06/ (create it if it is missing); * submit a pull request at . You should put your reports in the directory doc/website/content/en/status/report-2023-04-2023-06/ (create it if it is missing); * send an email to status-submissions@FreeBSD.org including your report. An AsciiDoc template is available at . We look forward to seeing your 2023Q2 reports! Thanks, Lorenzo Salvadore (on behalf of status@) From nobody Mon Jun 19 04:23:06 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QkxTT6nCQz4fkjy for ; Mon, 19 Jun 2023 04:23:25 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic311-25.consmr.mail.gq1.yahoo.com (sonic311-25.consmr.mail.gq1.yahoo.com [98.137.65.206]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4QkxTS3g2Tz3HnB for ; Mon, 19 Jun 2023 04:23:24 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="lsjuOnY/"; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.206 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1687148602; bh=xVs5mhpT9pzZuJIfD4nZZOy3UBb5tUfWZwN5tTWgCqc=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=lsjuOnY/PdZEjHkSFH93VnYv7ausJHq3cNGwOd4CfXH/0ZW2kpj3Hs8ziIGZb5+lySvPOfyppJpfNT2zRNFUYohmn7rKfQK4w78meXQlCIj9g6SvW5FToiDXgGRrXZpJUWgP3FW28yJI3agmbdKVKM/PgCssaKwgK13V1qcK6Ux3t0GQI20oHjj0KI25nyXXqMWYI5fQrSTqwExHy0OzAQdckTp087WWynGdlRN/KIuqqXaQ508HOlT4lMoHCz0YH6/MsmYnRsullTPsL8VxiFjnah3n772ZBm2eMGA0tmZ5VVs1EXvHuMlBiE7nHVDbak44bh5UJL+gYYimURXBgQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1687148602; bh=5h7HNRSoqwK7SozHEcZLhmkNEf2ISYEJcmxm9qiFr/j=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=rzZQuQ90AP5AXr/FxWHaPNsjXDrojEDybSbHcQVoBXSopVqqJBHUIb5BOZi7JR2rGgy3ZW8So1QWvMlQNXq2nWn68nT9gvQlLLPj5iJP0ZKTJuL52aaA864yyuJTaEp6ZU2/PIfSla5otN2MUlIB1cO3UeibZLkxZUr+24t+l3nb527Nz1he93JZk7kOr+vpsMtTNVIRm9wrwUhPPH8blYL9xyETt1Ak7QFaZG4nsnnCk9bMVBMyVrZ/9J1rEPAcTC37Tw02Kg9UZi+VC87ZGbN3wCQqAnazXyVqcdMBZL5RTLlICRtNEj/FZkRo9Hs1G5LyQ+V12qEjuEi1g7WLdg== X-YMail-OSG: 5C1O0soVM1mwRyPG6FEk0k6pResw.Lp9JiUSPbedv7X_hafdV15G6bWDw8k9vOp k6Rge4qwUnHNHujNJzxehCOvkelnTsvH1fxThiJusSNYcTwVCHwtUUqWS3TsOinLQI7iCdq.BRS4 gxlAlTYUF92gWb9X10AlPq.iFMm3vW9snITqohwTxLsepQ.gQLVjZqqlPbSpH2mkef97pnERDYsp Q7OsIuFjT8wRKV5hRRRjvsw7.xQ6Am5qW9R4TVSgcktILFWjgGIPnPF.noM.OxYY8R5thoaMOeSL WfHaAtNQYlffJNLjPzu.cQ2G5V4xaF8R1AnHulrOmVGL2nl_lDy5Gk6u.yK7w2ZDbjguIOObkReY 5q1q9MXQSc6byXYp62wotQ8y.xk1Nf6H9VKsH0AtQ1yNEO6SYLnQl5NuSmSPiK1mW3Y101gdVgPY 3mg0XjuoGyAhkEVi7p7YnaQcfcw9tH0A0pV0MxZsnC5VRMGU.Sjlapxqrzuz2cB.XJbdu7nfbeOo dlmq700Rl.THUwQ5pMPFnytlleOFPfqy549Z7dIg2mfHuyj1Tq4hvzCl5zg2v7M3hV9RbtlGw3R4 pHaVSR3jbyR9NoRgpJe2k763n7hSck7iuyVZIxAFOy5A.8J8DACgmLwCGWE049kDwI5Yjot8GBst aJnsDfCWUGYxoBgI6xiPj5t9UlRQNTNJYOW6JsUKrT9i2AnknyCIEN54rHMzIxOz6gRxrnuQINpR 7Bh.fCxDBrG88rCcFu_yFfOITpv.2LqXSlxT3efvuAN8mghtPE.FMA5o.ev5mBuTbDeQXtmrW92f tNBvew5GclfsqT1aMJ7Y2A4tf5qSP9ZHJ0jLqYOaNZM4uklSRo4DEPV8LdBdjli3ZvuqBpDESNi8 ixjI2p.kchWffv1RBk8mQY4dEj21KZU8Y3jG_.xLj8m_SybRVTjG1Wx.m0qr1kEMbTl5RP_8UnN2 gvFkHq.VGPnbPcsnS9iZAlpAaiv.BFGpqTRzOLjch3P.ukjzGvL2keT1vwjNeA2zcskUYzdHz9tX CinOJEHWsxuMkbJJGLNZ79eIEnHpwpWV.3eJiiKFT.l0UjpH3nvaqwMr9W3cQyBdv5TzODgzcToh OBCyVSBrtISnoCtKrJgImaQ.J6P74E.oiI7JFtAM86YU7NTBcfHP9U31VXlP_YwMDbjC4w6NBEcN EsaAE93DKWxjqBFlO_Zy2.Z5uDsMYlxAdiHmyirZhjFZxBYs53UNs1IVubgj1iU5o9WLhFiLkrem H1HeKicmjIW8z._p7XAnFcUJrhBqwPPvrXBOBZmLhBJ7TKVGS4xEWIE7.N0H14S_zJc6kIMJBx0z GU0Jve_lOOHGcV2FrNDkrYCFDsATpUzHB3tGl4pCoPBGtwnAnPWuHNg32HAOA1EKsm9nnsJTm10x D1GNHrVQnkIAeatVtuqPZLJq2MNjZ6dTRwQrtw5mAEBGamMJEKaAOo9_59kJyHf0IiGh7xDQ7zke bUqeTpdbkka_7io5efW_bfMI4REX89shDgOHM.NKzEPIJEVEMzUz6mJQo3eeAH6b3a4tsZ3hFfXL pzPomXvwgDtQKOp_eZFIcVeLvOXUKTVn9L5wrWLd2BTgacS3YgUcpW7s2HiKnfoSyt7UxQeIpJIc BA2TLd6BONulyD2465W2wMhzCUOv8KwT1McOHVI1MVyqaCvIoOEBwKKrDg81iB_ZYCY1q5D7LgR2 ZGutX5KBbSrkLm8uwpqJcPHNqNPdchX_mRFS_9cNXyNvWhHawaossULCUdNK52j9nswmgEwU.xVo PTRD3vOWdHaJRBlObH8Do_u2YMEUv4ElYIWnj7_A3dUoGpB6XVvWNsHw8zIqZwgdZNtUQvZ_YN_p uhZhf_G7S8Skh10oHfz6tn_Wp5tXVSzO82hmChvSqpRJEx4IALq9D7JyU61hNycssSzDj6EM32tb bmskyF_J6pJOZJWMWN10Wy_ZOQpVmO4exggf6MPHe9cUdT0TlK7W9mvWo88ZEhfqoawPSqtGoIm9 I0oIfUeSTLKWltqw9Y5cOa00kWb9a_6CHG2YaKxRhNx1EqRRNYFOg6Rq9DGz_WfMjR9q9KqL8upQ BBRdWP2OhubiHxUa4s8LzBxunsI__8_6W13dgzHLcXtu8liImzyLcPj8GE51c8jejA0zlLSXJFJP gb3.lFKlgQNlDb4PMyUwQCLnXF7qkFt0tLqAtwrmKzDpnkmKeke9twJxusCEAv5.LDvEvxx4f_9o iW80rEIv.VyxXX74YNtHNaL4OUCL4uGbu4bzVjcgL9YC_2X84N6QhUCZwiA-- X-Sonic-MF: X-Sonic-ID: fb8ed35b-23f4-449e-a8e7-94f12a8c2f31 Received: from sonic.gate.mail.ne1.yahoo.com by sonic311.consmr.mail.gq1.yahoo.com with HTTP; Mon, 19 Jun 2023 04:23:22 +0000 Received: by hermes--production-gq1-6db989bfb-hz24p (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 2aea78817ac687626930465d0529a87c; Mon, 19 Jun 2023 04:23:17 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.600.7\)) Subject: top vs. array sorted_state's out of bounds indexing (and related issues) Message-Id: Date: Sun, 18 Jun 2023 21:23:06 -0700 To: FreeBSD Hackers X-Mailer: Apple Mail (2.3731.600.7) References: X-Spamd-Result: default: False [-1.56 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-0.99)[-0.989]; NEURAL_SPAM_MEDIUM(0.93)[0.930]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.206:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.206:from] X-Rspamd-Queue-Id: 4QkxTS3g2Tz3HnB X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N usr.bin/top/machine.c (from main) has: /* . . . The process states are ordered as follows (from least * to most important): WAIT, zombie, sleep, stop, start, run. The * array declaration below maps a process state index into a number * that reflects this ordering. */ static int sorted_state[] = { 0, /* not used */ 3, /* sleep */ 1, /* ABANDONED (WAIT) */ 6, /* run */ 5, /* start */ 2, /* zombie */ 4 /* stop */ }; Note the elements are 0..6, so 7 elements. It is accessed via code like: sorted_state[(unsigned char)(b)->ki_stat] But ->ki_stat has the values listed in sys/sys/proc.h : /* * These were process status values (p_stat), now they are only used in * legacy conversion code. */ #define SIDL 1 /* Process being created by fork. */ #define SRUN 2 /* Currently runnable. */ #define SSLEEP 3 /* Sleeping on an address. */ #define SSTOP 4 /* Process debugging or suspension. */ #define SZOMB 5 /* Awaiting collection by parent. */ #define SWAIT 6 /* Waiting for interrupt. */ #define SLOCK 7 /* Blocked on a lock. */ This leads to element indexes (including the "unused" 0) of 0..7 so spanning 8 elements. Thus there is a out of bounds error involved when ->ki_stat == SLOCK . (I later show the ki_stat assignments.) There is also the issue that: SIDL is misidentified as: sleep SRUN is misidentified as: ABANDONED (WAIT) SSLEEP is misidentified as: run SZOMB is misidentified as: start SWAIT is misidentified as: zombie SLOCK is out of bounds (as indicated above). So the sort order in top is not as documented. For reference, from sys/kern/kern_proc.c : if (p->p_state == PRS_NORMAL) { /* approximate. */ if (TD_ON_RUNQ(td) || TD_CAN_RUN(td) || TD_IS_RUNNING(td)) { kp->ki_stat = SRUN; } else if (P_SHOULDSTOP(p)) { kp->ki_stat = SSTOP; } else if (TD_IS_SLEEPING(td)) { kp->ki_stat = SSLEEP; } else if (TD_ON_LOCK(td)) { kp->ki_stat = SLOCK; } else { kp->ki_stat = SWAIT; } } else if (p->p_state == PRS_ZOMBIE) { kp->ki_stat = SZOMB; } else { kp->ki_stat = SIDL; } === Mark Millard marklmi at yahoo.com From nobody Mon Jun 19 16:05:11 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QlF3N2mK2z4fs5F for ; Mon, 19 Jun 2023 16:05:20 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4QlF3M5yrRz3GYr for ; Mon, 19 Jun 2023 16:05:19 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.17.1/8.17.1) with ESMTPS id 35JG5BOF018492 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 19 Jun 2023 19:05:14 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 35JG5BOF018492 Received: (from kostik@localhost) by tom.home (8.17.1/8.17.1/Submit) id 35JG5B87018488; Mon, 19 Jun 2023 19:05:11 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 19 Jun 2023 19:05:11 +0300 From: Konstantin Belousov To: Mark Millard Cc: FreeBSD Hackers Subject: Re: top vs. array sorted_state's out of bounds indexing (and related issues) Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on tom.home X-Rspamd-Queue-Id: 4QlF3M5yrRz3GYr X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Sun, Jun 18, 2023 at 09:23:06PM -0700, Mark Millard wrote: > usr.bin/top/machine.c (from main) has: > > /* > . . . The process states are ordered as follows (from least > * to most important): WAIT, zombie, sleep, stop, start, run. The > * array declaration below maps a process state index into a number > * that reflects this ordering. > */ > > static int sorted_state[] = { > 0, /* not used */ > 3, /* sleep */ > 1, /* ABANDONED (WAIT) */ > 6, /* run */ > 5, /* start */ > 2, /* zombie */ > 4 /* stop */ > }; > > Note the elements are 0..6, so 7 elements. > > It is accessed via code like: > > sorted_state[(unsigned char)(b)->ki_stat] > > But ->ki_stat has the values listed in > sys/sys/proc.h : > > /* > * These were process status values (p_stat), now they are only used in > * legacy conversion code. > */ > #define SIDL 1 /* Process being created by fork. */ > #define SRUN 2 /* Currently runnable. */ > #define SSLEEP 3 /* Sleeping on an address. */ > #define SSTOP 4 /* Process debugging or suspension. */ > #define SZOMB 5 /* Awaiting collection by parent. */ > #define SWAIT 6 /* Waiting for interrupt. */ > #define SLOCK 7 /* Blocked on a lock. */ > > This leads to element indexes (including the > "unused" 0) of 0..7 so spanning 8 elements. > > Thus there is a out of bounds error involved when > ->ki_stat == SLOCK . > > (I later show the ki_stat assignments.) > > There is also the issue that: > > SIDL is misidentified as: sleep > SRUN is misidentified as: ABANDONED (WAIT) > SSLEEP is misidentified as: run > SZOMB is misidentified as: start > SWAIT is misidentified as: zombie > SLOCK is out of bounds (as indicated above). > > So the sort order in top is not as documented. > > > For reference, from sys/kern/kern_proc.c : > > if (p->p_state == PRS_NORMAL) { /* approximate. */ > if (TD_ON_RUNQ(td) || > TD_CAN_RUN(td) || > TD_IS_RUNNING(td)) { > kp->ki_stat = SRUN; > } else if (P_SHOULDSTOP(p)) { > kp->ki_stat = SSTOP; > } else if (TD_IS_SLEEPING(td)) { > kp->ki_stat = SSLEEP; > } else if (TD_ON_LOCK(td)) { > kp->ki_stat = SLOCK; > } else { > kp->ki_stat = SWAIT; > } > } else if (p->p_state == PRS_ZOMBIE) { > kp->ki_stat = SZOMB; > } else { > kp->ki_stat = SIDL; > } https://reviews.freebsd.org/D40607 From nobody Mon Jun 19 18:09:41 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QlHq416tjz4fNwH for ; Mon, 19 Jun 2023 18:09:52 +0000 (UTC) (envelope-from jo@bruelltuete.com) Received: from email.jo-t.de (seppel.jo-t.de [45.132.244.126]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4QlHq30wZwz3ndS for ; Mon, 19 Jun 2023 18:09:50 +0000 (UTC) (envelope-from jo@bruelltuete.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bruelltuete.com header.s=bruelltuete18a header.b=K33WW3DF; spf=pass (mx1.freebsd.org: domain of jo@bruelltuete.com designates 45.132.244.126 as permitted sender) smtp.mailfrom=jo@bruelltuete.com; dmarc=pass (policy=none) header.from=bruelltuete.com DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=bruelltuete.com; s=bruelltuete18a; t=1687198184; bh=xcXSgemwVYynXg+7ecTOgacVFaGsqGar1Ml26IFlAV0=; h=Message-ID:Date:MIME-Version:From:Subject:To:From; b=K33WW3DFHBiILuwjOw9pCk/OWbzg/Idmf9+odBetINkYrYdq/UzIx1wgalpD6WGVI v9mJAMQ4P5vx0f3g+chUsAC99VV/hTnFYxJEeE2/3hpYk0uvr/3Roc/iz0M0NYdWs6 6yLN7+GNYm1g/1kR/WKrq3if7VErDWDwix1ZTmuOx06YKgnEIvQwjwXf2/L2cHg2rX iHRsrv8iCgnxjVXiA2a0D+k+XaUcsl5pF6IP00W0GWdZ7zntpKmuTB75dHGMXgHJlD ZzwTJcDf7w5uGfTIGGykl1cjQKkxgADaKjXho4WP5f/WXdZ9q1Ro1sgUtph5K3B4nQ ZGzfRcidOU3vg== Message-ID: Date: Mon, 19 Jun 2023 19:09:41 +0100 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 From: Johannes Totz Subject: tpm for AMD Ryzen To: FreeBSD Hackers Content-Language: en-GB Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-3.88 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.997]; NEURAL_HAM_SHORT(-0.96)[-0.963]; NEURAL_HAM_LONG(-0.92)[-0.919]; DMARC_POLICY_ALLOW(-0.50)[bruelltuete.com,none]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[bruelltuete.com:s=bruelltuete18a]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-hackers@FreeBSD.org]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; TO_DN_ALL(0.00)[]; ASN(0.00)[asn:197540, ipnet:45.132.244.0/22, country:DE]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[bruelltuete.com:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4QlHq30wZwz3ndS X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Hi everyone, I'm trying to get the (f)TPM that comes with AMD's Zen2 to work. BIOS config screen says it's doing CRB mode. So I start poking in tpm_crb.c https://github.com/freebsd/freebsd-src/blob/main/sys/dev/tpm/tpm_crb.c Out of the box, it does not attach. The TPM2 ACPI table says its start method is 2 (aka ACPI). That's easy to hack around, just mess with the if-condition at https://github.com/freebsd/freebsd-src/blob/main/sys/dev/tpm/tpm_crb.c#L115 With that adjusted, tpmcrb now probes successfully but does not attach. As far as I can tell the register values it tries to read from the ACPI-provided memory window are just bogus. That makes me suspect that the BIOS has misconfigured it. The TPM2 table has a different address than what's reported at runtime. The table says 0xfd210510 is the (physical) address, but acpi says it's 0xbd13f000. Fiddling about with hint.tpmcrb.0.maddr and friends does not yield anything fruitful: anything I try to override with hints is just ignored. Hacking in a bus_set_resource(dev, SYS_RES_MEMORY, ... 0xfd210510 ...); ends up giving me that override but still no dice re actual tpm functionality. Has anyone gotten the tpm to work on (consumer) Ryzen? thanks, Johannes From nobody Mon Jun 19 18:57:43 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QlJtN0G6Nz4fRFJ for ; Mon, 19 Jun 2023 18:57:48 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QlJtM5TvTz4KbW for ; Mon, 19 Jun 2023 18:57:47 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1b53e1cd0ffso13321285ad.0 for ; Mon, 19 Jun 2023 11:57:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687201066; x=1689793066; h=to:references:message-id:cc:date:in-reply-to:from:subject :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=m7xd+MOkvKAlGKEc6m4lyBFxp75vopMR4FJd9wCpzZU=; b=e0/d97EROgbFufksTwoy6WfTAJCQSlNTHaziGPFcg0FqDHHN5ZHiHhlFxoFhwp8uMO RR30d1p2ngS2ZsYPUAcMLqCbQgO+q7t1rLi1lVOoSOfANQdhk+dptqUfdjDZw4PMw7fk 4rlh3ahD6p3P3QG1Eshqof/e3nV5ySyZyj7q5seZyQQ4yV7RjouIK8jNwgvKe/Qe7urs L8xtyX1cBdZqp/iZ4wEcyzFOrZfNNBV5kBBIsExq4DZS/CsEm6ulRsSr8tgnHTPkN3/p 35+dBTorbRFESilh8FSBBsU2FFaQDPIytZ4HYOkja4GbE77EDrAONtoDVS1pIL9ypZ+E ylJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687201066; x=1689793066; h=to:references:message-id:cc:date:in-reply-to:from:subject :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=m7xd+MOkvKAlGKEc6m4lyBFxp75vopMR4FJd9wCpzZU=; b=YtVpw2XPixddAAkkwBFBef5xZOLNk4vQKUxARkKoAYxIQGWUmiLDk5AA/6tjvKTasl igop1yIQ2aR8wTDn9uyjCDFmU6zO9150BFdenCX02OBfuIP/m+8ToaNUNPRmZAjtxEOU 0q8gUSFDF9AKGBoZTiB0S7whQWfhfItwuMGBJOY2Id/uaGLKsvGmpCOteiCWOwVsSt83 o3liE5FpAgOBnvl5yxOLvGbGeay6daG4rjC3WbZyJKgNK2sQWhahDjlLRAdTB+LLUDb9 hJ8SqCowsdJlS4gmNVSE2cDEX/1eseu1xVs18Jm88JEWLFw3aOTLI9eGtiM+raXnrkf/ juRA== X-Gm-Message-State: AC+VfDzBRQSzheDiEe34CvO66JTGqry78CJsoa10RshbEvOHjD0IL/PJ J9QeJ73FQWJzlA/sHl1EFh0DINsdXVg= X-Google-Smtp-Source: ACHHUZ5BJ1XcmPcwRGaW28sw701PNy8hW2iLb+H3IlCe55TG+nDB8NHMq33eJ18BgabzQGbpkiuv8w== X-Received: by 2002:a17:902:d34b:b0:1b3:f8db:6f0e with SMTP id l11-20020a170902d34b00b001b3f8db6f0emr7602962plk.43.1687201065576; Mon, 19 Jun 2023 11:57:45 -0700 (PDT) Received: from smtpclient.apple (c-73-19-52-228.hsd1.wa.comcast.net. [73.19.52.228]) by smtp.gmail.com with ESMTPSA id r1-20020a1709028bc100b001a1b66af22fsm190899plo.62.2023.06.19.11.57.44 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Jun 2023 11:57:44 -0700 (PDT) Content-Type: multipart/signed; boundary="Apple-Mail=_F8AA46D4-6834-4B10-A42D-A3A12094CB0C"; protocol="application/pgp-signature"; micalg=pgp-sha256 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.3\)) Subject: Re: tpm for AMD Ryzen From: Enji Cooper In-Reply-To: Date: Mon, 19 Jun 2023 11:57:43 -0700 Cc: FreeBSD Hackers Message-Id: <83976649-D24E-475C-9BA2-9922466062A9@gmail.com> References: To: Johannes Totz X-Mailer: Apple Mail (2.3696.120.41.1.3) X-Rspamd-Queue-Id: 4QlJtM5TvTz4KbW X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_F8AA46D4-6834-4B10-A42D-A3A12094CB0C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jun 19, 2023, at 11:09 AM, Johannes Totz = wrote: >=20 > Hi everyone, >=20 > I'm trying to get the (f)TPM that comes with AMD's Zen2 to work. > BIOS config screen says it's doing CRB mode. >=20 > So I start poking in tpm_crb.c = https://github.com/freebsd/freebsd-src/blob/main/sys/dev/tpm/tpm_crb.c >=20 > Out of the box, it does not attach. The TPM2 ACPI table says its start = method is 2 (aka ACPI). That's easy to hack around, just mess with the = if-condition at = https://github.com/freebsd/freebsd-src/blob/main/sys/dev/tpm/tpm_crb.c#L11= 5 >=20 > With that adjusted, tpmcrb now probes successfully but does not = attach. >=20 > As far as I can tell the register values it tries to read from the = ACPI-provided memory window are just bogus. >=20 > That makes me suspect that the BIOS has misconfigured it. The TPM2 = table has a different address than what's reported at runtime. > The table says 0xfd210510 is the (physical) address, but acpi says = it's 0xbd13f000. >=20 > Fiddling about with hint.tpmcrb.0.maddr and friends does not yield = anything fruitful: anything I try to override with hints is just = ignored. > Hacking in a > bus_set_resource(dev, SYS_RES_MEMORY, ... 0xfd210510 ...); > ends up giving me that override but still no dice re actual tpm = functionality. >=20 > Has anyone gotten the tpm to work on (consumer) Ryzen? Hi Johannes, I just built a Ryzen machine too with an ASUS Motherboard. Could = you please post the hack that you did to the if-else statement up on = gist so I can take a look at it? Also, if you can post "boot -v=E2=80=9D and =E2=80=9Cpciconf = -lv=E2=80=9D output to separate gists, that would be super helpful :). Cheers! -Enji --Apple-Mail=_F8AA46D4-6834-4B10-A42D-A3A12094CB0C Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEtvtxN6kOllEF3nmX5JFNMZeDGN4FAmSQpScACgkQ5JFNMZeD GN5h7hAApIkaGylnZ4rfBBe7LTD6kPPAz3TENin2Sk64tfR/KQMO9i4h2LhwvS2q 4yNobuABvSrL3tLiTRd21D+n29a9eztY1tfEZSn+y1GjCmOEXKqbcilBP1jkBJk1 RLRxTKR0BLx9P3M/hqcW3uQZt8r3v9mzuM8/i/IeQ3GE+ZZ2yY7CZyt7/0QpHQpE f7F4dYyEFW87z1lIxzUEEwgHnOyheboh+WAtyrNV44sKjzyjeQblMPIftjMSRV/u 04cEd4gueP/rCXCzg4lhA2BaUbW0qAIiwm6OCuOewOUrmiFmo7WbQesXI8auuZ5g rSww6Z6epfZr/GJm+kfGz3pyiXJvHQlO7LnS6uN0Dd6CCIrO+TnKh8ZAbGhbM9Rb Mfvr1xWE/Y1O1dvoxFh988gPBczNVSqzPFeaWDYbi7NhnasrPyE1Gkwb19kIXipe /KrarykzOLClG/fQ8kMmexAnrxFq7QqSALdl+l45e+TqoDu4DeJy+HF6BXKLsYfp 0oHhssyBtFsHLF/5ZCI2K26cKdWmq6VyrlzT+VzYbFAOjQPgJwxkKPm8aCA5pm/o 09Lmwp09dK0SsdHECMe1ZnuPgV6uY0+v+UaoJGVryJO5IRCcBfJDtFsQEtRR6tYB ADfhZSlUnHsU6NQk4JJkX0yuAPPQ0/TebSAXdtDXMkuRy2p1dzc= =5Vdc -----END PGP SIGNATURE----- --Apple-Mail=_F8AA46D4-6834-4B10-A42D-A3A12094CB0C-- From nobody Wed Jun 21 01:44:09 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Qm5rr0V3Nz4g57n; Wed, 21 Jun 2023 01:44:12 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Qm5rq6prJz3qkd; Wed, 21 Jun 2023 01:44:11 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1687311852; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=9qJ5bvOUUnATONoJL5Cguk7qlHJub4vGcVgwSWFU+Oc=; b=djPaGly741kmV3WYOiYqva32/wtnCilxpnZtLcc/2iyg5PImQIJf14xkiIHRhflF/PGnqX J+LFfNsbWJ+iXXE+GjRyWwE2Ce4pNDMJ3b/J2sRgeHbzkYKTqPzj0dGEVCGKTKmFk8t9nE X6KTn6Hzp6hNbY1tQa7tguMlaEa3p1bPyEfoI5tww+AIngoa9xouNCYx0UtGFTfvi6tB0Z 166gGV0wrQwGhzRfQTj+0W7Y1urRNyquCBOz7eIwCKTn6qhNiyApSF3YbkgmsTT9yvYobD 1ht5MkmixC0Zfmxm103GoFnoLRJ2jWFjpDAzsP7GErDMkmUDNnCr7W8t3zoU8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1687311852; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=9qJ5bvOUUnATONoJL5Cguk7qlHJub4vGcVgwSWFU+Oc=; b=GQG+pWPo45wbWmjq/mJEdYB9FMERQjm9nklVDRdAQOyeVMtFXNksCe1SZzglmKX9JX8Bb6 2ggOj9IqWA2TMPtQdNK718Zv2ORctVhcfCH5OhCQ1ImoFKFeO0I2p3OlEfifFNveNSf4jt H89NIL4j/mG0tKRGyFUy5acGLttbVSPVfDSDKtWOQuF+nwyi1HP+56sFoWp98jYQY7u6Fi 33kWFRyyO7bsIMINYyuNatP8kQ3KA/kc6L8uZUqDwqgZkBkCjZO511HJA/xokyhtidzeYq eVZGJ4FLBElttQ/5OmUSpIdoNiE/zNXO1t9hFPDLTxr9SJKwKkslq0pQ588GXw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1687311852; a=rsa-sha256; cv=none; b=RzLl4V2sHoqSO+FRmhgylINJJpVACAtXKD16QSJScp9OhD3b/auH8RWDc6XWGjwE4XKzaC 0HxT+RLzmRUYBrmbyZ0ODxgFT5a3Ti7yaayP+1xmPYKqZzyhRBP0xtaEA7HRUz2qZ2jH1f FZlgep0yM4z/G7XF8tcE0+u+mziMRALECl0JUNtsGUEgpPqndZthKL9GPlOuMxrYJysGqn KosQClJQNtAcxw8afySk9xAGphpX5+6Z1LPiJDLKWzOCya17VlUswUxTSQ4YEHIPx7/bsu CjNrhY0Y34oi6UIbpicDk+hkO4OpUzABsxcJ2P+lcXo15FkPy6ssjK+VRdgR0Q== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 7071EE041; Wed, 21 Jun 2023 01:44:11 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Wed, 21 Jun 2023 01:44:09 +0000 From: Glen Barber To: freebsd-arch@freebsd.org, freebsd-hackers@freebsd.org Cc: freebsd-current@freebsd.org, FreeBSD Release Engineering Team , FreeBSD Security Team Subject: Re: Delay in 14.0-RELEASE cycle and blocking items Message-ID: <20230621014409.GA85543@FreeBSD.org> References: <20230501181449.GJ1219@FreeBSD.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="fUYQa+Pmc3FrFX/N" Content-Disposition: inline In-Reply-To: <20230501181449.GJ1219@FreeBSD.org> X-ThisMailContainsUnwantedMimeParts: N --fUYQa+Pmc3FrFX/N Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, May 01, 2023 at 06:14:49PM +0000, Glen Barber wrote: [...] > A more up-to-date schedule for the 14.0 release will be published in the > near future, though nothing is yet set in stone. >=20 > Thank you for your patience, and for any help in getting us through > these outstanding items. >=20 Just a quick update on this: I am working on the updated draft schedule for 14.0-RELEASE, and will send several teams internally for comments. A new schedule should be expected to be available on the website and announced on these mailing lists within the week, give or take a few days. Thank you again for your patience and the need for the delay. And certainly not to be excluded - thank you to everyone who helped in ironing out the issues blocking 14.0. Glen On behalf of: re@ --fUYQa+Pmc3FrFX/N Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmSSVekACgkQAxRYpUeP 4pOW9g//cNgqj+/fvBv72xh86SN737e42B2ZSjzIidmpxtMn+YL6H7hamQS0wrEw P2cxADjC5Fg91jZ84xx+aHV1kL996v5inANEENnI6VrKJAKInqHDUro7S/32ehhT p6oojfjlnAZn6edyxkXdjTAgfOYXrT0L46ANA7j2PsKAi3axAmqZPePmwRMJ7kZ0 qPvvEOtYBe8Om9zeH1sHHGukNj5IKTJ+OZAXOuQ5acg+DrbduBIp7w5FXveQloic wRCJKpDx2rmAjPkbwJ5OHUzmwf0mI2mJlRlQH51ozl4yQzBoozH2LR0TXsw8hdui 4hDfHwoVs+RyU9LQVv7WgxAHYhLlzN9G9Ysjsul5FIjCaQaY6BW81CIbjDnWWGAg v7CCZ1KFMh01WX1THFNa4YjnmGryWhhcOM3GXyKv6IURFqowh9a4Zk1QOXMoogAP iDHAHRuUulH4pbht+vYVzO011tMsqpAkQSepOmKHGhPvhVIx7DaaZaR6jjrlFM3I 7kYJAIJLPIDIcQGxllBLUBMl5tJHtuQP2QDOQGlqTJ+NQWxhK59sPWBMbOyRXQ20 73LHH3Zvf4OAsq0X/Gx4OkDI8m/a7lYH1SpJkRQc357R+EQyW+8MS3p0Z9kgKeO+ 4E7M4iTEvaaf7Fl/ICxVjjX652YJGIPVdFVtUnMIdYsYKoiuk8A= =wMji -----END PGP SIGNATURE----- --fUYQa+Pmc3FrFX/N-- From nobody Fri Jun 23 19:00:36 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Qnmm33qLtz4gk6q for ; Fri, 23 Jun 2023 19:00:51 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-vk1-f175.google.com (mail-vk1-f175.google.com [209.85.221.175]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Qnmm242yGz48Bh for ; Fri, 23 Jun 2023 19:00:50 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.221.175 as permitted sender) smtp.mailfrom=asomers@gmail.com; dmarc=none Received: by mail-vk1-f175.google.com with SMTP id 71dfb90a1353d-45739737afcso316706e0c.2 for ; Fri, 23 Jun 2023 12:00:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687546849; x=1690138849; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=GVmq+Z7aIB5np7voqeLhBHujXZjh3gpAShSyL5Xxvm8=; b=Osf7MtO3CgItGlHslQrF+wl+6zLwizjKLfwP3i5F3QygwDH+cwPsi8SoqfNkmoX02c OSw7b5W0+HYjXgNrF4ZoO8W8GPBiMVHUb1a9BAEhH6gxhuaYGSckNelvYecrJy2Qy9ha 5BQLnCeKdrJgk2MUhmmIQ2rm59cwHk9n5B2cWKZygo3NNUooq+pqlPOjg9lcx8Yj/FBt ZefvZiBqR/OKWNP6bFzxDtbRnfREGviKKsMUg20wPztPvctHwH7J+OVVqsiouXrlqbs6 gN98AJCRQ7b5/o8+oTv6ge50+H7ie08G+72bz8WvvIwRklsHu1/LiqvmAH6hRcMELmPg rFXA== X-Gm-Message-State: AC+VfDwFpi1PLZCUQsAQk3UP318T8e16KjBG5Po0E458XgbNdoLw+qxf A58vl+v/9YlFIUZ2RxMRRmwQ8pD8twftKpsb1/vtOBojFyW4gw== X-Google-Smtp-Source: ACHHUZ6SbE52rX73+fAje8enjKNPHghpWiQDGTIbooX38dbRsPKmwnOhw9Ij6fG02DfjnGepOqApfa+BXfTn0hvUohk= X-Received: by 2002:a1f:e643:0:b0:474:3a0d:3dee with SMTP id d64-20020a1fe643000000b004743a0d3deemr6289197vkh.5.1687546848720; Fri, 23 Jun 2023 12:00:48 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 From: Alan Somers Date: Fri, 23 Jun 2023 12:00:36 -0700 Message-ID: Subject: Should close() release locks atomically? To: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [-1.79 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; NEURAL_HAM_SHORT(-0.99)[-0.990]; NEURAL_HAM_LONG(-0.81)[-0.805]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[209.85.221.175:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; FROM_HAS_DN(0.00)[]; TO_DOM_EQ_FROM_DOM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.221.175:from]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; FREEFALL_USER(0.00)[asomers]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; ARC_NA(0.00)[]; R_DKIM_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4Qnmm242yGz48Bh X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N The close() syscall automatically releases locks. Should it do so atomically or is a delay permitted? I can't find anything in our man pages or the open group specification that says. The distinction matters when using O_NONBLOCK. For example: fd = open(..., O_DIRECT | O_EXLOCK | O_NONBLOCK); //succeeds // do some I/O close(fd); fd = open(..., O_DIRECT | O_EXLOCK | O_NONBLOCK); //fails with EAGAIN! I see this error frequently on a heavily loaded system. It isn't a typical thread race though; ktrace shows that only one thread tries to open the file in question. From the ktrace, I can see that the final open() comes immediately after the close(), with no intervening syscalls from that thread. It seems that close() doesn't release the lock right away. I wouldn't notice if I weren't using O_NONBLOCK. Should this be considered a bug? If so I could try to come up with a minimal test case. But it's somewhat academic, since I plan to refactor the code in a way that will eliminate the duplicate open(). -Alan From nobody Fri Jun 23 20:02:59 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Qnp7w2YD1z4gcsK for ; Fri, 23 Jun 2023 20:03:08 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Qnp7v5yC3z3NMJ; Fri, 23 Jun 2023 20:03:07 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.17.1/8.17.1) with ESMTPS id 35NK2xFC011037 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 23 Jun 2023 23:03:02 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 35NK2xFC011037 Received: (from kostik@localhost) by tom.home (8.17.1/8.17.1/Submit) id 35NK2xHa011036; Fri, 23 Jun 2023 23:02:59 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 23 Jun 2023 23:02:59 +0300 From: Konstantin Belousov To: Alan Somers Cc: FreeBSD Hackers Subject: Re: Should close() release locks atomically? Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on tom.home X-Rspamd-Queue-Id: 4Qnp7v5yC3z3NMJ X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Fri, Jun 23, 2023 at 12:00:36PM -0700, Alan Somers wrote: > The close() syscall automatically releases locks. Should it do so > atomically or is a delay permitted? I can't find anything in our man > pages or the open group specification that says. > > The distinction matters when using O_NONBLOCK. For example: > > fd = open(..., O_DIRECT | O_EXLOCK | O_NONBLOCK); //succeeds > // do some I/O > close(fd); > fd = open(..., O_DIRECT | O_EXLOCK | O_NONBLOCK); //fails with EAGAIN! > > I see this error frequently on a heavily loaded system. It isn't a > typical thread race though; ktrace shows that only one thread tries to > open the file in question. From the ktrace, I can see that the final > open() comes immediately after the close(), with no intervening > syscalls from that thread. It seems that close() doesn't release the > lock right away. I wouldn't notice if I weren't using O_NONBLOCK. > > Should this be considered a bug? If so I could try to come up with a > minimal test case. But it's somewhat academic, since I plan to > refactor the code in a way that will eliminate the duplicate open(). What type of the object is behind fd? O_NONBLOCK affects open itself. We release flock after object close method, but before close(2) returns. From nobody Fri Jun 23 20:11:34 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QnpKw30WPz4gmj5 for ; Fri, 23 Jun 2023 20:11:48 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ua1-f47.google.com (mail-ua1-f47.google.com [209.85.222.47]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QnpKv5s6dz3hwv for ; Fri, 23 Jun 2023 20:11:47 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ua1-f47.google.com with SMTP id a1e0cc1a2514c-78cee27c08aso415682241.2 for ; Fri, 23 Jun 2023 13:11:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687551106; x=1690143106; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c5XSsh7658/ln6TELE1WbwFESaD587/kgAs/zv+bIhs=; b=NTr9xEIYY9DruLKeJ9IRwDNWwNHpvgtsH+hstjJRhW1pdb5pkMCmxefrDPeczNie76 1EEuRIIKPn1zgFhD7CRdhD0ijLqOapIJKe+miJfDg84pcONKd+Ca7bge2+YpiwvzdWQk D75qGwugXdHWgslMrN8M7EIxuzzK1Lhr6ztGnExWrceEUc6jh47dOT6N5kpwSHH4GXm8 VT4gIleHotrPTcdvknPjArdKF/I7qO4YP5nkQ7fRZC5tCk+F5IHg2lhIWw0diqLX6FKr UeAGTHqL5/tCTFgxca38GzfIxa/e0JMfh6js6X4zMiZPjfC06XYpsV33Z8W9NEQadkgj 0w5Q== X-Gm-Message-State: AC+VfDz1JGUMSHPQn/vPLXJukwzTxH3pAF+hnTKyR3rr66+yjmI6LT8P WU0Gohd7zQtlube4RZ95piGFMy6EKSoOb7EfP7cevS19y+Y= X-Google-Smtp-Source: ACHHUZ44qbct5vdVWnn52CDkKRRjbkUFG2P7Dt9qj9GUnLiKPm7k613jzE/gJsk4lQxe6tjq0kntl+aVKCDjTV1pz14= X-Received: by 2002:a67:e8d4:0:b0:440:a6fb:909f with SMTP id y20-20020a67e8d4000000b00440a6fb909fmr10010316vsn.32.1687551106260; Fri, 23 Jun 2023 13:11:46 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Fri, 23 Jun 2023 13:11:34 -0700 Message-ID: Subject: Re: Should close() release locks atomically? To: Konstantin Belousov Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4QnpKv5s6dz3hwv X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Fri, Jun 23, 2023 at 1:03=E2=80=AFPM Konstantin Belousov wrote: > > On Fri, Jun 23, 2023 at 12:00:36PM -0700, Alan Somers wrote: > > The close() syscall automatically releases locks. Should it do so > > atomically or is a delay permitted? I can't find anything in our man > > pages or the open group specification that says. > > > > The distinction matters when using O_NONBLOCK. For example: > > > > fd =3D open(..., O_DIRECT | O_EXLOCK | O_NONBLOCK); //succeeds > > // do some I/O > > close(fd); > > fd =3D open(..., O_DIRECT | O_EXLOCK | O_NONBLOCK); //fails with EAGAIN= ! > > > > I see this error frequently on a heavily loaded system. It isn't a > > typical thread race though; ktrace shows that only one thread tries to > > open the file in question. From the ktrace, I can see that the final > > open() comes immediately after the close(), with no intervening > > syscalls from that thread. It seems that close() doesn't release the > > lock right away. I wouldn't notice if I weren't using O_NONBLOCK. > > > > Should this be considered a bug? If so I could try to come up with a > > minimal test case. But it's somewhat academic, since I plan to > > refactor the code in a way that will eliminate the duplicate open(). > What type of the object is behind fd? O_NONBLOCK affects open itself. > We release flock after object close method, but before close(2) returns. This is a plain file on ZFS. From nobody Fri Jun 23 20:48:25 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Qnq8K4KwGz4gBVF for ; Fri, 23 Jun 2023 20:48:33 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Qnq8K0qGkz3xDL; Fri, 23 Jun 2023 20:48:32 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.17.1/8.17.1) with ESMTPS id 35NKmPWo022234 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 23 Jun 2023 23:48:28 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 35NKmPWo022234 Received: (from kostik@localhost) by tom.home (8.17.1/8.17.1/Submit) id 35NKmP0g022233; Fri, 23 Jun 2023 23:48:25 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 23 Jun 2023 23:48:25 +0300 From: Konstantin Belousov To: Alan Somers Cc: FreeBSD Hackers Subject: Re: Should close() release locks atomically? Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on tom.home X-Rspamd-Queue-Id: 4Qnq8K0qGkz3xDL X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Fri, Jun 23, 2023 at 01:11:34PM -0700, Alan Somers wrote: > On Fri, Jun 23, 2023 at 1:03 PM Konstantin Belousov wrote: > > > > On Fri, Jun 23, 2023 at 12:00:36PM -0700, Alan Somers wrote: > > > The close() syscall automatically releases locks. Should it do so > > > atomically or is a delay permitted? I can't find anything in our man > > > pages or the open group specification that says. > > > > > > The distinction matters when using O_NONBLOCK. For example: > > > > > > fd = open(..., O_DIRECT | O_EXLOCK | O_NONBLOCK); //succeeds > > > // do some I/O > > > close(fd); > > > fd = open(..., O_DIRECT | O_EXLOCK | O_NONBLOCK); //fails with EAGAIN! > > > > > > I see this error frequently on a heavily loaded system. It isn't a > > > typical thread race though; ktrace shows that only one thread tries to > > > open the file in question. From the ktrace, I can see that the final > > > open() comes immediately after the close(), with no intervening > > > syscalls from that thread. It seems that close() doesn't release the > > > lock right away. I wouldn't notice if I weren't using O_NONBLOCK. > > > > > > Should this be considered a bug? If so I could try to come up with a > > > minimal test case. But it's somewhat academic, since I plan to > > > refactor the code in a way that will eliminate the duplicate open(). > > What type of the object is behind fd? O_NONBLOCK affects open itself. > > We release flock after object close method, but before close(2) returns. > > This is a plain file on ZFS. Can you write a self-contained example, and check the same issue e.g. on tmpfs? From nobody Fri Jun 23 20:53:20 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QnqG56Hp8z4gH3t for ; Fri, 23 Jun 2023 20:53:33 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-vk1-f177.google.com (mail-vk1-f177.google.com [209.85.221.177]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QnqG53MhRz41kV for ; Fri, 23 Jun 2023 20:53:33 +0000 (UTC) (envelope-from asomers@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-vk1-f177.google.com with SMTP id 71dfb90a1353d-4718ddce780so487558e0c.1 for ; Fri, 23 Jun 2023 13:53:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687553612; x=1690145612; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2Z01/vlrmXYpwwtqMi5qhHYBcgdP8OILRwxIcZtwo1E=; b=bzHJFB9uxjFDOUd/2AOWb2yF9mVAaAAxUKVedLYeJQaDMFkjHOBhycxyormXmZRB2W at9p73Bn/OnoFOLzNkmoA1D5DFgiKcqlJLAIgsi50XxI1kbzklmlwA+9WlqV6YVUdnpw sLYM9g/kE0lM4f1oWeICtJ54RANXthhN4AwEoW5M5sHtwlyOMmLDmaF2wOGHDlQKfas4 fYucbx19RcNNHdnX4K/nAqlPumK2OIyCFa9ymlJRdvMtbcr+//o5h6eZV1nnDskx+Ftz 3lZoj5rRJcUKWxhGm6QQ+l7jydz8DYR4Me5xUkRUzGSMrQOOajQkGkS2csQkLER08mlK 3OZw== X-Gm-Message-State: AC+VfDwVq3IiRVQ/I8mOhUF8doToDmporwJIxVPV9cP5OVcFAZsdSZRE 7nrIB6Ic5g9j00qSBc/yDrzbF7MJL4iv7g6pXR4= X-Google-Smtp-Source: ACHHUZ4Waovmszv3o+aZf30pZJP8hqWfsOHR9yX7z7DKhEXB/1+RehhEKSZ2jz4+BTI036XqAQ0zpTV6rOMhPKxTtsQ= X-Received: by 2002:a1f:bfd3:0:b0:471:6b1d:378c with SMTP id p202-20020a1fbfd3000000b004716b1d378cmr12197298vkf.0.1687553612159; Fri, 23 Jun 2023 13:53:32 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Fri, 23 Jun 2023 13:53:20 -0700 Message-ID: Subject: Re: Should close() release locks atomically? To: Konstantin Belousov Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4QnqG53MhRz41kV X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Fri, Jun 23, 2023 at 1:48=E2=80=AFPM Konstantin Belousov wrote: > > On Fri, Jun 23, 2023 at 01:11:34PM -0700, Alan Somers wrote: > > On Fri, Jun 23, 2023 at 1:03=E2=80=AFPM Konstantin Belousov wrote: > > > > > > On Fri, Jun 23, 2023 at 12:00:36PM -0700, Alan Somers wrote: > > > > The close() syscall automatically releases locks. Should it do so > > > > atomically or is a delay permitted? I can't find anything in our m= an > > > > pages or the open group specification that says. > > > > > > > > The distinction matters when using O_NONBLOCK. For example: > > > > > > > > fd =3D open(..., O_DIRECT | O_EXLOCK | O_NONBLOCK); //succeeds > > > > // do some I/O > > > > close(fd); > > > > fd =3D open(..., O_DIRECT | O_EXLOCK | O_NONBLOCK); //fails with EA= GAIN! > > > > > > > > I see this error frequently on a heavily loaded system. It isn't a > > > > typical thread race though; ktrace shows that only one thread tries= to > > > > open the file in question. From the ktrace, I can see that the fin= al > > > > open() comes immediately after the close(), with no intervening > > > > syscalls from that thread. It seems that close() doesn't release t= he > > > > lock right away. I wouldn't notice if I weren't using O_NONBLOCK. > > > > > > > > Should this be considered a bug? If so I could try to come up with= a > > > > minimal test case. But it's somewhat academic, since I plan to > > > > refactor the code in a way that will eliminate the duplicate open()= . > > > What type of the object is behind fd? O_NONBLOCK affects open itself= . > > > We release flock after object close method, but before close(2) retur= ns. > > > > This is a plain file on ZFS. > > Can you write a self-contained example, and check the same issue e.g. on > tmpfs? I just reproduced it on tmpfs. A minimal test case will take some more tim= e... From nobody Sat Jun 24 00:00:27 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QnvPm6YMzz4gpxc; Sat, 24 Jun 2023 00:00:28 +0000 (UTC) (envelope-from salvadore@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QnvPm3JRjz3xwx; Sat, 24 Jun 2023 00:00:28 +0000 (UTC) (envelope-from salvadore@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1687564828; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc; bh=BZDboLs55RS+2CA6Wp814zfKjcwV1ra3lerb9u3JLZI=; b=itp1q0zpm/GF3P4h93cnNwqor88HXNe1lR7wE8pm4i/orOoYXXSrkT5STogwezhUVI7auy n0fUvMdFT7ahJvOpCsWQSYzQbPcn3SzJygcbZkMwr9piCYbkcBvxqzVlJmVRAUVXmIcDz3 1fVrpmpo4A9NtALk1kM4VWp2gC+tqeXKxNRBJzUJNzRaI+qTKHlDd3Y05h1sEQPyWYHLWe RL895IVMEPdLvgV5+zq/UHUq8QVr1WBdOPRyf/yyevxlyYBVQW0lGenwE6PdAvENZLNJfi FaKI56KnJhAQZGvaQYYjIwplZwjTiycTv0B8wzpbXXHVZ2fwrbXPhbk+X7nyAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1687564828; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc; bh=BZDboLs55RS+2CA6Wp814zfKjcwV1ra3lerb9u3JLZI=; b=YaeNScbd7MU7OSkk6S27L1cw8HVp3HKSYWjs64uoBI9xCU8iOA3ctZNuXOfq1FRWYs9zmh tZ1oQptsajou1pDodGH3kERjTTodJlESOaUPVLID/geVbwvBjTWqEvEY0Wws5RFxvQKZSt r9ELAfqytUgcRTjZni/GxJfj6gF7VfxuLuRRRrChHdV/liCVBnqep/+OJ4jLwe1ty0s0Yo ya6bEJ6Lk9n8DgiRrVXXTYsjM2QS1bkLIgNA3kh/6XbvEzNPj7RVF75pBeCMVBhCWLHJwh 53FF0VK7/1GDdX4vOKStODgzC7tk2azkelmpbjpgnI6ZGE09xpLIK0nV1FMT0A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1687564828; a=rsa-sha256; cv=none; b=pUq3dS8OCpCTpDcxowdYd7MMJQDEp5x1VMtihItGOVWpO6IWL6NG7bfI5/gu1JB/fOjOQI QICQh5iDocW9R+LUbsfUFC/M/TrluEgTE3hMRvf6lBOqcixyZIpnaul6reDwqTvr3c8j/e GweRO4iVu4d6iR3rdmWeNUSE+lQln+oBn0lFlsFzEdQOIjmet+3Zo/TFJtMAjqE3TG4NAm pGEniITXuvdtZQDYz8AoJE4S3+weVFPcGHQyNfmzRiq8Wbv/AaDuG1F20KkwJxGHfs7c2Q LJylxhriCsIuUL56QqDSd7wUKo7pFI1vQnrrmanVXSooNjau5eKvdkCZzvv8PA== Received: by freefall.freebsd.org (Postfix, from userid 1472) id 2927619CC8; Sat, 24 Jun 2023 00:00:27 +0000 (UTC) To: freebsd-status-calls@FreeBSD.org Subject: [LAST OFFICIAL REMINDER] Call for 2023Q2 status reports Cc: freebsd-current@FreeBSD.org,freebsd-hackers@FreeBSD.org,devsummit@FreeBSD.org,info@bsdcan.org,soc-students@FreeBSD.org,soc-mentors@FreeBSD.org Message-Id: <20230624000028.2927619CC8@freefall.freebsd.org> Date: Sat, 24 Jun 2023 00:00:27 +0000 (UTC) From: Lorenzo Salvadore X-ThisMailContainsUnwantedMimeParts: N List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Dear FreeBSD Community, The deadline for the next FreeBSD Status Report update is June, 30th 2023 for work done since the last round of quarterly reports: April 2023 - June 2023. I would like to remind you that reports are published on a quarterly basis and are usually collected during the last month of each quarter, You are also welcome to submit them even earlier if you want, and the earlier you submit them, the more time we have for reviewing. Status report submissions do not need to be very long. They may be about anything happening in the FreeBSD project and community, and they provide a great way to inform FreeBSD users and developers about work that is underway or has been completed. Report submissions are not limited to committers; anyone doing anything interesting and FreeBSD related can -- and should -- write one! The following methods are available to submit your reports: * submit a review on Phabricator and add the group "status" to the reviewers list. You should put your reports in the directory doc/website/content/en/status/report-2023-04-2023-06/ (create it if it is missing); * submit a pull request at . You should put your reports in the directory doc/website/content/en/status/report-2023-04-2023-06/ (create it if it is missing); * send an email to status-submissions@FreeBSD.org including your report. An AsciiDoc template is available at . We look forward to seeing your 2023Q2 reports! Thanks, Lorenzo Salvadore (on behalf of status@)