Date: Wed, 2 Aug 2023 19:25:50 -0700 From: Rick Macklem <rick.macklem@gmail.com> To: Konstantin Belousov <kostikbel@gmail.com> Cc: David Chisnall <theraven@freebsd.org>, FreeBSD CURRENT <freebsd-current@freebsd.org> Subject: Re: Review of patch that uses "volatile sig_atomic_t" Message-ID: <CAM5tNy7smc%2BNRgJvqZdUWPqQzovkPU3DZ0H8zGzP_yQcqOSk_w@mail.gmail.com> In-Reply-To: <ZMrfGhS6eFU2aQDQ@kib.kiev.ua> References: <CAM5tNy5aAyRk_CML57Q7OEPXGVEjM=o8fqWdLJCRkHubBYzCNQ@mail.gmail.com> <A5E9074A-5D63-4351-9C0B-D7990E12AC9E@freebsd.org> <CAM5tNy7jNQ2NB91V96vgMQ8AeZSVp5rHMZcZ7PG0WPQyBEJ6XQ@mail.gmail.com> <ZMrfGhS6eFU2aQDQ@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Aug 2, 2023 at 6:14=E2=80=AFPM Konstantin Belousov <kostikbel@gmail= .com> wrote: > > On Tue, Aug 01, 2023 at 04:33:16PM -0700, Rick Macklem wrote: > > On Mon, Jul 31, 2023 at 11:33=E2=80=AFPM David Chisnall <theraven@freeb= sd.org> wrote: > > > > > > Hi, > > > > > > This bit of the C spec is a bit of a mess. There was, I believe, a de= sire to return volatile to its original use and make any use of volatile ot= her than MMIO discouraged. This broke too much legacy code and so now it=E2= =80=99s a confusing state. > > > > > > The requirements for volatile are that the compiler must not elide lo= ads or stores and may not narrow them (I am not sure if it=E2=80=99s allowe= d to widen them). Loads and stores to a volatile variable may not be reorde= red with respect to other loads or stores to the same object but *may* be r= eordered with respect to any other accesses. > > > > > > The sig_atomic_t typedef just indicates an integer type that can be l= oaded and stored with a single instruction and so is immune to tearing if u= pdated from a signal handler. There is no requirement to use this from sign= al handlers in preference to int on FreeBSD (whether other types work is im= plementation defined and int works on all supported architectures for us). > > > > > > The weak ordering guarantees for volatile mean that any code using vo= latile for detecting whether a signal has fired is probably wrong if if doe= s not include a call to automic_signal_fence(). This guarantees that the co= mpiler will not reorder the load of the volatile with respect to other acce= sses. In practice, compilers tend to be fairly conservative about reorderin= g volatile accesses and so it probably won=E2=80=99t break until you upgrad= e your compiler in a few years time. > > > > > > My general recommendation is to use _Atomic(int) (or ideally a enum t= ype) for this. If you just use it like a normal int, you will get sequentia= lly consistent atomics. On a weakly ordered platform like Arm this will inc= lude some more atomic barrier instructions but it will do the right thing i= f you add additional threads monitoring the same variable later. In somethi= ng like mountd, the extra performance overhead from the barriers is unlikel= y to be measurable, if it is then you can weaken the atomicity (sequentiall= y consistent unless specified otherwise is a good default in C/C++, for onc= e prioritising correctness over performance). > > > > Just trying to understand what you are suggesting... > > 1 - Declare the variable _Atomic(int) OR atomic_int (is there a prefere= nce) and > > not volatile. > Really does not matter. > > > 2 - Is there a need for signal_atomic_fence(memory_order_acquire); befo= re the > > assignment of the variable in the signal handler. (This exists in > > one place in > > the source tree (bin/dd/misc,c), although for this example, > > neither volatile nor > > _Atomic() are used for the variable's declaration. > In mountd, there are two signal handlers. One for SIGHUP, and another fo= r > SIGTERM. They are very different, let me explain. > > For SIGHUP, the only action in the signal handler is the assignment to > the variable. The assignment itself is atomic on FreeBSD. But what is m= ore > important, there is no ordering issues between this assignment and any ot= her > action. Eventually the main execution context notices that the variable = is > set and does something (re-read config). As consequence, there is no nee= d > in any fencing of the SIGHUP handler. The only concern would be that the setting of "got_sighup" would get lost d= ue to some compiler optimization and it sounds like _Atomic(int) is sufficient= to guarantee that will not happen. rick > > > 3 - Is there any need for other atomic_XXX() calls where the variable i= s used > > outside of the signal handler? > > The SIGTERM is different, it does some rpc bind calls to unregister itsel= f > as a mount program. If these actions can interfere with the main context > execution (I believe they are), then userspace rpc bind client state can > be corrupted, resulting in failing attempt to unregister. > > No amount of atomics or fencing would fix it, the unregister action shoul= d > be either moved out of the signal handler context to main context. Anoth= er > option might be to block SIGTERM, and only unblock it in places where it > is safe to do rpcb calls from interrupt. This is approximately what dd(1= ) > does. Yes, I think moving the termination into the main context should work. I'll do that as a separate commit/review. I suspect that failing to de-register doesn't cause too much of a problem. since the client won't be able to connect to the port# after mountd has terminated and will fail (just maybe not a gracefully). Same would happen if mountd crashes for some reason (and not terminated via SIGTERM). rick > > > > > In general, it is looking like FreeBSD needs to have a standard way of = dealing > > with this and there will be assorted places that need to be fixed? > > > > Thanks, rick > > > > > > David > > > > > > > On 1 Aug 2023, at 06:14, Rick Macklem <rick.macklem@gmail.com> wrot= e: > > > > > > > > =EF=BB=BFHi, > > > > > > > > I just put D41265 up on phabricator. It is a trivial > > > > change to mountd.c that defines the variable set > > > > by got_sighup() (the SIGHUP handler) as > > > > static volatile sig_atomic_t > > > > instead of > > > > static int > > > > > > > > I did list a couple of reviewers, but if you are familiar > > > > with this C requirement, please take a look at it and > > > > review it. > > > > > > > > Thanks, rick > > > > ps: I was unaware of this C requirement until Peter Eriksson > > > > reported it to me yesterday. Several of the other NFS > > > > related daemons probably need to same fix, which I will > > > > do after this is reviewed. > > > > > > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAM5tNy7smc%2BNRgJvqZdUWPqQzovkPU3DZ0H8zGzP_yQcqOSk_w>