From owner-freebsd-hackers@freebsd.org Wed Aug 22 06:49:39 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DB455108ECA4 for ; Wed, 22 Aug 2018 06:49:38 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8AAA8853A0 for ; Wed, 22 Aug 2018 06:49:38 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from mail-it0-f53.google.com (mail-it0-f53.google.com [209.85.214.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) (Authenticated sender: mmacy) by smtp.freebsd.org (Postfix) with ESMTPSA id 4874B11F1F for ; Wed, 22 Aug 2018 06:49:38 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: by mail-it0-f53.google.com with SMTP id s7-v6so1633565itb.4 for ; Tue, 21 Aug 2018 23:49:38 -0700 (PDT) X-Gm-Message-State: APzg51A/b6j4RO/g5WRSQJAhUon+pa+9J5cZ65hFPi67lGMDG5dnXSdR CHKUKQb7hf4eYXj96d7vo7cbDz387xJByBLpsqo= X-Google-Smtp-Source: ANB0VdbYfHKactwVG1eNyNUJaH28bdPTgzP/kX/F0kp3zigr7DycKWuEIBUAVcZEyJHqVoDj5XvJ0MCruDp8REsEpMM= X-Received: by 2002:a24:704f:: with SMTP id f76-v6mr2120864itc.30.1534920577728; Tue, 21 Aug 2018 23:49:37 -0700 (PDT) MIME-Version: 1.0 References: <15e3f080-2f82-a243-80e9-f0a916445828@embedded-brains.de> In-Reply-To: <15e3f080-2f82-a243-80e9-f0a916445828@embedded-brains.de> From: Matthew Macy Date: Tue, 21 Aug 2018 23:49:26 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: epoch(9) background information? To: sebastian.huber@embedded-brains.de Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Aug 2018 06:49:39 -0000 On Tue, Aug 21, 2018 at 11:42 PM Sebastian Huber < sebastian.huber@embedded-brains.de> wrote: > On 22/08/18 08:34, Sebastian Huber wrote: > > On 21/08/18 15:38, Jacques Fourie wrote: > >> > >> > >> On Tue, Aug 21, 2018 at 8:33 AM, Sebastian Huber > >> >> > wrote: > >> > >> Hello, > >> > >> I update currently a port of the FreeBSD network stack, etc. to > >> the real-time operating system RTEMS from the head version at > >> 2017-04-04 to the head version of today. I noticed that some > >> read-write locks are replaced by a relatively new stuff called > >> EPOCH(9). Is there some background information available for this? > >> The man page is a bit vague and searching for something named > >> epoch on the internet is not really great. For example, what is > >> the motivation for this change? How is this related to > >> read-copy-update (RCU)? > >> > >> -- Sebastian Huber, embedded brains GmbH > >> > >> Address : Dornierstr. 4, D-82178 Puchheim, Germany > >> < > https://maps.google.com/?q=3DDornierstr.+4,+D-82178+Puchheim,+Germany&ent= ry=3Dgmail&source=3Dg > > > >> Phone : +49 89 189 47 41-16 > >> Fax : +49 89 189 47 41-09 > >> E-Mail : sebastian.huber@embedded-brains.de > >> > >> PGP : Public key available on request. > >> > >> Diese Nachricht ist keine gesch=C3=A4ftliche Mitteilung im Sinne d= es > >> EHUG. > >> > >> _______________________________________________ > >> freebsd-hackers@freebsd.org > >> mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > >> > >> To unsubscribe, send any mail to > >> "freebsd-hackers-unsubscribe@freebsd.org > >> " > >> > >> > >> Additional information is available here : > >> http://concurrencykit.org/presentations/ebr.pdf > >> . The way I > >> understand it is that it is mostly used in place of read locks to > >> provide liveness guarantees without using atomics. Additional detail > >> is available in the commit messages. As an example see r333813 for > >> some performance data. > >> > > > > Thanks, for the reference. The "epoch reclamation" are good keywords > > to find more information. > > > > What is the right mailing list to ask questions about the epoch > > implementation of the FreeBSD kernel? > > > > To support this machinery in RTEMS is a bit difficult (in particular > > EPOCH_LOCKED). Since RTEMS is supposed to be a real-time operating > > system it supports only fixed-priority and job-level fixed priority > > (EDF) schedulers. To allow some scaling to larger SMP systems it > > supports clustered scheduling together with the mutual exclusion > > locking protocols MrsP > > (http://www-users.cs.york.ac.uk/~burns/MRSPpaper.pdf) and OMIP > > (http://www.mpi-sws.org/~bbb/papers/pdf/ecrts13b.pdf). This makes the > > thread pinning hard to implement (which is very easy to support in > > FreeBSD). The locking protocols may temporarily move a thread which > > owns a mutex to a foreign scheduler instance, e.g. a thread which > > wants to obtain the mutex helps the owner to make progress if it was > > pre-empted in its home scheduler instance. Due to a timeout of the > > helper the owner may loose the right to execute in the foreign > > scheduler instance. This would make it impossible to fulfil the > > processor pinning constraint (e.g. the thread priority in the foreign > > scheduler instance is undefined). > > > > It would save me a lot of trouble if I could assume that EPOCH_LOCKED > > is an exotic feature which is unlikely to get used in FreeBSD. > > > > Another question, is it a common use case to call epoch_enter_preempt() > and epoch_exit_preempt() while owning a mutex? > Yes. Very. It is generally not permitted to hold a mutex across epoch_wait() that's why there's the special flag EPOCH_LOCKED. If you have a very limited number of threads, you might want to have each thread have its own record registered with the epoch. Then you wouldn't need the CPU pinning. The pinning is just away of providing a limited number of records to an unbounded number of threads. -M