From owner-svn-src-stable@FreeBSD.ORG Thu Jun 18 13:46:33 2015 Return-Path: Delivered-To: svn-src-stable@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B3328C21; Thu, 18 Jun 2015 13:46:33 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 947FD1DA; Thu, 18 Jun 2015 13:46:33 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id t5IDkXsu021677; Thu, 18 Jun 2015 13:46:33 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id t5IDkXLb021676; Thu, 18 Jun 2015 13:46:33 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201506181346.t5IDkXLb021676@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Thu, 18 Jun 2015 13:46:33 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r284548 - stable/10/sys/kern X-SVN-Group: stable-10 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for all the -stable branches of the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jun 2015 13:46:33 -0000 Author: kib Date: Thu Jun 18 13:46:32 2015 New Revision: 284548 URL: https://svnweb.freebsd.org/changeset/base/284548 Log: MFC r284178: Add barriers when updating and reading th_generation. MFC r284256: Tweaks for r284178. Modified: stable/10/sys/kern/kern_tc.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/kern/kern_tc.c ============================================================================== --- stable/10/sys/kern/kern_tc.c Thu Jun 18 13:45:07 2015 (r284547) +++ stable/10/sys/kern/kern_tc.c Thu Jun 18 13:46:32 2015 (r284548) @@ -70,7 +70,7 @@ struct timehands { struct timeval th_microtime; struct timespec th_nanotime; /* Fields not to be copied in tc_windup start with th_generation. */ - volatile u_int th_generation; + u_int th_generation; struct timehands *th_next; }; @@ -189,6 +189,33 @@ tc_delta(struct timehands *th) tc->tc_counter_mask); } +static inline u_int +tc_getgen(struct timehands *th) +{ + +#ifdef SMP + return (atomic_load_acq_int(&th->th_generation)); +#else + u_int gen; + + gen = th->th_generation; + __compiler_membar(); + return (gen); +#endif +} + +static inline void +tc_setgen(struct timehands *th, u_int newgen) +{ + +#ifdef SMP + atomic_store_rel_int(&th->th_generation, newgen); +#else + __compiler_membar(); + th->th_generation = newgen; +#endif +} + /* * Functions for reading the time. We have to loop until we are sure that * the timehands that we operated on was not updated under our feet. See @@ -204,10 +231,10 @@ fbclock_binuptime(struct bintime *bt) do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); *bt = th->th_offset; bintime_addx(bt, th->th_scale * tc_delta(th)); - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); } void @@ -262,9 +289,9 @@ fbclock_getbinuptime(struct bintime *bt) do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); *bt = th->th_offset; - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); } void @@ -275,9 +302,9 @@ fbclock_getnanouptime(struct timespec *t do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); bintime2timespec(&th->th_offset, tsp); - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); } void @@ -288,9 +315,9 @@ fbclock_getmicrouptime(struct timeval *t do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); bintime2timeval(&th->th_offset, tvp); - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); } void @@ -301,9 +328,9 @@ fbclock_getbintime(struct bintime *bt) do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); *bt = th->th_offset; - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); bintime_add(bt, &boottimebin); } @@ -315,9 +342,9 @@ fbclock_getnanotime(struct timespec *tsp do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); *tsp = th->th_nanotime; - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); } void @@ -328,9 +355,9 @@ fbclock_getmicrotime(struct timeval *tvp do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); *tvp = th->th_microtime; - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); } #else /* !FFCLOCK */ void @@ -341,10 +368,10 @@ binuptime(struct bintime *bt) do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); *bt = th->th_offset; bintime_addx(bt, th->th_scale * tc_delta(th)); - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); } void @@ -399,9 +426,9 @@ getbinuptime(struct bintime *bt) do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); *bt = th->th_offset; - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); } void @@ -412,9 +439,9 @@ getnanouptime(struct timespec *tsp) do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); bintime2timespec(&th->th_offset, tsp); - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); } void @@ -425,9 +452,9 @@ getmicrouptime(struct timeval *tvp) do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); bintime2timeval(&th->th_offset, tvp); - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); } void @@ -438,9 +465,9 @@ getbintime(struct bintime *bt) do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); *bt = th->th_offset; - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); bintime_add(bt, &boottimebin); } @@ -452,9 +479,9 @@ getnanotime(struct timespec *tsp) do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); *tsp = th->th_nanotime; - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); } void @@ -465,9 +492,9 @@ getmicrotime(struct timeval *tvp) do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); *tvp = th->th_microtime; - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); } #endif /* FFCLOCK */ @@ -880,11 +907,11 @@ ffclock_read_counter(ffcounter *ffcount) */ do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); ffth = fftimehands; delta = tc_delta(th); *ffcount = ffth->tick_ffcount; - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); *ffcount += delta; } @@ -988,9 +1015,9 @@ dtrace_getnanotime(struct timespec *tsp) do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); *tsp = th->th_nanotime; - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); } /* @@ -1028,7 +1055,7 @@ sysclock_getsnapshot(struct sysclock_sna do { th = timehands; - gen = th->th_generation; + gen = tc_getgen(th); fbi->th_scale = th->th_scale; fbi->tick_time = th->th_offset; #ifdef FFCLOCK @@ -1042,7 +1069,7 @@ sysclock_getsnapshot(struct sysclock_sna #endif if (!fast) delta = tc_delta(th); - } while (gen == 0 || gen != th->th_generation); + } while (gen == 0 || gen != tc_getgen(th)); clock_snap->delta = delta; clock_snap->sysclock_active = sysclock_active; @@ -1260,7 +1287,7 @@ tc_windup(void) tho = timehands; th = tho->th_next; ogen = th->th_generation; - th->th_generation = 0; + tc_setgen(th, 0); bcopy(tho, th, offsetof(struct timehands, th_generation)); /* @@ -1377,7 +1404,7 @@ tc_windup(void) */ if (++ogen == 0) ogen = 1; - th->th_generation = ogen; + tc_setgen(th, ogen); /* Go live with the new struct timehands. */ #ifdef FFCLOCK @@ -1643,13 +1670,13 @@ pps_capture(struct pps_state *pps) KASSERT(pps != NULL, ("NULL pps pointer in pps_capture")); th = timehands; - pps->capgen = th->th_generation; + pps->capgen = tc_getgen(th); pps->capth = th; #ifdef FFCLOCK pps->capffth = fftimehands; #endif pps->capcount = th->th_counter->tc_get_timecount(th->th_counter); - if (pps->capgen != th->th_generation) + if (pps->capgen != tc_getgen(th)) pps->capgen = 0; } @@ -1669,7 +1696,7 @@ pps_event(struct pps_state *pps, int eve KASSERT(pps != NULL, ("NULL pps pointer in pps_event")); /* If the timecounter was wound up underneath us, bail out. */ - if (pps->capgen == 0 || pps->capgen != pps->capth->th_generation) + if (pps->capgen == 0 || pps->capgen != tc_getgen(pps->capth)) return; /* Things would be easier with arrays. */ @@ -1719,7 +1746,7 @@ pps_event(struct pps_state *pps, int eve bintime2timespec(&bt, &ts); /* If the timecounter was wound up underneath us, bail out. */ - if (pps->capgen != pps->capth->th_generation) + if (pps->capgen != tc_getgen(pps->capth)) return; *pcount = pps->capcount;