From owner-freebsd-arch@FreeBSD.ORG Tue Jun 4 04:43:21 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id CE81EB18 for ; Tue, 4 Jun 2013 04:43:21 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-x22f.google.com (mail-ie0-x22f.google.com [IPv6:2607:f8b0:4001:c03::22f]) by mx1.freebsd.org (Postfix) with ESMTP id 9FB8E1A6D for ; Tue, 4 Jun 2013 04:43:21 +0000 (UTC) Received: by mail-ie0-f175.google.com with SMTP id a11so1341757iee.20 for ; Mon, 03 Jun 2013 21:43:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=Hi/EKc4snZ+PBS/UVEUSxyh7IB7PY1jEqB3WBPP6uYE=; b=Igd9UP9+51Ma0AE6u36jZ9Hij6CoF9R2F0OSdzXaaG6pEBdHhqu+lOmdLCF64JH9O1 fsPhrn6dTHqf4rAvxmnsPTv2Vd3UlMXy9lDaDDJaDoFnxTfmmOjMf5WgmH6iPGzI32Ew LAMeKVqCqrl9DUlAxKmwBxXvhxEodOIUAiUGWI3tkruXxxp42pats+1I700qIzTLXxXP E/SQUF/XNeEDQg3BT2BEmn1ch7IV6p03SZsX3rD563+cn9Ybq8XacWIW+y+80lAHoYq+ SzqFZT6G0y/j8s/JSkGpH+qe1LTRhFHpmQDVSjG2TwlhYG6FthYMvgahEdSYxZnLchKJ WlCQ== X-Received: by 10.50.87.4 with SMTP id t4mr648650igz.76.1370321001338; Mon, 03 Jun 2013 21:43:21 -0700 (PDT) Received: from 53.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id l14sm22938417igf.9.2013.06.03.21.43.19 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 03 Jun 2013 21:43:20 -0700 (PDT) Sender: Warner Losh Subject: Re: Kernelspace C11 atomics for MIPS Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Mon, 3 Jun 2013 22:43:19 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Adrian Chadd X-Mailer: Apple Mail (2.1085) X-Gm-Message-State: ALoCoQmiPFMtAdKsuyL9EU6OrpVGr28+UHOOA+HgKEU5z7qKO9f/92L9BFSWQtm/5FvXbIX5JdZb Cc: Ed Schouten , freebsd-mips@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jun 2013 04:43:21 -0000 On Jun 3, 2013, at 8:45 PM, Adrian Chadd wrote: > Speaking of this; any idea why the SYNC operators have 8 NOPs = following them? Yes, that's the exact issue that I've had with them, but have never had = time to sort it out... Warner > I noticed that when going through disassemblies of various mips24k .o = files. >=20 >=20 >=20 > Adrian >=20 > On 3 June 2013 10:53, Warner Losh wrote: >>=20 >> On Jun 3, 2013, at 8:04 AM, Ed Schouten wrote: >>=20 >>> Hi, >>>=20 >>> As of r251230, it should be possible to use C11 atomics in >>> kernelspace, by including ! Even when not using = Clang >>> (but GCC 4.2), it is possible to use quite a large portion of the = API. >>> A couple of limitations: >>>=20 >>> - The memory order argument is simply ignored, making all the calls = do >>> a full memory barrier. >>> - At least Clang allows you to do arithmetic on C11 atomics directly >>> (e.g. "a +=3D 5" =3D=3D "atomic_fetch_add(&a, 5)"), which is of = course not >>> possible to mimick. >>> - The atomic functions only work on 1,2,4,8-byte types, which is >>> probably a good thing. >>>=20 >>> Amazingly, it turns out that it most of the architectures, with the >>> exception of ARM and MIPS. To make MIPS work, we need to implement >>> some of the __sync_* functions that are described here: >>>=20 >>> http://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Atomic-Builtins.html >>>=20 >>> Some time ago I already added some of these functions to our >>> libcompiler-rt in userspace, to make atomics work there. >>> Unfortunately, these functions were quite horribly implemented, as I >>> tried to build them on top of , which is far from >>> trivial/efficient. It is also restricted to 4 and 8-byte types. = That's >>> why I thought: why not spend some time learning MIPS assembly and >>> write some decent implementations for these functions? >>>=20 >>> The result: >>>=20 >>> http://80386.nl/pub/mips-stdatomic.txt >>=20 >> The number of necessary syncs varies by processor type. There's also = newer synchronization instructions that make this as efficient as = possible for all mips32r2 and mips64r2-based machines. Older Caviums, at = least and maybe newer ones, also have their own variants. What you have = will mostly work for the processors we have to support. mips_sync could = therefore be better. Doing it before AND after seems like overkill as = well. Since sync is a fairly performance killing assembler instruction, = how would you feel about allowing optimizations? >>=20 >> This is my biggest single concern about the patch, but it also my = current biggest concern about the MIPS atomic operators in general. >>=20 >>> For now, please focus on sys/mips/mips/stdatomic.c. It implements = all >>> the __sync_* functions called by for 1, 2, 4 and 8 = byte >>> types. There is some testing code in there as well, which can be >>> ignored. This code disassembles to the following: >>>=20 >>> http://80386.nl/pub/mips-stdatomic-disasm.txt >>>=20 >>> As I don't own a MIPS system myself, I was thinking about tinkering = a >>> bit with qemu to see whether these functions work properly. My >>> questions are: >>>=20 >>> - Does anyone have any comments on the C code and/or the machine = code >>> generated? Are there some nifty tricks I can apply to make the = machine >>> code more efficient that I am unaware o? >>> - Is there anyone interested in testing this code a bit more >>> thoroughly on physical hardware? >>> - Would anyone mind if I committed this to HEAD? >>=20 >> I have some cavium gear I can easily test on, and some other stuff I = can less-easily test on. >>=20 >> It wouldn't be horrible to commit to head, but it would affect = performance in many places. >>=20 >> Don't commit the kern/bla.c standard change to conf/files, it looks = to be bogus :) >>=20 >> Warner >>=20 >> _______________________________________________ >> freebsd-mips@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-mips >> To unsubscribe, send any mail to = "freebsd-mips-unsubscribe@freebsd.org"