From owner-freebsd-mips@FreeBSD.ORG Mon Jun 3 17:53:14 2013 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 413E56D3 for ; Mon, 3 Jun 2013 17:53:14 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ve0-x22f.google.com (mail-ve0-x22f.google.com [IPv6:2607:f8b0:400c:c01::22f]) by mx1.freebsd.org (Postfix) with ESMTP id F33231CFB for ; Mon, 3 Jun 2013 17:53:13 +0000 (UTC) Received: by mail-ve0-f175.google.com with SMTP id da11so3079467veb.34 for ; Mon, 03 Jun 2013 10:53:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=zZ9JiRMvGJMFntbqO8iJk2bKrHzgv6TJ/3Bb1fz2rI4=; b=M/C59X8um5PrD8xuYSoM5rKrgvtyEQRo6Y/oRhsiE0+tORGe/BOsSrhWpDuakB+rp1 Fh7b+F7E+AY361H7T7sbpf65WlsPoqvrVOWqE3ARycYICBpnzuXYKt/1bibOOERTB41O ViazIxGPqkRlqtM0AZgKwgY/sr7nTB005ajiIiiN1Swids+NsdLwEHverMsNnVSBaH3v UwPAkI3U3OTpvXOP4MSfRoErzYcc5rVn57gT/3Vy/9TKfBgU21BW9s3ktbMLsJawrNkC 3zfDvAzBEVNlz1ap/V87pE/QF15zSkQHe9ADPpCXuY8lvlYXK5FN12eKdSLUi4/6KiAZ Z4eA== X-Received: by 10.52.30.14 with SMTP id o14mr13771998vdh.106.1370281993273; Mon, 03 Jun 2013 10:53:13 -0700 (PDT) Received: from monkey-bot.int.fusionio.com ([209.117.142.2]) by mx.google.com with ESMTPSA id s6sm37041931vdj.5.2013.06.03.10.53.11 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 03 Jun 2013 10:53:12 -0700 (PDT) Sender: Warner Losh Subject: Re: Kernelspace C11 atomics for MIPS Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: Date: Mon, 3 Jun 2013 11:53:09 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Ed Schouten X-Mailer: Apple Mail (2.1085) X-Gm-Message-State: ALoCoQnYyk6dBKqoS8a5Yov8o8MUgpyEFEKE1YSR/npPNc2UXTWm6m4RpYnH9GzAZJAliXzgOSx3 Cc: freebsd-mips@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Jun 2013 17:53:14 -0000 On Jun 3, 2013, at 8:04 AM, Ed Schouten wrote: > Hi, >=20 > As of r251230, it should be possible to use C11 atomics in > kernelspace, by including ! Even when not using Clang > (but GCC 4.2), it is possible to use quite a large portion of the API. > A couple of limitations: >=20 > - The memory order argument is simply ignored, making all the calls do > a full memory barrier. > - At least Clang allows you to do arithmetic on C11 atomics directly > (e.g. "a +=3D 5" =3D=3D "atomic_fetch_add(&a, 5)"), which is of course = not > possible to mimick. > - The atomic functions only work on 1,2,4,8-byte types, which is > probably a good thing. >=20 > Amazingly, it turns out that it most of the architectures, with the > exception of ARM and MIPS. To make MIPS work, we need to implement > some of the __sync_* functions that are described here: >=20 > http://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Atomic-Builtins.html >=20 > Some time ago I already added some of these functions to our > libcompiler-rt in userspace, to make atomics work there. > Unfortunately, these functions were quite horribly implemented, as I > tried to build them on top of , which is far from > trivial/efficient. It is also restricted to 4 and 8-byte types. That's > why I thought: why not spend some time learning MIPS assembly and > write some decent implementations for these functions? >=20 > The result: >=20 > http://80386.nl/pub/mips-stdatomic.txt The number of necessary syncs varies by processor type. There's also = newer synchronization instructions that make this as efficient as = possible for all mips32r2 and mips64r2-based machines. Older Caviums, at = least and maybe newer ones, also have their own variants. What you have = will mostly work for the processors we have to support. mips_sync could = therefore be better. Doing it before AND after seems like overkill as = well. Since sync is a fairly performance killing assembler instruction, = how would you feel about allowing optimizations? This is my biggest single concern about the patch, but it also my = current biggest concern about the MIPS atomic operators in general. > For now, please focus on sys/mips/mips/stdatomic.c. It implements all > the __sync_* functions called by for 1, 2, 4 and 8 byte > types. There is some testing code in there as well, which can be > ignored. This code disassembles to the following: >=20 > http://80386.nl/pub/mips-stdatomic-disasm.txt >=20 > As I don't own a MIPS system myself, I was thinking about tinkering a > bit with qemu to see whether these functions work properly. My > questions are: >=20 > - Does anyone have any comments on the C code and/or the machine code > generated? Are there some nifty tricks I can apply to make the machine > code more efficient that I am unaware o? > - Is there anyone interested in testing this code a bit more > thoroughly on physical hardware? > - Would anyone mind if I committed this to HEAD? I have some cavium gear I can easily test on, and some other stuff I can = less-easily test on. It wouldn't be horrible to commit to head, but it would affect = performance in many places. Don't commit the kern/bla.c standard change to conf/files, it looks to = be bogus :) Warner