From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 27 13:27:09 2009 Return-Path: Delivered-To: hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7ADC2106568F; Sun, 27 Sep 2009 13:27:09 +0000 (UTC) (envelope-from gabor@FreeBSD.org) Received: from server.mypc.hu (server.mypc.hu [87.229.73.95]) by mx1.freebsd.org (Postfix) with ESMTP id 02AB58FC08; Sun, 27 Sep 2009 13:27:08 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by server.mypc.hu (Postfix) with ESMTP id 2522914D9720; Sun, 27 Sep 2009 15:27:08 +0200 (CEST) X-Virus-Scanned: amavisd-new at example.com Received: from server.mypc.hu ([127.0.0.1]) by localhost (server.mypc.hu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id vG6w3+3kamNs; Sun, 27 Sep 2009 15:27:05 +0200 (CEST) Received: from [192.168.1.105] (catv-89-132-179-104.catv.broadband.hu [89.132.179.104]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by server.mypc.hu (Postfix) with ESMTPSA id 33C6414D970A; Sun, 27 Sep 2009 15:27:05 +0200 (CEST) Message-ID: <4ABF6824.9090601@FreeBSD.org> Date: Sun, 27 Sep 2009 15:27:00 +0200 From: Gabor Kovesdan User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: hackers@FreeBSD.ORG Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Roman Divacky Subject: BSDL texttools status and further thoughts... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Sep 2009 13:27:09 -0000 Hello all, recently, I've had a discussion with rdivacky@ about the status of these tools. It's about bc, dc, grep, sort and iconv. He has persuaded me to write a summary here in case someone else is interested in contributing to these tools. So here I come with a little summary. BSD bc/dc will come just after 8.0-RELEASE. They are quite mature and delphij@ offered to help me getting this into the three by reviewing and approving my changes (I only have doc/ports bit). BSD grep is also quite mature, I've fixed the last critical bug recently. My only concern is the performance. GNU is fast but has ~8 KSLOC. BSD grep is slightly slower but has only ~1.5 KSLOC. It's a huge difference in complexity and GNU grep is very hard to read but they use a lot of custom optimizations to get this performance. I think we should go another way and have a well-optimized and mature regex library. The current one is very old and doesn't have wchar support, it's slow like hell and doesn't support custom GNU bullshit, which is unfortunately necessary to maintain compatiblity. (e.g. "(a|)" is considered invalid in strict POSIX regex but GNU accepts it!) Because of this, BSD grep is linked to the GNU regex library at the moment but because of the custom magic in grep it's still slower a bit. If we can live with this slight performance hit, we can commit it, I think because it's quite feature-complete. You know, I'm a beginner but I think that the code of BSD grep is so tiny and simple that there are almost absolutely no ways to optimize it more by simplifying the code, so I think further optimization should be done in the regex library. As for the regex library, NetBSD's SoC project is worth a look. I'm interested in this but I have too much things in the queue to start another one... As for sort, it isn't so mature yet. I've just made a TODO list of the known missing features or bugs: - sometimes it segfaults when reading huge files - the -k option isn't implemented yet - the -n option doesn't work correctly - preproc() optimization (I don't what it refers to actually but I had it on my previous TODO list, will have to check) - polishing man page - adding some more test cases to the regression test - checking performance (in this case, it really matters because sorting is an algorithmic piece of cake and I'm not an algorithmic guru... And this version of sort was written by me from scratch. The OpenBSD-one isn't wchar-clean and can't be fixed by design. This sort is much more tiny but it seems the algorithm isn't optimal.) As for iconv, I'll keep working on it in my BSc thesis. The forward (foo -> utf32) conversions are almost completely GNU-compatible, the reverse ones not so much. GNU has an optional transliteration, while BSD iconv uses it at default so I compared the output to GNU's transliterated output and it has some more advanced mappings to do this. Apart from this, almost all encodings are supported, that we have in locale(1) charmaps but the Big5 module segfaults. I hope I'll be able to solve these issues and check performance as part of my BSc thesis. Regards, -- Gabor Kovesdan FreeBSD Volunteer EMAIL: gabor@FreeBSD.org .:|:. gabor@kovesdan.org WEB: http://people.FreeBSD.org/~gabor .:|:. http://kovesdan.org From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 27 23:30:53 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 02925106568B for ; Sun, 27 Sep 2009 23:30:53 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from shell.rawbw.com (shell.rawbw.com [198.144.192.42]) by mx1.freebsd.org (Postfix) with ESMTP id E5D4E8FC18 for ; Sun, 27 Sep 2009 23:30:52 +0000 (UTC) Received: from eagle.syrec.org (c-24-6-221-126.hsd1.ca.comcast.net [24.6.221.126]) (authenticated bits=0) by shell.rawbw.com (8.13.6/8.13.6) with ESMTP id n8RNUpbq052322 for ; Sun, 27 Sep 2009 16:30:51 -0700 (PDT) Message-ID: <4ABFF5AA.70303@rawbw.com> Date: Sun, 27 Sep 2009 16:30:50 -0700 From: Yuri User-Agent: Thunderbird 2.0.0.23 (X11/20090824) MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: 72-STABLE compilation errors X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: yuri@rawbw.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Sep 2009 23:30:53 -0000 I updated the source today (RELENG_7) and got the errors, see below. I compile on i386 platform. Yuri ---- errors ---- cc -I/usr/local/include -DIN_GCC -DHAVE_CONFIG_H -DPREFIX=\"/usr/obj/usr/src/tmp/usr\" -I/usr/obj/usr/src/tmp/usr/src/gnu/usr.bin/cc/cc_int/../cc_tools -I/usr/src/gnu/usr.bin/cc/cc_int/../cc_tools -I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc -I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc/config -I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcclibs/include -I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcclibs/libcpp/include -I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcclibs/libdecnumber -I/usr/obj/usr/src/tmp/legacy/usr/include -c /usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc/config/i386/i386.c /usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc/config/i386/i386.c: In function 'ix86_function_regparm': /usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc/config/i386/i386.c:2502: warning: initialization makes pointer from integer without a cast /usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc/config/i386/i386.c:2503: error: dereferencing pointer to incomplete type /usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc/config/i386/i386.c: In function 'ix86_function_sseregparm': /usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc/config/i386/i386.c:2581: warning: initialization makes pointer from integer without a cast /usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc/config/i386/i386.c:2582: error: dereferencing pointer to incomplete type *** Error code 1 Stop in /usr/src/gnu/usr.bin/cc/cc_int. *** Error code 1 From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 28 16:51:23 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79FEA106566B for ; Mon, 28 Sep 2009 16:51:23 +0000 (UTC) (envelope-from forensec@yahoo.de) Received: from web28510.mail.ukl.yahoo.com (web28510.mail.ukl.yahoo.com [87.248.110.189]) by mx1.freebsd.org (Postfix) with SMTP id 033358FC19 for ; Mon, 28 Sep 2009 16:51:22 +0000 (UTC) Received: (qmail 87708 invoked by uid 60001); 28 Sep 2009 16:24:41 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.de; s=s1024; t=1254155081; bh=dfRmyb7ckDVCT3qzr2nZncaWIwPbxxKvm+UnFA1h9dg=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=X1pqxFw785TIp6uFoVif8jD+MkA3Dqp/bXuwP1963yshIagySxvCOmno3YK4KoLFslrMTBk5iVLdSZ8EzYhagYQ3tlV4G4Kvr2HS4JS76KnVezeasgjddRKQmcHp1S4sJWf2M+h65Upq8qFloC8TjZkHxcczAEvi4WySke7NqC0= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.de; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=1BzSOHNaPkNqZytt17ePN6K9OJPWSjyTUFN0w9fhPt3QBdXUsFrXJZDQLE3zfrwb7Zme2hczGk4JKMqj4an8YDDdET3ZaXb6g4QTaIcxaxHjIy8qjEZ/S5HHeWcXDAU6O5dSCARI5I4O6U6p4PJbatJO9F9/vR/w5hLDEpDveRk=; Message-ID: <112995.77056.qm@web28510.mail.ukl.yahoo.com> X-YMail-OSG: uCBKw48VM1nUvtjrEpQBVo_l5YSUxDXZBzjYzWMJt7mjP8DnQrV545E7 Received: from [78.53.192.36] by web28510.mail.ukl.yahoo.com via HTTP; Mon, 28 Sep 2009 09:24:40 PDT X-Mailer: YahooMailClassic/7.0.14 YahooMailWebService/0.7.347.3 Date: Mon, 28 Sep 2009 09:24:40 -0700 (PDT) From: Leunam Elebek To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Trouble with copyout, memcpy.... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Sep 2009 16:51:23 -0000 Hey list,I currently code a driver under Current 8.0 for Current 8.0.But th= ere are some problems with kernel/user-space interaction.I've the following= structure:struct daq_kitinfo {=A0 =A0 =A0 =A0 uint32_t ki_maxdata;=A0 =A0 = =A0 =A0 uint32_t ki_flags;=A0 =A0 =A0 =A0 uint32_t ki_rng_type;=A0 =A0 =A0 = =A0 int =A0 =A0 =A0 =A0 =A0 ki_type;=A0 =A0 =A0 =A0 int =A0 =A0 =A0 =A0 =A0= ki_nchan;=A0 =A0 =A0 =A0 int =A0 =A0 =A0 =A0 =A0 ki_len_chanl;};The above = structure is used in my user-space app:int main(void) {=A0 =A0 =A0 =A0 stru= ct daq_kitinfo *info;=A0 =A0 =A0 =A0 struct daq_kit kit; =A0 =A0 =A0 =A0 in= t fd, size; =A0 =A0 =A0 =A0 ...=A0 =A0 =A0 =A0 ...=A0 =A0 =A0 =A0 ...=A0 = =A0 =A0 =A0 /* =A0 =A0 =A0 =A0 =A0* At this point I'll try to alloc memory. Notice that =A0 =A0 =A0 =A0 =A0* the size i dependet = from another struct entry.=A0 =A0 =A0 =A0 =A0*/=A0 =A0 =A0 =A0 size =3D siz= eof(*info) * kit.k_nkits;=A0 =A0 =A0 =A0 info =3D malloc(size);=A0 =A0 =A0 = =A0 if (info =3D=3D NULL)=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 exit(ENOMEM);=A0 = =A0 =A0 =A0=A0=A0 =A0 =A0 =A0 /*=A0 =A0 =A0 =A0 =A0* The next step is to ca= ll the drivers ioctl() interface=A0 =A0 =A0 =A0 =A0* (the reason for that i= s described below).=A0 =A0 =A0 =A0 =A0*/=A0 =A0 =A0 =A0 if (ioctl(fd, DAQ_K= ITINFO, info)) {=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 printf("errno: %d\n", errno= ); =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0exit(errno);=A0 =A0 =A0 =A0 }=A0 =A0 = =A0 =A0 =A0printf("[ki_nchan] %d\n", ki_nchan);=A0 =A0 =A0 =A0 =A0...=A0 = =A0 =A0 =A0 =A0...=A0 =A0 =A0 =A0 =A0return (0);}and inside the driver (put= it simply):static intmy_ioctl(struct cdev *dev, u_long cmd, caddr_t arg, int flags,=A0 =A0 struct thread *td){=A0 =A0 =A0 =A0 struct daq_kitin= fo *info;=A0 =A0 =A0 =A0 struct daq_kit =3D dev->si_drv1; =A0 =A0 =A0 =A0 i= nt size; =A0 =A0 =A0 =A0 ...=A0 =A0 =A0 =A0 /* Do something useful e.g mute= x'ing... */=A0 =A0 =A0 =A0 ...=A0 =A0 =A0 =A0 /* The same as in user-space.= .. */=A0 =A0 =A0 =A0 size =3D sizeof(*info) * kit.k_nkits;=A0 =A0 =A0 =A0 i= nfo =3D malloc(sz, M_DAQ, M_NOWAIT | M_ZERO);=A0 =A0 =A0 =A0 if (info =3D= =3D NULL)=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ....=A0 =A0 =A0 =A0 /*=A0 =A0 =A0 = =A0 =A0* Here I want to copy struct info from kernel to user-space.=A0 =A0 = =A0 =A0 =A0* If i use memcpy, the result is that the system hangs=A0 =A0 = =A0 =A0 =A0* and I need to reboot the machine. OK, I thought=A0 =A0 =A0 =A0= =A0* copyout() should be able to do the job for me...=A0 =A0 =A0 =A0 =A0*/= =A0 =A0 =A0 =A0 =A0if (copyout(info, arg, sz))=A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0/*=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * Fuc[k-k] i still come inside this block. I alway= s=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * get an EFAULT error. =A0 =A0 =A0 =A0= =A0 =A0 =A0 =A0 =A0 */}I really don't know what I should do to let the dri= ver workingproperly. The driver should grap some informations/attributes,an= d fill up the info structure, so we can copy the filled info structto the u= ser's app.I hope somebody can help me to resolve that problem.Ah, the corre= sponding ioctl is:#define GRP =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0'd'#define DAQ_KITINFO =A0 =A0 =A0 =A0_IOR(GRP, 3, struct daq_kitinfo)Th= anks for attention and greatz from germanyMG=0A=0A=0A From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 28 17:19:09 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA0011065670 for ; Mon, 28 Sep 2009 17:19:09 +0000 (UTC) (envelope-from forensec@yahoo.de) Received: from web28503.mail.ukl.yahoo.com (web28503.mail.ukl.yahoo.com [87.248.110.182]) by mx1.freebsd.org (Postfix) with SMTP id 46F6F8FC13 for ; Mon, 28 Sep 2009 17:19:09 +0000 (UTC) Received: (qmail 71852 invoked by uid 60001); 28 Sep 2009 17:19:08 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.de; s=s1024; t=1254158348; bh=+r4Khi3ETss0sCO07+MQOf05/q1kmAn4llG9obFfcuI=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=Q2ZMULq43teWKPTm4dM+7UyDosL60pzjg3zdjk3lrnspsJmzRPIWJSU4jrmsHzl7sWjkJYAzZy7I4OcaF5om1UhM9RDjXmrHm3DgYkHK00sceQo1C9r/hP/1wMS/j3bInhQzZm/T7CF5VBhB5/t60kNJSYmaPY9zzrKHudmyIkM= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.de; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=qcu9kCQnqcrtEMVoarW/Vel5L8+1/YnmOT9SOLi0FEfQ6QV256mflkOlMsnVvDS5KUO3oc7qef+WG0eauus66ArPFLIdj9W3MYmo6AC2dETztFVaxGyVWIjErV6joqTcmS7Qt9TzCWCV7D5avSsrQiIxhMq4mPucN2DWiNGCcFE=; Message-ID: <389605.70197.qm@web28503.mail.ukl.yahoo.com> X-YMail-OSG: c17Dvk8VM1lv4qUFS..9Ckjhy55YOg3LmRtdwwzRkghcVZts5sbhQphMMVuhbpZo3SpFByzVaYqP0Yqi_hVLKEcBx.N5riZkMY8SqneGlN.Wwj25VfVa7VmJ8vPRtwBSP2eRdih6mBJ9ODx6eAvHWkrkrvu2u_hj69DQ169tacdisJIJycJP_4VTfvdC5xq98NCtvvr0jBKjnd8uaosxkjaIPhbDBLJQwGWKHjB_ewtmyh48wFL634Oy9VX5f..K.7H1aPVd Received: from [78.53.192.36] by web28503.mail.ukl.yahoo.com via HTTP; Mon, 28 Sep 2009 10:19:08 PDT X-Mailer: YahooMailClassic/7.0.14 YahooMailWebService/0.7.347.3 Date: Mon, 28 Sep 2009 10:19:08 -0700 (PDT) From: Leunam Elebek To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Trouble with copyout, memcpy.... Plain-Text version =) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Sep 2009 17:19:09 -0000 Hey list, I currently code a driver under Current 8.0 for Current 8.0. But there are some problems with kernel/user-space interaction. I've the following structure: struct daq_kitinfo { =A0 =A0 =A0 =A0 uint32_t ki_maxdata; =A0 =A0 =A0 =A0 uint32_t ki_flags; =A0 =A0 =A0 =A0 uint32_t ki_rng_type; =A0 =A0 =A0 =A0 int=A0 =A0 =A0 ki_type; =A0 =A0 =A0 =A0 int=A0 =A0 =A0 ki_nchan; =A0 =A0 =A0 =A0 int=A0 =A0 =A0 ki_len_chanl; }; The above structure is used in my user-space app: int main(void) { =A0 =A0 =A0 =A0 struct daq_kitinfo *info; =A0 =A0 =A0 =A0 struct daq_kit kit;=20 =A0 =A0 =A0 =A0 int fd, size;=20 =A0 =A0 =A0 =A0 ... =A0 =A0 =A0 =A0 ... =A0 =A0 =A0 =A0 ... =A0 =A0 =A0 =A0 /*=20 =A0 =A0 =A0 =A0=A0=A0* At this point I'll try to alloc memory. Notice that= =20 =A0 =A0 =A0 =A0=A0=A0* the size i dependet from another struct entry. =A0 =A0 =A0 =A0=A0=A0*/ =A0 =A0 =A0 =A0 size =3D sizeof(*info) * kit.k_nkits; =A0 =A0 =A0 =A0 info =3D malloc(size); =A0 =A0 =A0 =A0 if (info =3D=3D NULL) =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 exit(ENOMEM); =A0 =A0 =A0 =A0 /* =A0 =A0 =A0 =A0=A0=A0* The next step is to call the drivers ioctl() interfa= ce =A0 =A0 =A0 =A0=A0=A0* (the reason for that is described below). =A0 =A0 =A0 =A0=A0=A0*/ =A0 =A0 =A0 =A0 if (ioctl(fd, DAQ_KITINFO, info)) { =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 printf("errno: %d\n", errno);=20 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0=A0exit(errno); =A0 =A0 =A0 =A0 } =A0 =A0 =A0 =A0=A0=A0printf("[ki_nchan] %d\n", info.ki_nchan); =A0 =A0 =A0 =A0=A0=A0... =A0 =A0 =A0 =A0=A0=A0... =A0 =A0 =A0 =A0=A0=A0return (0); } and inside the driver (put it simply): static int my_ioctl(struct cdev *dev, u_long cmd, caddr_t arg, int flags, =A0 =A0 struct thread *td) { =A0 =A0 =A0 =A0 struct daq_kitinfo *info; =A0 =A0 =A0 =A0 struct daq_kit =3D dev->si_drv1;=20 =A0 =A0 =A0 =A0 int size;=20 =A0 =A0 =A0 =A0 ... =A0 =A0 =A0 =A0 /* Do something useful e.g mutex'ing... */ =A0 =A0 =A0 =A0 ... =A0 =A0 =A0 =A0 /* The same as in user-space... */ =A0 =A0 =A0 =A0 size =3D sizeof(*info) * kit.k_nkits; =A0 =A0 =A0 =A0 info =3D malloc(sz, M_DAQ, M_NOWAIT | M_ZERO); =A0 =A0 =A0 =A0 if (info =3D=3D NULL) =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 .... =A0 =A0 =A0 =A0 /* =A0 =A0 =A0 =A0=A0=A0* Here I want to copy struct info from kernel to user-= space. =A0 =A0 =A0 =A0=A0=A0* If i use memcpy, the result is that the system hangs =A0 =A0 =A0 =A0=A0=A0* and I need to reboot the machine. OK, I thought =A0 =A0 =A0 =A0=A0=A0* copyout() should be able to do the job for me... =A0 =A0 =A0 =A0=A0=A0*/ =A0 =A0 =A0 =A0=A0=A0if (copyout(info, arg, sz)) =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=A0=A0/* =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * Fuc[k-k] i still come inside this blo= ck. I always =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * get an EFAULT error.=20 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ } I really don't know what I should do to let the driver working properly. The driver should grap some informations/attributes, and fill up the info structure, so we can copy the filled info struct to the user's app. I hope somebody can help me to resolve that problem. Ah, the corresponding ioctl is: #define GRP=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 'd' #define DAQ_KITINFO=A0 =A0 =A0 =A0 _IOR(GRP, 3, struct daq_kitinfo) Thanks for attention and greatz from germany MG=0A=0A=0A From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 28 18:57:35 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD21F106568B for ; Mon, 28 Sep 2009 18:57:35 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 9CA098FC20 for ; Mon, 28 Sep 2009 18:57:35 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id CB8716D41C; Mon, 28 Sep 2009 18:57:33 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 98B7A84529; Mon, 28 Sep 2009 20:57:33 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Leunam Elebek References: <389605.70197.qm@web28503.mail.ukl.yahoo.com> Date: Mon, 28 Sep 2009 20:57:33 +0200 In-Reply-To: <389605.70197.qm@web28503.mail.ukl.yahoo.com> (Leunam Elebek's message of "Mon, 28 Sep 2009 10:19:08 -0700 (PDT)") Message-ID: <86ws3i3nj6.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Trouble with copyout, memcpy.... Plain-Text version =) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Sep 2009 18:57:35 -0000 Leunam Elebek writes: > /* The same as in user-space... */ > size =3D sizeof(*info) * kit.k_nkits; > info =3D malloc(sz, M_DAQ, M_NOWAIT | M_ZERO); You shouldn't use M_NOWAIT unless there is absolutely no way around it. > if (info =3D=3D NULL) > .... Unnecessary if you use M_WAITOK instead of M_NOWAIT. > /* > * Here I want to copy struct info from kernel to user-space. > * If i use memcpy, the result is that the system hangs > * and I need to reboot the machine. OK, I thought > * copyout() should be able to do the job for me... > */ > if (copyout(info, arg, sz)) Nope, ioctl() takes care of the copyin() / copyout(). At this point, arg is a pointer to a malloc()ed buffer of the right size (as specified by the definition of DAQ_KITINFO). > /* > * Fuc[k-k] i still come inside this block. I always > * get an EFAULT error.=20 > */ This means that either a) info doesn't point where you think it does, b) arg doesn't point where you think it does, or c) sz doesn't have the value you think it does. In this case, it's a combination of the latter two: arg points to a kernel buffer, so the use of copyout(9) is inappropriate, but in addition, the size of that buffer is sizeof(daq_kitinfo), and you're trying to copy far more. You need to rethink your interface: either return only one struct daq_kitinfo per ioctl() call, or pass in a struct that contains a pointer to a userland buffer and a length, or use something else than ioctl(2). option 2 would be something like: struct daq_ioctl { struct daq_kitinfo *info; int nkits; }; #define GRP 'd' #define DAQ_KITINFO _IOWR(GRP, 3, struct daq_ioctl) static int my_ioctl(struct cdev *dev, u_long cmd, caddr_t arg, int flags, struct threa= d *td) { struct daq_ioctl *di =3D (struct daq_ioctl *)arg; struct daq_kitinfo *info; struct daq_kit kit;=20 int nkits, ret; /* ... */ nkits =3D (kit.k_nkits > di->nkits) ? di->nkits : kit.k_nkits; info =3D malloc(nkits * sizeof(struct daq_kitinfo)) /* ... */ ret =3D copyout(info, di->info, nkits * sizeof(struct daq_kitinfo)); /* let userland know what it got */ if (ret =3D=3D 0) di->nkits =3D nkits; return (ret); } DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 28 19:12:45 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 52E7D1065670 for ; Mon, 28 Sep 2009 19:12:45 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from tomjudge.vm.bytemark.co.uk (tomjudge.vm.bytemark.co.uk [80.68.91.100]) by mx1.freebsd.org (Postfix) with ESMTP id D3C148FC16 for ; Mon, 28 Sep 2009 19:12:44 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by tomjudge.vm.bytemark.co.uk (Postfix) with ESMTP id 85DE14896C for ; Mon, 28 Sep 2009 19:56:10 +0100 (BST) X-Virus-Scanned: Debian amavisd-new at tomjudge.vm.bytemark.co.uk Received: from tomjudge.vm.bytemark.co.uk ([127.0.0.1]) by localhost (tomjudge.vm.bytemark.co.uk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OtAhxY7xWwFO for ; Mon, 28 Sep 2009 19:56:01 +0100 (BST) Received: from rita.nodomain (unknown [192.168.205.6]) by tomjudge.vm.bytemark.co.uk (Postfix) with ESMTP id 1BA154894F for ; Mon, 28 Sep 2009 19:56:00 +0100 (BST) Message-ID: <4AC106AA.9000305@tomjudge.com> Date: Mon, 28 Sep 2009 18:55:38 +0000 From: Tom Judge User-Agent: Thunderbird 2.0.0.23 (X11/20090822) MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Help debugging: Fatal kernel mode data abort: 'External Linefetch Abort (P)' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Sep 2009 19:12:45 -0000 Hi, I am working on getting FreeBSD to boot on a new ARM based board, and am hitting this issue any time I load a driver for the PCI based devices on the board. My current code can be found here: http://www.tomjudge.com/tmp/em7210.patch Here is the back trace of the problem (which i can repeat with em and ohci drivers): RedBoot> load -b 0x01008000 kernel Using default protocol (TFTP) Address offset = 0x40000000 Entry point: 0x01008100, address range: 0x01008000-0x01349e28 RedBoot> go KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #12: Sat Sep 26 05:00:06 UTC 2009 root@rita.nodomain:/data/arm_build/arm/usr/src/sys/EM-7210 CPU: i80219 400MHz step A-0 (XScale core) DC enabled IC enabled WB enabled LABT branch prediction enabled 32KB/32B 32-way Instruction cache 32KB/32B 32-way write-back-locking Data cache real memory = 536870912 (512 MB) avail memory = 503738368 (480 MB) iq0: on motherboard obio0 on iq0 uart0: <16550 or compatible> on obio0 uart0: [FILTER] uart0: console (115200,n,8,1) itimer0: on iq0 iopwdog0: on iq0 pcib0: on iq0 pci0: on pcib0 No mapping for 0/5/0/ No mapping for 0/5/1/ No mapping for 0/5/2/ pci0: at device 1.0 (no driver attached) pci0: at device 2.0 (no driver attached) atapci0: mem 0x80000-0x80fff irq 27 at device 3.0 on pci0 atapci0: [ITHREAD] ata2: on atapci0 Fatal kernel mode data abort: 'External Linefetch Abort (P)' trapframe: 0xc00faaf0 FSR=00000406, FAR=Invalid, spsr=600000d3 r0 =c13e2c00, r1 =cd5bc000, r2 =00000004, r3 =c13e2d7c r4 =c13e2c00, r5 =cd5bc000, r6 =c1298290, r7 =c1388800 r8 =ffffffff, r9 =00000009, r10=c13d8c00, r11=c00fab58 r12=c00fab24, ssp=c00fab3c, slr=c102389c, pc =c1023898 [thread pid 0 tid 100000 ] Stopped at ata_ahci_chipinit+0x4d68: ldr r15, [r3, #0x024] db> bt Tracing pid 0 tid 100000 td 0xc134dca0 db_trace_thread() at db_trace_thread+0xc scp=0xc129c68c rlv=0xc100d0f0 (db_command_init+0x2a8) rsp=0xc00fa7ec rfp=0xc00fa808 r10=0x00000001 r9=0xc13537f4 r8=0xc134b0c4 r7=0x00000062 r6=0x00000002 r5=0x00000010 r4=0xc134dca0 db_command_init() at db_command_init+0x1d0 scp=0xc100d018 rlv=0xc100cba0 (db_skip_to_eol+0x49c) rsp=0xc00fa80c rfp=0xc00fa8b0 r5=0x00000000 r4=0xc1327938 db_skip_to_eol() at db_skip_to_eol+0x1d0 scp=0xc100c8d4 rlv=0xc100cd0c (db_command_loop+0x60) rsp=0xc00fa8b4 rfp=0xc00fa8c0 r10=0x00000000 r8=0x00000406 r7=0xc00faaf0 r6=0xc13537f0 r5=0x600000d3 r4=0xc00fa8cc db_command_loop() at db_command_loop+0xc scp=0xc100ccb8 rlv=0xc100f050 (X_db_sym_numargs+0xf4) rsp=0xc00fa8c4 rfp=0xc00fa9e0 X_db_sym_numargs() at X_db_sym_numargs+0x14 scp=0xc100ef70 rlv=0xc1106e40 (kdb_trap+0xa4) rsp=0xc00fa9e4 rfp=0xc00faa0c r4=0x000000c0 kdb_trap() at kdb_trap+0xc scp=0xc1106da8 rlv=0xc12acb44 (badaddr_read+0x280) rsp=0xc00faa10 rfp=0xc00faa2c r10=0x00000000 r9=0x00000009 r8=0xc00faaf0 r7=0x00000406 r6=0x00000000 r5=0x00000406 r4=0xc00faaf0 badaddr_read() at badaddr_read+0xfc scp=0xc12ac9c0 rlv=0xc12acfdc (prefetch_abort_handler+0x440) rsp=0xc00faa30 rfp=0xc00faa50 r6=0xc134dca0 r5=0xc00faef8 r4=0xc00faaf0 prefetch_abort_handler() at prefetch_abort_handler+0x378 scp=0xc12acf14 rlv=0xc12ad1a8 (data_abort_handler+0x110) rsp=0xc00faa54 rfp=0xc00faaec r7=0xc134dca0 r6=0xc1298290 r5=0xc00faef8 r4=0xc134d9d8 data_abort_handler() at data_abort_handler+0xc scp=0xc12ad0a4 rlv=0xc129e0c8 (address_exception_entry+0x50) rsp=0xc00faaf0 rfp=0xc00fab58 r10=0xc13d8c00 r9=0x00000009 r8=0xffffffff r7=0xc1388800 r6=0xc1298290 r5=0xffff1004 r4=0xc13e2c00 ata_ahci_chipinit() at ata_ahci_chipinit+0x4c44 scp=0xc1023774 rlv=0xc101b664 (ata_mode2idx+0x464) rsp=0xc00fab5c rfp=0xc00fab78 r7=0xc1391900 r6=0xc13d8c00 r5=0xc1388800 r4=0xc13a01b0 ata_mode2idx() at ata_mode2idx+0x3ec scp=0xc101b5ec rlv=0xc1101a98 (device_attach+0x2c8) rsp=0xc00fab7c rfp=0xc00fabb8 r7=0xc1100140 r6=0xc13d8c4c r5=0x80000000 r4=0xc1383080 device_attach() at device_attach+0xc scp=0xc11017dc rlv=0xc1102e5c (device_probe_and_attach+0x34) rsp=0xc00fabbc rfp=0xc00fabcc r10=0xc1383080 r8=0xffffffff r7=0xc1100140 r6=0xc1383080 r5=0xc1391900 r4=0xc13d8c00 device_probe_and_attach() at device_probe_and_attach+0xc scp=0xc1102e34 rlv=0xc1102e80 (bus_generic_attach+0x20) rsp=0xc00fabd0 rfp=0xc00fabe0 r4=0xc13d8c00 bus_generic_attach() at bus_generic_attach+0xc scp=0xc1102e6c rlv=0xc101d11c (ata_pci_attach+0x2a4) rsp=0xc00fabe4 rfp=0xc00fac0c r4=0x00000004 ata_pci_attach() at ata_pci_attach+0xc scp=0xc101ce84 rlv=0xc1101a98 (device_attach+0x2c8) rsp=0xc00fac10 rfp=0xc00fac4c r7=0xc1100140 r6=0xc13830cc r5=0x80000000 r4=0xc1383200 device_attach() at device_attach+0xc scp=0xc11017dc rlv=0xc1102e5c (device_probe_and_attach+0x34) rsp=0xc00fac50 rfp=0xc00fac60 r10=0xc1383200 r8=0xffffffff r7=0xc1100140 r6=0x00000000 r5=0x00000000 r4=0xc1383080 device_probe_and_attach() at device_probe_and_attach+0xc scp=0xc1102e34 rlv=0xc1102e80 (bus_generic_attach+0x20) rsp=0xc00fac64 rfp=0xc00fac74 r4=0xc1383080 bus_generic_attach() at bus_generic_attach+0xc scp=0xc1102e6c rlv=0xc1067094 (pci_add_children+0x240) rsp=0xc00fac78 rfp=0xc00fac98 r4=0xc1383200 pci_add_children() at pci_add_children+0x154 scp=0xc1066fa8 rlv=0xc1101a98 (device_attach+0x2c8) rsp=0xc00fac9c rfp=0xc00facd8 r6=0xc138324c r5=0x80000000 r4=0xc13d9300 device_attach() at device_attach+0xc scp=0xc11017dc rlv=0xc1102e5c (device_probe_and_attach+0x34) rsp=0xc00facdc rfp=0xc00facec r10=0xc13d9300 r8=0xffffffff r7=0x00000000 r6=0xc13d9300 r5=0xc13bec18 r4=0xc1383200 device_probe_and_attach() at device_probe_and_attach+0xc scp=0xc1102e34 rlv=0xc1102e80 (bus_generic_attach+0x20) rsp=0xc00facf0 rfp=0xc00fad00 r4=0xc1383200 bus_generic_attach() at bus_generic_attach+0xc scp=0xc1102e6c rlv=0xc12b25a4 (i80321_sdram_bounds+0x860) rsp=0xc00fad04 rfp=0xc00fad20 r4=0xc13bec60 i80321_sdram_bounds() at i80321_sdram_bounds+0x6f4 scp=0xc12b2438 rlv=0xc1101a98 (device_attach+0x2c8) rsp=0xc00fad24 rfp=0xc00fad60 r7=0xc1100140 r6=0xc13d934c r5=0x80000000 r4=0xc13d9580 device_attach() at device_attach+0xc scp=0xc11017dc rlv=0xc1102e5c (device_probe_and_attach+0x34) rsp=0xc00fad64 rfp=0xc00fad74 r10=0xc13d9580 r8=0xc13d9580 r7=0x40000004 r6=0xc13e2c00 r5=0x00000000 r4=0xc13d9300 device_probe_and_attach() at device_probe_and_attach+0xc scp=0xc1102e34 rlv=0xc1102e80 (bus_generic_attach+0x20) rsp=0xc00fad78 rfp=0xc00fad88 r4=0xc13d9300 bus_generic_attach() at bus_generic_attach+0xc scp=0xc1102e6c rlv=0xc12b31e0 (iq80321_attach+0x370) rsp=0xc00fad8c rfp=0xc00fadb8 r4=0xc12eeca8 iq80321_attach() at iq80321_attach+0xc scp=0xc12b2e7c rlv=0xc1101a98 (device_attach+0x2c8) rsp=0xc00fadbc rfp=0xc00fadf8 r8=0xffffffff r7=0xc1100140 r6=0xc13d95cc r5=0x80000000 r4=0xc13d9680 device_attach() at device_attach+0xc scp=0xc11017dc rlv=0xc1102e5c (device_probe_and_attach+0x34) rsp=0xc00fadfc rfp=0xc00fae0c r10=0xc13d9680 r8=0xffffffff r7=0xc1100140 r6=0xc13d96cc r5=0x80000000 r4=0xc13d9580 device_probe_and_attach() at device_probe_and_attach+0xc scp=0xc1102e34 rlv=0xc1102e80 (bus_generic_attach+0x20) rsp=0xc00fae10 rfp=0xc00fae20 r4=0xc13d9580 bus_generic_attach() at bus_generic_attach+0xc scp=0xc1102e6c rlv=0xc12a0fec (minidumpsys+0xaf4) rsp=0xc00fae24 rfp=0xc00fae34 r4=0xc13d9680 minidumpsys() at minidumpsys+0xae4 scp=0xc12a0fdc rlv=0xc1101a98 (device_attach+0x2c8) rsp=0xc00fae38 rfp=0xc00fae74 r4=0xc12c30b8 device_attach() at device_attach+0xc scp=0xc11017dc rlv=0xc1102e5c (device_probe_and_attach+0x34) rsp=0xc00fae78 rfp=0xc00fae88 r10=0x0000000a r8=0x00000000 r7=0xa10081a4 r6=0xc13d9b80 r5=0xc1347e7c r4=0xc13d9680 device_probe_and_attach() at device_probe_and_attach+0xc scp=0xc1102e34 rlv=0xc1103140 (bus_generic_new_pass+0xe4) rsp=0xc00fae8c rfp=0xc00faea4 r4=0xc13d9680 bus_generic_new_pass() at bus_generic_new_pass+0xc scp=0xc1103068 rlv=0xc10ff13c (bus_set_pass+0x98) rsp=0xc00faea8 rfp=0xc00faec0 r6=0x7fffffff r5=0xc13d9b80 r4=0xc13f16c0 bus_set_pass() at bus_set_pass+0xc scp=0xc10ff0b0 rlv=0xc10ff184 (root_bus_configure+0x14) rsp=0xc00faec4 rfp=0xc00faed0 r6=0x00000006 r5=0xa10081b0 r4=0xc12f0cb4 root_bus_configure() at root_bus_configure+0xc scp=0xc10ff17c rlv=0xc129708c (xdr_sizeof+0x1d0) rsp=0xc00faed4 rfp=0xc00faee0 xdr_sizeof() at xdr_sizeof+0x1cc scp=0xc1297088 rlv=0xc108e39c (mi_startup+0xdc) rsp=0xc00faee4 rfp=0xc00faef4 mi_startup() at mi_startup+0xc scp=0xc108e2cc rlv=0xc1008248 (btext+0x148) rsp=0xc00faef8 rfp=0x00000000 r4=0xa1008288 Thanks Tom From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 28 20:48:35 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2077F1065676 for ; Mon, 28 Sep 2009 20:48:35 +0000 (UTC) (envelope-from mlfbsd@kanar.ci0.org) Received: from kanar.ci0.org (kanar.ci0.org [88.191.50.96]) by mx1.freebsd.org (Postfix) with ESMTP id AE9268FC12 for ; Mon, 28 Sep 2009 20:48:34 +0000 (UTC) Received: from kanar.ci0.org (pluxor@localhost [127.0.0.1]) by kanar.ci0.org (8.14.2/8.14.3) with ESMTP id n8SKLWut015395; Mon, 28 Sep 2009 22:21:32 +0200 (CEST) (envelope-from mlfbsd@kanar.ci0.org) Received: (from mlfbsd@localhost) by kanar.ci0.org (8.14.2/8.14.3/Submit) id n8SKLWof015394; Mon, 28 Sep 2009 22:21:32 +0200 (CEST) (envelope-from mlfbsd) Date: Mon, 28 Sep 2009 22:21:32 +0200 From: Olivier Houchard To: Tom Judge Message-ID: <20090928202132.GA15236@ci0.org> References: <4AC106AA.9000305@tomjudge.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4AC106AA.9000305@tomjudge.com> User-Agent: Mutt/1.4.2.1i X-Mailman-Approved-At: Mon, 28 Sep 2009 20:51:18 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: Help debugging: Fatal kernel mode data abort: 'External Linefetch Abort (P)' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Sep 2009 20:48:35 -0000 On Mon, Sep 28, 2009 at 06:55:38PM +0000, Tom Judge wrote: > Hi, > > I am working on getting FreeBSD to boot on a new ARM based board, and am > hitting this issue any time I load a driver for the PCI based devices on > the board. > > My current code can be found here: > > http://www.tomjudge.com/tmp/em7210.patch > Hi Tom, My guess is, you should include std.i80219 instead of std.i80321 in std.em7210. If you do not, CPU_XSCALE_80219 won't be defined, and the 80321 code to check if the board is host or not will be used, and will wrongly assume it is not, and thus won't map the PCI mem correctly. Regards, Olivier From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 02:05:59 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4EB0B106566C for ; Tue, 29 Sep 2009 02:05:59 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from tomjudge.vm.bytemark.co.uk (tomjudge.vm.bytemark.co.uk [80.68.91.100]) by mx1.freebsd.org (Postfix) with ESMTP id 1285B8FC0A for ; Tue, 29 Sep 2009 02:05:58 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by tomjudge.vm.bytemark.co.uk (Postfix) with ESMTP id 1636FDCF89; Tue, 29 Sep 2009 03:05:58 +0100 (BST) X-Virus-Scanned: Debian amavisd-new at tomjudge.vm.bytemark.co.uk Received: from tomjudge.vm.bytemark.co.uk ([127.0.0.1]) by localhost (tomjudge.vm.bytemark.co.uk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kmcU1gI1XhTC; Tue, 29 Sep 2009 03:05:48 +0100 (BST) Received: from rita.nodomain (unknown [192.168.205.6]) by tomjudge.vm.bytemark.co.uk (Postfix) with ESMTP id 4258548992; Tue, 29 Sep 2009 03:05:39 +0100 (BST) Message-ID: <4AC16B5A.8090407@tomjudge.com> Date: Tue, 29 Sep 2009 02:05:14 +0000 From: Tom Judge User-Agent: Thunderbird 2.0.0.23 (X11/20090822) MIME-Version: 1.0 To: Olivier Houchard References: <4AC106AA.9000305@tomjudge.com> <20090928202132.GA15236@ci0.org> In-Reply-To: <20090928202132.GA15236@ci0.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: Help debugging: Fatal kernel mode data abort: 'External Linefetch Abort (P)' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 02:05:59 -0000 Olivier Houchard wrote: > On Mon, Sep 28, 2009 at 06:55:38PM +0000, Tom Judge wrote: > >> Hi, >> >> I am working on getting FreeBSD to boot on a new ARM based board, and am >> hitting this issue any time I load a driver for the PCI based devices on >> the board. >> >> My current code can be found here: >> >> http://www.tomjudge.com/tmp/em7210.patch >> >> > > Hi Tom, > > My guess is, you should include std.i80219 instead of std.i80321 in std.em7210. > If you do not, CPU_XSCALE_80219 won't be defined, and the 80321 code to > check if the board is host or not will be used, and will wrongly assume > it is not, and thus won't map the PCI mem correctly. > > Hi Olivier, I have switched out the std file and am now using std.i80219 but am still having issues. I think the problems are the pci memory mappings in the controller devices. On linux em0 gets mapped as follows: cd 0000\:00\:01.0/ # ls class device local_cpus subsystem_device config driver resource subsystem_vendor detach_state irq rom vendor # cat resource 0x0000000080000000 0x000000008001ffff 0x0000000000000200 0x0000000080020000 0x000000008003ffff 0x0000000000000200 0x00000000fe000000 0x00000000fe00003f 0x0000000000000101 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000080040000 0x000000008005ffff 0x0000000000007200 # Where as on FreeBSD I am seeing this: em0: port 0xfe400000-0xfe40003f mem 0-0x1ffff,0x20000-0x3ffff irq 29 at device 1.0 on pci0 Seems that I am missing the 0x800 off the front of the PCI memory mappings. I have confirmed this with the ata driver also and see the same issues. Where should I be looking to fix this? Thanks Tom From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 02:46:12 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D31E41065670 for ; Tue, 29 Sep 2009 02:46:12 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from tomjudge.vm.bytemark.co.uk (tomjudge.vm.bytemark.co.uk [80.68.91.100]) by mx1.freebsd.org (Postfix) with ESMTP id 95FE28FC0C for ; Tue, 29 Sep 2009 02:46:12 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by tomjudge.vm.bytemark.co.uk (Postfix) with ESMTP id E7A6C48997; Tue, 29 Sep 2009 03:46:11 +0100 (BST) X-Virus-Scanned: Debian amavisd-new at tomjudge.vm.bytemark.co.uk Received: from tomjudge.vm.bytemark.co.uk ([127.0.0.1]) by localhost (tomjudge.vm.bytemark.co.uk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id d7SQiQgZMX1Y; Tue, 29 Sep 2009 03:46:03 +0100 (BST) Received: from rita.nodomain (unknown [192.168.205.6]) by tomjudge.vm.bytemark.co.uk (Postfix) with ESMTP id EF54948995; Tue, 29 Sep 2009 03:46:01 +0100 (BST) Message-ID: <4AC174D1.8080900@tomjudge.com> Date: Tue, 29 Sep 2009 02:45:37 +0000 From: Tom Judge User-Agent: Thunderbird 2.0.0.23 (X11/20090822) MIME-Version: 1.0 To: Olivier Houchard References: <4AC106AA.9000305@tomjudge.com> <20090928202132.GA15236@ci0.org> <4AC16B5A.8090407@tomjudge.com> In-Reply-To: <4AC16B5A.8090407@tomjudge.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: Help debugging: Fatal kernel mode data abort: 'External Linefetch Abort (P)' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 02:46:13 -0000 Tom Judge wrote: > Olivier Houchard wrote: >> On Mon, Sep 28, 2009 at 06:55:38PM +0000, Tom Judge wrote: >> >>> Hi, >>> >>> I am working on getting FreeBSD to boot on a new ARM based board, >>> and am hitting this issue any time I load a driver for the PCI based >>> devices on the board. >>> >>> My current code can be found here: >>> >>> http://www.tomjudge.com/tmp/em7210.patch >>> >>> >> >> Hi Tom, >> >> My guess is, you should include std.i80219 instead of std.i80321 in >> std.em7210. >> If you do not, CPU_XSCALE_80219 won't be defined, and the 80321 code to >> check if the board is host or not will be used, and will wrongly >> assume it is not, and thus won't map the PCI mem correctly. >> >> > Hi Olivier, > > I have switched out the std file and am now using std.i80219 but am > still having issues. > > I think the problems are the pci memory mappings in the controller > devices. > > On linux em0 gets mapped as follows: > > cd 0000\:00\:01.0/ > # ls > class device local_cpus subsystem_device > config driver resource subsystem_vendor > detach_state irq rom vendor > # cat resource > 0x0000000080000000 0x000000008001ffff 0x0000000000000200 > 0x0000000080020000 0x000000008003ffff 0x0000000000000200 > 0x00000000fe000000 0x00000000fe00003f 0x0000000000000101 > 0x0000000000000000 0x0000000000000000 0x0000000000000000 > 0x0000000000000000 0x0000000000000000 0x0000000000000000 > 0x0000000000000000 0x0000000000000000 0x0000000000000000 > 0x0000000080040000 0x000000008005ffff 0x0000000000007200 > # > > > > Where as on FreeBSD I am seeing this: > em0: port > 0xfe400000-0xfe40003f mem 0-0x1ffff,0x20000-0x3ffff irq 29 at device > 1.0 on pci0 > > Seems that I am missing the 0x800 off the front of the PCI memory > mappings. > > I have confirmed this with the ata driver also and see the same issues. > > Where should I be looking to fix this? > Forgot to include the output from VERBOSE_INIT_ARM iq0: on motherboard i80321: BAR0 = 20000004.00000000 BAR1 = 40000004.00000000 i80219: BAR0 = 20000000.00000000 BAR1 = 40000000.00000000 i80321: SBDR = 0xa0000000 SBR0 = 0x00000018 SBR1 = 0x00000020 i80321: BANK0 = 0x10000000 BANK1 = 0x10000000 i80321: Reserve space for private devices (Inbound Window 1) hi:0x00000000 lo:0x8000000c xlate:0x80000000 size:0x04000000 i80321: RAM access (Inbound Window 2) hi:0x00000000 lo:0xa000000c xlate:0xa0000000 size:0x20000000 obio0 on iq0 uart0: <16550 or compatible> on obio0 From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 09:38:01 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1017F1065670 for ; Tue, 29 Sep 2009 09:38:01 +0000 (UTC) (envelope-from mlfbsd@kanar.ci0.org) Received: from kanar.ci0.org (kanar.ci0.org [88.191.50.96]) by mx1.freebsd.org (Postfix) with ESMTP id B425A8FC17 for ; Tue, 29 Sep 2009 09:38:00 +0000 (UTC) Received: from kanar.ci0.org (pluxor@localhost [127.0.0.1]) by kanar.ci0.org (8.14.2/8.14.3) with ESMTP id n8T9cPag026721; Tue, 29 Sep 2009 11:38:25 +0200 (CEST) (envelope-from mlfbsd@kanar.ci0.org) Received: (from mlfbsd@localhost) by kanar.ci0.org (8.14.2/8.14.3/Submit) id n8T9cP7k026720; Tue, 29 Sep 2009 11:38:25 +0200 (CEST) (envelope-from mlfbsd) Date: Tue, 29 Sep 2009 11:38:25 +0200 From: Olivier Houchard To: Tom Judge Message-ID: <20090929093825.GA26424@ci0.org> References: <4AC106AA.9000305@tomjudge.com> <20090928202132.GA15236@ci0.org> <4AC16B5A.8090407@tomjudge.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="qDbXVdCdHGoSgWSk" Content-Disposition: inline In-Reply-To: <4AC16B5A.8090407@tomjudge.com> User-Agent: Mutt/1.4.2.1i X-Mailman-Approved-At: Tue, 29 Sep 2009 11:36:53 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: Help debugging: Fatal kernel mode data abort: 'External Linefetch Abort (P)' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 09:38:01 -0000 --qDbXVdCdHGoSgWSk Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Sep 29, 2009 at 02:05:14AM +0000, Tom Judge wrote: > Olivier Houchard wrote: > >On Mon, Sep 28, 2009 at 06:55:38PM +0000, Tom Judge wrote: > > > >>Hi, > >> > >>I am working on getting FreeBSD to boot on a new ARM based board, and am > >>hitting this issue any time I load a driver for the PCI based devices on > >>the board. > >> > >>My current code can be found here: > >> > >>http://www.tomjudge.com/tmp/em7210.patch > >> > >> > > > >Hi Tom, > > > >My guess is, you should include std.i80219 instead of std.i80321 in > >std.em7210. > >If you do not, CPU_XSCALE_80219 won't be defined, and the 80321 code to > >check if the board is host or not will be used, and will wrongly assume > >it is not, and thus won't map the PCI mem correctly. > > > > > Hi Olivier, > > I have switched out the std file and am now using std.i80219 but am > still having issues. > > I think the problems are the pci memory mappings in the controller devices. > > On linux em0 gets mapped as follows: > > cd 0000\:00\:01.0/ > # ls > class device local_cpus subsystem_device > config driver resource subsystem_vendor > detach_state irq rom vendor > # cat resource > 0x0000000080000000 0x000000008001ffff 0x0000000000000200 > 0x0000000080020000 0x000000008003ffff 0x0000000000000200 > 0x00000000fe000000 0x00000000fe00003f 0x0000000000000101 > 0x0000000000000000 0x0000000000000000 0x0000000000000000 > 0x0000000000000000 0x0000000000000000 0x0000000000000000 > 0x0000000000000000 0x0000000000000000 0x0000000000000000 > 0x0000000080040000 0x000000008005ffff 0x0000000000007200 > # > > > > Where as on FreeBSD I am seeing this: > em0: port > 0xfe400000-0xfe40003f mem 0-0x1ffff,0x20000-0x3ffff irq 29 at device 1.0 > on pci0 > > Seems that I am missing the 0x800 off the front of the PCI memory mappings. > Ok I'm a bit confused about this code, it's been too long since I haven't read it :) Could you try the attached patch ? Thanks ! If it doesn't help, you can print adapter->osdep.mem_bus_space_handle in if_em.c to make sure it is the same as in linux. Regards, Olivier --qDbXVdCdHGoSgWSk Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="i80321_pci.c.diff" Index: arm/xscale/i80321/i80321_pci.c =================================================================== --- arm/xscale/i80321/i80321_pci.c (revision 196158) +++ arm/xscale/i80321/i80321_pci.c (working copy) @@ -92,8 +92,7 @@ sc->sc_busno = busno; sc->sc_pciio = &i80321_softc->sc_pci_iot; sc->sc_pcimem = &i80321_softc->sc_pci_memt; - sc->sc_mem = i80321_softc->sc_owin[0].owin_xlate_lo + - VERDE_OUT_XLATE_MEM_WIN_SIZE; + sc->sc_mem = i80321_softc->sc_owin[0].owin_xlate_lo; sc->sc_io = i80321_softc->sc_iow_vaddr; /* Initialize memory and i/o rmans. */ --qDbXVdCdHGoSgWSk-- From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 14:53:29 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 910781065694 for ; Tue, 29 Sep 2009 14:53:29 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from tomjudge.vm.bytemark.co.uk (tomjudge.vm.bytemark.co.uk [80.68.91.100]) by mx1.freebsd.org (Postfix) with ESMTP id 3FC768FC24 for ; Tue, 29 Sep 2009 14:53:29 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by tomjudge.vm.bytemark.co.uk (Postfix) with ESMTP id EEC86489B3; Tue, 29 Sep 2009 15:53:27 +0100 (BST) X-Virus-Scanned: Debian amavisd-new at tomjudge.vm.bytemark.co.uk Received: from tomjudge.vm.bytemark.co.uk ([127.0.0.1]) by localhost (tomjudge.vm.bytemark.co.uk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9GyjLu3s35SY; Tue, 29 Sep 2009 15:53:17 +0100 (BST) Received: from rita.nodomain (unknown [192.168.205.6]) by tomjudge.vm.bytemark.co.uk (Postfix) with ESMTP id 1B68F489AE; Tue, 29 Sep 2009 15:53:16 +0100 (BST) Message-ID: <4AC21F44.6060004@tomjudge.com> Date: Tue, 29 Sep 2009 14:52:52 +0000 From: Tom Judge User-Agent: Thunderbird 2.0.0.23 (X11/20090822) MIME-Version: 1.0 To: Olivier Houchard References: <4AC106AA.9000305@tomjudge.com> <20090928202132.GA15236@ci0.org> <4AC16B5A.8090407@tomjudge.com> <20090929093825.GA26424@ci0.org> In-Reply-To: <20090929093825.GA26424@ci0.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: Help debugging: Fatal kernel mode data abort: 'External Linefetch Abort (P)' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 14:53:29 -0000 Olivier Houchard wrote: > On Tue, Sep 29, 2009 at 02:05:14AM +0000, Tom Judge wrote: > >> Hi Olivier, >> >> I have switched out the std file and am now using std.i80219 but am >> still having issues. >> >> I think the problems are the pci memory mappings in the controller devices. >> >> On linux em0 gets mapped as follows: >> >> cd 0000\:00\:01.0/ >> # ls >> class device local_cpus subsystem_device >> config driver resource subsystem_vendor >> detach_state irq rom vendor >> # cat resource >> 0x0000000080000000 0x000000008001ffff 0x0000000000000200 >> 0x0000000080020000 0x000000008003ffff 0x0000000000000200 >> 0x00000000fe000000 0x00000000fe00003f 0x0000000000000101 >> 0x0000000000000000 0x0000000000000000 0x0000000000000000 >> 0x0000000000000000 0x0000000000000000 0x0000000000000000 >> 0x0000000000000000 0x0000000000000000 0x0000000000000000 >> 0x0000000080040000 0x000000008005ffff 0x0000000000007200 >> # >> >> >> >> Where as on FreeBSD I am seeing this: >> em0: port >> 0xfe400000-0xfe40003f mem 0-0x1ffff,0x20000-0x3ffff irq 29 at device 1.0 >> on pci0 >> >> Seems that I am missing the 0x800 off the front of the PCI memory mappings. >> >> > > Ok I'm a bit confused about this code, it's been too long since I haven't > read it :) > Could you try the attached patch ? > > Thanks ! > > If it doesn't help, you can print adapter->osdep.mem_bus_space_handle in > if_em.c to make sure it is the same as in linux. > > Hi Olivier, I have tried the patch and here are the boot results: i80321: BAR0 = 20000004.00000000 BAR1 = 40000004.00000000 i80219: BAR0 = 20000000.00000000 BAR1 = 40000000.00000000 i80219: I/O Processor, acting as PCI host i80321: SBDR = 0xa0000000 SBR0 = 0x00000018 SBR1 = 0x00000020 i80321: BANK0 = 0x10000000 BANK1 = 0x10000000 i80321: Reserve space for private devices (Inbound Window 1) hi:0x00000000 lo:0x8000000c xlate:0x80000000 size:0x04000000 i80321: RAM access (Inbound Window 2) hi:0x00000000 lo:0xa000000c xlate:0xa0000000 size:0x20000000 obio0 on iq0 uart0: <16550 or compatible> on obio0 uart0: [FILTER] uart0: console (115200,n,8,1) itimer0: on iq0 iopwdog0: on iq0 pcib0: on iq0 pci0: on pcib0 Device 1 routed to irq 27 Device 2 routed to irq 30 Device 3 routed to irq 29 Device 5 routed to irq 30 Device 5 routed to irq 29 Device 5 routed to irq 27 em0: port 0xfe400000-0xfe40003f mem 0-0x1ffff,0x20000-0x3ffff irq 27 at device 1.0 on pci0 em0: Start: 0x00000000 em0: End: 0x0001FFFF em0: Size: 0x00020000 Fatal kernel mode data abort: 'External Linefetch Abort (P)' trapframe: 0xc00faad0 FSR=00000406, FAR=Invalid, spsr=200000d3 r0 =c00d0400, r1 =cd5bf000, r2 =00000010, r3 =0000000a r4 =c317e008, r5 =cd5bf000, r6 =c00d0400, r7 =c130212c r8 =c317e008, r9 =c0071180, r10=c317e000, r11=c00fab40 r12=c00fab44, ssp=c00fab1c, slr=c106a96c, pc =c106a968 [thread pid 0 tid 100000 ] Stopped at e1000_init_script_state_82541+0x24c: blx r7 db> As you can see I added some debug to if_em.c as such: Index: sys/dev/e1000/if_em.c =================================================================== --- sys/dev/e1000/if_em.c (revision 197472) +++ sys/dev/e1000/if_em.c (working copy) @@ -2770,6 +2770,9 @@ rman_get_bustag(adapter->memory); adapter->osdep.mem_bus_space_handle = rman_get_bushandle(adapter->memory); + device_printf(dev,"Start: 0x%08lX\n", rman_get_start(adapter->memory)); + device_printf(dev,"End: 0x%08lX\n", rman_get_end(adapter->memory)); + device_printf(dev,"Size: 0x%08lX\n", rman_get_size(adapter->memory)); adapter->hw.hw_addr = (u8 *)&adapter->osdep.mem_bus_space_handle; /* Only older adapters use IO mapping */ But the memory mapping seems to be missing the most significant 0x8. Thanks Tom From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 16:09:04 2009 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B0A641065693; Tue, 29 Sep 2009 16:09:04 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-bw0-f227.google.com (mail-bw0-f227.google.com [209.85.218.227]) by mx1.freebsd.org (Postfix) with ESMTP id 0E4EB8FC1D; Tue, 29 Sep 2009 16:09:03 +0000 (UTC) Received: by bwz27 with SMTP id 27so3995680bwz.43 for ; Tue, 29 Sep 2009 09:09:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=wzrcKIMdiTDx5Kc9S75bUIsMwF9fQRfv1VHz2btbiTA=; b=lBvxBU2aQc1sPQHRdeaFb+/Kh3ohGWwpg9onHvOH22HFiBPHpr5EpANnoIOS49dNH7 XUGcduQ7kW6EHveqztwT1Mn49rxegQVgqprFSnsztGVz0fTNx157pDDLoARCu43Uvwq6 q8qR/TdRf4dQCmBBczp4F8lSEJGUuksLF2SYQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=KuVlIXsjbRgpyXi7/vk19C3D+5BCXPvDZaftLVz94rCC4R4KaUm7JWNjHgnZcQ+R9n MN0zKmhZGg9zIe8MTUdDzI07BNS+rOxQk3D9Gre1iuSSypak9xIrPELAUOQZWomgLJqe bXESVIfdwnwVcyz5vdHK848S+VzBVxk4todLs= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.144.70 with SMTP id y6mr1330962fau.12.1254238777691; Tue, 29 Sep 2009 08:39:37 -0700 (PDT) In-Reply-To: <20090924224935.GW473@gandalf.sssup.it> References: <20090924224935.GW473@gandalf.sssup.it> Date: Tue, 29 Sep 2009 17:39:37 +0200 X-Google-Sender-Auth: 07c424f4586d0684 Message-ID: <3bbf2fe10909290839w305c85c3t1532bd7733c39a6a@mail.gmail.com> From: Attilio Rao To: Fabio Checconi Content-Type: text/plain; charset=UTF-8 Cc: hackers@freebsd.org Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 16:09:04 -0000 2009/9/25 Fabio Checconi : > Hi all, > looking at sys/sx.h I have some troubles understanding this comment: > > * A note about memory barriers. Exclusive locks need to use the same > * memory barriers as mutexes: _acq when acquiring an exclusive lock > * and _rel when releasing an exclusive lock. On the other side, > * shared lock needs to use an _acq barrier when acquiring the lock > * but, since they don't update any locked data, no memory barrier is > * needed when releasing a shared lock. > > In particular, I'm not understanding what prevents the following sequence > from happening: > > CPU A CPU B > > sx_slock(&data->lock); > > sx_sunlock(&data->lock); > > /* reordered after the unlock > by the cpu */ > if (data->buffer) > sx_xlock(&data->lock); > free(data->buffer); > data->buffer = NULL; > sx_xunlock(&data->lock); > > a = *data->buffer; > > IOW, even if readers do not modify the data protected by the lock, > without a release barrier a memory access may leak past the unlock (as > the cpu won't notice any dependency between the unlock and the fetch, > feeling free to reorder them), thus potentially racing with an exclusive > writer accessing the data. > > On architectures where atomic ops serialize memory accesses this would > never happen, otherwise the sequence above seems possible; am I missing > something? I think your concerns are right, possibly we need this patch: http://www.freebsd.org/~attilio/sxrw_unlockb.diff However speaking with John we agreed possibly there is a more serious breakage. Possibly, memory barriers would also require to ensure the compiler to not reorder the operation, while right now, in FreeBSD, they just take care of the reordering from the architecture perspective. The only way I'm aware of GCC offers that is to clobber memory. I will provide a patch that address this soon, hoping that GCC will be smart enough to not overhead too much the memory clobbering but just try to understand what's our purpose and servers it (I will try to compare code generated before and after the patch at least for tier-1 architectures). Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 17:53:39 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A639B1065670 for ; Tue, 29 Sep 2009 17:53:39 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.186]) by mx1.freebsd.org (Postfix) with ESMTP id 4F8288FC0A for ; Tue, 29 Sep 2009 17:53:39 +0000 (UTC) Received: from vampire.homelinux.org (dslb-088-067-228-142.pools.arcor-ip.net [88.67.228.142]) by mrelayeu.kundenserver.de (node=mreu2) with ESMTP (Nemesis) id 0MSVpQ-1MkxgI13MC-00RrzR; Tue, 29 Sep 2009 19:53:34 +0200 Received: (qmail 15651 invoked from network); 29 Sep 2009 17:53:34 -0000 Received: from kvm.laiers.local (HELO kvm.localnet) (192.168.4.200) by router.laiers.local with SMTP; 29 Sep 2009 17:53:34 -0000 From: Max Laier Organization: FreeBSD To: freebsd-hackers@freebsd.org Date: Tue, 29 Sep 2009 19:53:36 +0200 User-Agent: KMail/1.12.1 (Linux/2.6.30-ARCH; KDE/4.3.1; x86_64; ; ) References: <20090924224935.GW473@gandalf.sssup.it> <3bbf2fe10909290839w305c85c3t1532bd7733c39a6a@mail.gmail.com> In-Reply-To: <3bbf2fe10909290839w305c85c3t1532bd7733c39a6a@mail.gmail.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <200909291953.36373.max@love2party.net> X-Provags-ID: V01U2FsdGVkX18qVRQYFf1PVC1qzVwMsFNnHgarA7fQV7fohnS K4J6zSLPdqGPUHBmCnLk0l2YiiEk13oEUFTfBvTpy7be+g4s83 Tf3vU72Qjs= Cc: Attilio Rao , Fabio Checconi Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 17:53:39 -0000 On Tuesday 29 September 2009 17:39:37 Attilio Rao wrote: > 2009/9/25 Fabio Checconi : > > Hi all, > > looking at sys/sx.h I have some troubles understanding this comment: > > > > * A note about memory barriers. Exclusive locks need to use the same > > * memory barriers as mutexes: _acq when acquiring an exclusive lock > > * and _rel when releasing an exclusive lock. On the other side, > > * shared lock needs to use an _acq barrier when acquiring the lock > > * but, since they don't update any locked data, no memory barrier is > > * needed when releasing a shared lock. > > > > In particular, I'm not understanding what prevents the following sequence > > from happening: > > > > CPU A CPU B > > > > sx_slock(&data->lock); > > > > sx_sunlock(&data->lock); > > > > /* reordered after the unlock > > by the cpu */ > > if (data->buffer) > > sx_xlock(&data->lock); > > free(data->buffer); > > data->buffer = NULL; > > sx_xunlock(&data->lock); > > > > a = *data->buffer; > > > > IOW, even if readers do not modify the data protected by the lock, > > without a release barrier a memory access may leak past the unlock (as > > the cpu won't notice any dependency between the unlock and the fetch, > > feeling free to reorder them), thus potentially racing with an exclusive > > writer accessing the data. > > > > On architectures where atomic ops serialize memory accesses this would > > never happen, otherwise the sequence above seems possible; am I missing > > something? > > I think your concerns are right, possibly we need this patch: > http://www.freebsd.org/~attilio/sxrw_unlockb.diff > > However speaking with John we agreed possibly there is a more serious > breakage. Possibly, memory barriers would also require to ensure the > compiler to not reorder the operation, while right now, in FreeBSD, they > just take care of the reordering from the architecture perspective. > The only way I'm aware of GCC offers that is to clobber memory. > I will provide a patch that address this soon, hoping that GCC will be > smart enough to not overhead too much the memory clobbering but just > try to understand what's our purpose and servers it (I will try to > compare code generated before and after the patch at least for tier-1 > architectures). Does GCC really reorder accesses to volatile objects? The C Standard seems to object: 5.1.2.3 - 2 Accessing a volatile object, modifying an object, modifying a file, or calling a function that does any of those operations are all side effects,11) which are changes in the state of the execution environment. Evaluation of an expression may produce side effects. At certain specified points in the execution sequence called sequence points, all side effects of previous evaluations shall be complete and no side effects of subsequent evaluations shall have taken place. (A summary of the sequence points is given in annex C.) I might be reading this wrong, of course. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 18:30:07 2009 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DBF741065679; Tue, 29 Sep 2009 18:30:07 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 95D368FC1F; Tue, 29 Sep 2009 18:30:07 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 115BB46B06; Tue, 29 Sep 2009 14:30:07 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 209858A01B; Tue, 29 Sep 2009 14:30:06 -0400 (EDT) From: John Baldwin To: Attilio Rao Date: Tue, 29 Sep 2009 14:25:45 -0400 User-Agent: KMail/1.9.7 References: <20090924224935.GW473@gandalf.sssup.it> <3bbf2fe10909290839w305c85c3t1532bd7733c39a6a@mail.gmail.com> In-Reply-To: <3bbf2fe10909290839w305c85c3t1532bd7733c39a6a@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200909291425.46134.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Tue, 29 Sep 2009 14:30:06 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: hackers@freebsd.org, Fabio Checconi Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 18:30:07 -0000 On Tuesday 29 September 2009 11:39:37 am Attilio Rao wrote: > 2009/9/25 Fabio Checconi : > > Hi all, > > looking at sys/sx.h I have some troubles understanding this comment: > > > > * A note about memory barriers. Exclusive locks need to use the same > > * memory barriers as mutexes: _acq when acquiring an exclusive lock > > * and _rel when releasing an exclusive lock. On the other side, > > * shared lock needs to use an _acq barrier when acquiring the lock > > * but, since they don't update any locked data, no memory barrier is > > * needed when releasing a shared lock. > > > > In particular, I'm not understanding what prevents the following sequence > > from happening: > > > > CPU A CPU B > > > > sx_slock(&data->lock); > > > > sx_sunlock(&data->lock); > > > > /* reordered after the unlock > > by the cpu */ > > if (data->buffer) > > sx_xlock(&data->lock); > > free(data->buffer); > > data->buffer = NULL; > > sx_xunlock(&data->lock); > > > > a = *data->buffer; > > > > IOW, even if readers do not modify the data protected by the lock, > > without a release barrier a memory access may leak past the unlock (as > > the cpu won't notice any dependency between the unlock and the fetch, > > feeling free to reorder them), thus potentially racing with an exclusive > > writer accessing the data. > > > > On architectures where atomic ops serialize memory accesses this would > > never happen, otherwise the sequence above seems possible; am I missing > > something? > > I think your concerns are right, possibly we need this patch: > http://www.freebsd.org/~attilio/sxrw_unlockb.diff Actually, since you are only worried about reads, I think this should be an "acq" barrier rather than a "rel". In some cases "acq" is cheaper, so we should prefer the cheapest barrier that provides what we need. You would still need to keep some language about the memory barriers since using "acq" for shared unlocking is different from exclusive unlocking. I can't recall why I thought this was ok originally, sadly my p4 logs didn't include the reasoning either. :-/ > However speaking with John we agreed possibly there is a more serious breakage. > Possibly, memory barriers would also require to ensure the compiler to > not reorder the operation, while right now, in FreeBSD, they just take > care of the reordering from the architecture perspective. > The only way I'm aware of GCC offers that is to clobber memory. > I will provide a patch that address this soon, hoping that GCC will be > smart enough to not overhead too much the memory clobbering but just > try to understand what's our purpose and servers it (I will try to > compare code generated before and after the patch at least for tier-1 > architectures). The memory clobber is quite heavyweight. It actually forces gcc to forget any cached memory items in registers and reload everything again. What I really want is just a barrier to tell GCC to not reorder things. If I read a value in the program before acquiring a lock it is in theory fine to keep that cached across the barrier. However, there isn't a way to do this sort of thing with GCC currently. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 19:15:42 2009 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 31135106566B; Tue, 29 Sep 2009 19:15:42 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-bw0-f210.google.com (mail-bw0-f210.google.com [209.85.218.210]) by mx1.freebsd.org (Postfix) with ESMTP id 598608FC36; Tue, 29 Sep 2009 19:15:41 +0000 (UTC) Received: by mail-bw0-f210.google.com with SMTP id 6so2267243bwz.37 for ; Tue, 29 Sep 2009 12:15:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=vqn7gWCxCd9ic9U1E6fckFGgXcCxbJDMfdPW0a6y8to=; b=cFWjTz4NbgZn+quPU9ufVLcoDgFcAlmDwpvGk8neatyqokW/JccIiES7sNABBQwjMV CsqCn3to3krGE5a3Kmv4pIiKXFXL8Sa9Eq39tWB1gYP/69VvldLu9kkuEEuh81GFVCUu LV3ZepubrlDtcFp4yhALeBchvXKsyrcEgLWow= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=XaIJvlEW41yd1zEa8txTYV3C+xwPLV4h9k5JBP58OvB2xwcCrQHq7YNZXjuz4Jf6Ln bapLUT/OOyX37He8vBEI1O8X7gX6m3C75/w4xxqToVbx1tdDOjZ9G//YUiCsMtyU81E+ LjHe4+Xbeex2+2I4U2Zh11/DsoiWCh8xUqmW0= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.60.68 with SMTP id o4mr1395543fah.102.1254251740833; Tue, 29 Sep 2009 12:15:40 -0700 (PDT) In-Reply-To: <200909291425.46134.jhb@freebsd.org> References: <20090924224935.GW473@gandalf.sssup.it> <3bbf2fe10909290839w305c85c3t1532bd7733c39a6a@mail.gmail.com> <200909291425.46134.jhb@freebsd.org> Date: Tue, 29 Sep 2009 21:15:40 +0200 X-Google-Sender-Auth: 5e2c84c90e02b721 Message-ID: <3bbf2fe10909291215i2bdd73aj13c1ac433152cab4@mail.gmail.com> From: Attilio Rao To: John Baldwin Content-Type: text/plain; charset=UTF-8 Cc: hackers@freebsd.org, Fabio Checconi Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 19:15:42 -0000 2009/9/29 John Baldwin : > On Tuesday 29 September 2009 11:39:37 am Attilio Rao wrote: >> 2009/9/25 Fabio Checconi : >> > Hi all, >> > looking at sys/sx.h I have some troubles understanding this comment: >> > >> > * A note about memory barriers. Exclusive locks need to use the same >> > * memory barriers as mutexes: _acq when acquiring an exclusive lock >> > * and _rel when releasing an exclusive lock. On the other side, >> > * shared lock needs to use an _acq barrier when acquiring the lock >> > * but, since they don't update any locked data, no memory barrier is >> > * needed when releasing a shared lock. >> > >> > In particular, I'm not understanding what prevents the following sequence >> > from happening: >> > >> > CPU A CPU B >> > >> > sx_slock(&data->lock); >> > >> > sx_sunlock(&data->lock); >> > >> > /* reordered after the unlock >> > by the cpu */ >> > if (data->buffer) >> > sx_xlock(&data->lock); >> > free(data->buffer); >> > data->buffer = NULL; >> > sx_xunlock(&data->lock); >> > >> > a = *data->buffer; >> > >> > IOW, even if readers do not modify the data protected by the lock, >> > without a release barrier a memory access may leak past the unlock (as >> > the cpu won't notice any dependency between the unlock and the fetch, >> > feeling free to reorder them), thus potentially racing with an exclusive >> > writer accessing the data. >> > >> > On architectures where atomic ops serialize memory accesses this would >> > never happen, otherwise the sequence above seems possible; am I missing >> > something? >> >> I think your concerns are right, possibly we need this patch: >> http://www.freebsd.org/~attilio/sxrw_unlockb.diff > > Actually, since you are only worried about reads, I think this should be > an "acq" barrier rather than a "rel". In some cases "acq" is cheaper, so we > should prefer the cheapest barrier that provides what we need. You would > still need to keep some language about the memory barriers since using "acq" > for shared unlocking is different from exclusive unlocking. Actually, I don't think that an acq barrier ensures enough protection against the reordering of 'earlier' operation thus not fixing the architecture ordering problem reported by Fabio. Also, I don't think we just have to care about reads (or I don't understand what you mean here). However, I'm not even sure that we have faster read barriers than the write one. As long as it should be true in theory I don't think that's what happen in practice. > The memory clobber is quite heavyweight. It actually forces gcc to forget any > cached memory items in registers and reload everything again. What I really > want is just a barrier to tell GCC to not reorder things. If I read a value > in the program before acquiring a lock it is in theory fine to keep that > cached across the barrier. However, there isn't a way to do this sort of > thing with GCC currently. Yes, that's the only tool we have right now with GCC. I will try to look for another way, but it sounds difficult to discover. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 19:39:33 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8C9151065698 for ; Tue, 29 Sep 2009 19:39:33 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from tomjudge.vm.bytemark.co.uk (tomjudge.vm.bytemark.co.uk [80.68.91.100]) by mx1.freebsd.org (Postfix) with ESMTP id 3EF608FC0A for ; Tue, 29 Sep 2009 19:39:32 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by tomjudge.vm.bytemark.co.uk (Postfix) with ESMTP id DD1EC489B8; Tue, 29 Sep 2009 20:39:31 +0100 (BST) X-Virus-Scanned: Debian amavisd-new at tomjudge.vm.bytemark.co.uk Received: from tomjudge.vm.bytemark.co.uk ([127.0.0.1]) by localhost (tomjudge.vm.bytemark.co.uk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W4Ezw+uNgHjS; Tue, 29 Sep 2009 20:39:19 +0100 (BST) Received: from rita.nodomain (unknown [192.168.205.6]) by tomjudge.vm.bytemark.co.uk (Postfix) with ESMTP id 9F074489B6; Tue, 29 Sep 2009 20:39:18 +0100 (BST) Message-ID: <4AC2624D.30200@tomjudge.com> Date: Tue, 29 Sep 2009 19:38:53 +0000 From: Tom Judge User-Agent: Thunderbird 2.0.0.23 (X11/20090822) MIME-Version: 1.0 To: Mark Tinguely References: <200909291908.n8TJ8UA2042239@casselton.net> In-Reply-To: <200909291908.n8TJ8UA2042239@casselton.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, mlfbsd@kanar.ci0.org Subject: Re: Help debugging: Fatal kernel mode data abort: 'External Linefetch Abort (P)' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 19:39:33 -0000 Mark Tinguely wrote: > I don't know anything about the code other than what I read today ... > > It appears from you boot traces the owin[0].owin_xlate_[lo | hi] values > should be fine in iq80321.c - an "VERBOSE_INIT_ARM" would confirm it. > > You might want to test if the "sc" pointer in iq80321.c has the same value > as the global "i80321_softc" pointer. You can add those print statements > to an "VERBOSE_INIT_ARM". It will tell you if something changed the global > pointer or if something overwrote the owin values in the structure. > > If global pointer or owin was changed before the pci attach code, you > can put the appropriate test into the earlier (obio, uart, itimer, iopwdtimer) > attach. None of these attaches use the global "i80321_softc" pointer. > > --Mark. > Hi Mark, Here is the log of a verbose_init_arm with added debug for owin[0].owin_xlate_[lo|hi]: RedBoot> go KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #30: Tue Sep 29 19:02:49 UTC 2009 root@rita.nodomain:/data/arm_build/arm/usr/src/sys/EM-7210 CPU: i80219 400MHz step A-0 (XScale core) DC enabled IC enabled WB enabled LABT branch prediction enabled 32KB/32B 32-way Instruction cache 32KB/32B 32-way write-back-locking Data cache real memory = 536870912 (512 MB) avail memory = 503738368 (480 MB) iq0: on motherboard i80321: BAR0 = 20000004.00000000 BAR1 = 40000004.00000000 i80219: BAR0 = 20000000.00000000 BAR1 = 40000000.00000000 i80219: I/O Processor, acting as PCI host i80321: SBDR = 0xa0000000 SBR0 = 0x00000018 SBR1 = 0x00000020 i80321: BANK0 = 0x10000000 BANK1 = 0x10000000 i80321: Reserve space for private devices (Inbound Window 1) hi:0x00000000 lo:0x8000000c xlate:0x80000000 size:0x04000000 i80321: RAM access (Inbound Window 2) hi:0x00000000 lo:0xa000000c xlate:0xa0000000 size:0x20000000 i80321: Reserve space for private devices (Outbound Window 1) xlate_hi:0x00000000 xlate_lo:0x80000000 obio0 on iq0 uart0: <16550 or compatible> on obio0 uart0: [FILTER] uart0: console (115200,n,8,1) itimer0: on iq0 iopwdog0: on iq0 pcib0: on iq0 pci0: on pcib0 Device 1 routed to irq 27 Device 2 routed to irq 30 Device 3 routed to irq 29 Device 5 routed to irq 30 Device 5 routed to irq 29 Device 5 routed to irq 27 em0: port 0xfe400000-0xfe40003f mem 0-0x1ffff,0x20000-0x3ffff irq 27 at device 1.0 on pci0 em0: Start: 0x00000000 em0: End: 0x0001FFFF em0: Size: 0x00020000 Fatal kernel mode data abort: 'External Linefetch Abort (P)' trapframe: 0xc00faae0 FSR=00000406, FAR=Invalid, spsr=200000d3 r0 =c00d0400, r1 =cd5bf000, r2 =00000010, r3 =0000000a r4 =c317e008, r5 =cd5bf000, r6 =c00d0400, r7 =c1302070 r8 =c317e008, r9 =c0071180, r10=c317e000, r11=c00fab50 r12=c00fab54, ssp=c00fab2c, slr=c106a96c, pc =c106a968 [thread pid 0 tid 100000 ] Stopped at e1000_init_script_state_82541+0x24c: blx r7 db> The only places i can see owin used is in i80321.c and iq80312.c and they only written to in i80312.c if the controller is a slave not a host: if (!sc->sc_is_host) { sc->sc_owin[0].owin_xlate_lo = sc->sc_iwin[1].iwin_base_lo; sc->sc_owin[0].owin_xlate_hi = sc->sc_iwin[1].iwin_base_hi; } I will see if I can compare the global softc with the ones returned by the get function. Thanks Tom From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 20:34:49 2009 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61216106566C; Tue, 29 Sep 2009 20:34:49 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 1CE898FC1B; Tue, 29 Sep 2009 20:34:49 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 36BCD46B0C; Tue, 29 Sep 2009 16:34:48 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 4AC548A01B; Tue, 29 Sep 2009 16:34:46 -0400 (EDT) From: John Baldwin To: Attilio Rao Date: Tue, 29 Sep 2009 16:28:15 -0400 User-Agent: KMail/1.9.7 References: <20090924224935.GW473@gandalf.sssup.it> <200909291425.46134.jhb@freebsd.org> <3bbf2fe10909291215i2bdd73aj13c1ac433152cab4@mail.gmail.com> In-Reply-To: <3bbf2fe10909291215i2bdd73aj13c1ac433152cab4@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200909291628.15516.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Tue, 29 Sep 2009 16:34:46 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: hackers@freebsd.org, Fabio Checconi Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 20:34:49 -0000 On Tuesday 29 September 2009 3:15:40 pm Attilio Rao wrote: > 2009/9/29 John Baldwin : > > On Tuesday 29 September 2009 11:39:37 am Attilio Rao wrote: > >> 2009/9/25 Fabio Checconi : > >> > Hi all, > >> > looking at sys/sx.h I have some troubles understanding this comment: > >> > > >> > * A note about memory barriers. Exclusive locks need to use the same > >> > * memory barriers as mutexes: _acq when acquiring an exclusive lock > >> > * and _rel when releasing an exclusive lock. On the other side, > >> > * shared lock needs to use an _acq barrier when acquiring the lock > >> > * but, since they don't update any locked data, no memory barrier is > >> > * needed when releasing a shared lock. > >> > > >> > In particular, I'm not understanding what prevents the following sequence > >> > from happening: > >> > > >> > CPU A CPU B > >> > > >> > sx_slock(&data->lock); > >> > > >> > sx_sunlock(&data->lock); > >> > > >> > /* reordered after the unlock > >> > by the cpu */ > >> > if (data->buffer) > >> > sx_xlock(&data->lock); > >> > free(data->buffer); > >> > data->buffer = NULL; > >> > sx_xunlock(&data->lock); > >> > > >> > a = *data->buffer; > >> > > >> > IOW, even if readers do not modify the data protected by the lock, > >> > without a release barrier a memory access may leak past the unlock (as > >> > the cpu won't notice any dependency between the unlock and the fetch, > >> > feeling free to reorder them), thus potentially racing with an exclusive > >> > writer accessing the data. > >> > > >> > On architectures where atomic ops serialize memory accesses this would > >> > never happen, otherwise the sequence above seems possible; am I missing > >> > something? > >> > >> I think your concerns are right, possibly we need this patch: > >> http://www.freebsd.org/~attilio/sxrw_unlockb.diff > > > > Actually, since you are only worried about reads, I think this should be > > an "acq" barrier rather than a "rel". In some cases "acq" is cheaper, so we > > should prefer the cheapest barrier that provides what we need. You would > > still need to keep some language about the memory barriers since using "acq" > > for shared unlocking is different from exclusive unlocking. > > Actually, I don't think that an acq barrier ensures enough protection > against the reordering of 'earlier' operation thus not fixing the > architecture ordering problem reported by Fabio. Also, I don't think > we just have to care about reads (or I don't understand what you mean > here). Hmmm, it might not on certain archs. It does on x86 (i.e. an lfence would work here on x86), but probably not on ia64 and sparc64. Also, we certainly only care about reads. A read/share lock cannot resolve any races where the lock holder is writing data, it can only ensure that the lock holder can safely read shared data without the data changing out from underneath it. > However, I'm not even sure that we have faster read barriers than the > write one. As long as it should be true in theory I don't think that's > what happen in practice. bde@ found that sfence was generally much more expensive than lfence on x86. However, since x86 guarantees the order of all stores we don't actually need to use sfence at all on x86 anyway. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 20:39:45 2009 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D52061065679; Tue, 29 Sep 2009 20:39:45 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-bw0-f227.google.com (mail-bw0-f227.google.com [209.85.218.227]) by mx1.freebsd.org (Postfix) with ESMTP id EAD148FC0C; Tue, 29 Sep 2009 20:39:44 +0000 (UTC) Received: by bwz27 with SMTP id 27so4194805bwz.43 for ; Tue, 29 Sep 2009 13:39:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=qW2U9d8YpA940/f8735p54sjdZcjtB8DtqOpOhb2JRo=; b=nsgsjMVhC90V7AkgFv2XdrMq5MSRJrNNkbWHOCVdzA3ydhedn2lFzTRI61DC+j9g2b driLt8OuPamhNMEROwoqobvLgFEBYfg4NQ8sPtEFh/IgLUZv5EpZ7kR4XCikKrxsmKUp gv68lMlMDv8qV/vwRLT2fLldfhpWvh6EY1xPs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=Sr7sO7JO0eXjJYukcSVV9Nbkipsna83JP9QR5KJjIA7fNLPTCeykNZDVUfpky+/LVc p94TJLeDiJ5d81/YntLkLEyaRq+jpcoI9A26/XX9qgDdzF1EDspAjkz+efe8QEnaf/H1 Wl+GuDR3Ty85S34QTQI0tDjm6EM+h53dloWqQ= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.6.9 with SMTP id 9mr1469442fax.84.1254256783571; Tue, 29 Sep 2009 13:39:43 -0700 (PDT) In-Reply-To: <200909291628.15516.jhb@freebsd.org> References: <20090924224935.GW473@gandalf.sssup.it> <200909291425.46134.jhb@freebsd.org> <3bbf2fe10909291215i2bdd73aj13c1ac433152cab4@mail.gmail.com> <200909291628.15516.jhb@freebsd.org> Date: Tue, 29 Sep 2009 22:39:43 +0200 X-Google-Sender-Auth: 2055cafbe55f7990 Message-ID: <3bbf2fe10909291339s5705a9bendb4c9331293b45a4@mail.gmail.com> From: Attilio Rao To: John Baldwin Content-Type: text/plain; charset=UTF-8 Cc: hackers@freebsd.org, Fabio Checconi Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 20:39:46 -0000 2009/9/29 John Baldwin : > On Tuesday 29 September 2009 3:15:40 pm Attilio Rao wrote: >> 2009/9/29 John Baldwin : >> > On Tuesday 29 September 2009 11:39:37 am Attilio Rao wrote: >> >> 2009/9/25 Fabio Checconi : >> >> > Hi all, >> >> > looking at sys/sx.h I have some troubles understanding this comment: >> >> > >> >> > * A note about memory barriers. Exclusive locks need to use the same >> >> > * memory barriers as mutexes: _acq when acquiring an exclusive lock >> >> > * and _rel when releasing an exclusive lock. On the other side, >> >> > * shared lock needs to use an _acq barrier when acquiring the lock >> >> > * but, since they don't update any locked data, no memory barrier is >> >> > * needed when releasing a shared lock. >> >> > >> >> > In particular, I'm not understanding what prevents the following sequence >> >> > from happening: >> >> > >> >> > CPU A CPU B >> >> > >> >> > sx_slock(&data->lock); >> >> > >> >> > sx_sunlock(&data->lock); >> >> > >> >> > /* reordered after the unlock >> >> > by the cpu */ >> >> > if (data->buffer) >> >> > sx_xlock(&data->lock); >> >> > free(data->buffer); >> >> > data->buffer = NULL; >> >> > sx_xunlock(&data->lock); >> >> > >> >> > a = *data->buffer; >> >> > >> >> > IOW, even if readers do not modify the data protected by the lock, >> >> > without a release barrier a memory access may leak past the unlock (as >> >> > the cpu won't notice any dependency between the unlock and the fetch, >> >> > feeling free to reorder them), thus potentially racing with an exclusive >> >> > writer accessing the data. >> >> > >> >> > On architectures where atomic ops serialize memory accesses this would >> >> > never happen, otherwise the sequence above seems possible; am I missing >> >> > something? >> >> >> >> I think your concerns are right, possibly we need this patch: >> >> http://www.freebsd.org/~attilio/sxrw_unlockb.diff >> > >> > Actually, since you are only worried about reads, I think this should be >> > an "acq" barrier rather than a "rel". In some cases "acq" is cheaper, so we >> > should prefer the cheapest barrier that provides what we need. You would >> > still need to keep some language about the memory barriers since using "acq" >> > for shared unlocking is different from exclusive unlocking. >> >> Actually, I don't think that an acq barrier ensures enough protection >> against the reordering of 'earlier' operation thus not fixing the >> architecture ordering problem reported by Fabio. Also, I don't think >> we just have to care about reads (or I don't understand what you mean >> here). > > Hmmm, it might not on certain archs. It does on x86 (i.e. an lfence would > work here on x86), but probably not on ia64 and sparc64. Also, we certainly > only care about reads. A read/share lock cannot resolve any races where the > lock holder is writing data, it can only ensure that the lock holder can > safely read shared data without the data changing out from underneath it. > >> However, I'm not even sure that we have faster read barriers than the >> write one. As long as it should be true in theory I don't think that's >> what happen in practice. > > bde@ found that sfence was generally much more expensive than lfence on x86. > However, since x86 guarantees the order of all stores we don't actually need > to use sfence at all on x86 anyway. Yes, x86 guarantees that the stores are strong ordered so I don't think acq to be faster than rel. Can I assume the patch I already sent as reviewed by you and commit then, right? Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 20:42:15 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C76F0106568B; Tue, 29 Sep 2009 20:42:15 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-bw0-f227.google.com (mail-bw0-f227.google.com [209.85.218.227]) by mx1.freebsd.org (Postfix) with ESMTP id DF5DA8FC14; Tue, 29 Sep 2009 20:42:14 +0000 (UTC) Received: by bwz27 with SMTP id 27so4196458bwz.43 for ; Tue, 29 Sep 2009 13:42:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=tQ0FHKDMhi8mDaGcGoF5QD1AnO2xGxq8Cmig41j29SA=; b=StcYjmKFJFgGhOL84O9IKnn4teEMmQPKC0bsDXJFrXo+X0hpKNa7pSFdLl9W5x7MNl 2EkTG2f3sVUBsvb0zby9Kts/fNOeyTG9wcd1lyGq+4SIZQebadB6KPYj7NIlCA7agJ5+ Zn2SZsrIKAf8ce10onRIRZytCcTMVezG1QH9s= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=CDaa3D9Q48As6BilgP0z3P1vcrcaVqbs1M0sMOcark79IuCylNyzV/cq8EUCeq/bOL leO0dv0lSr0kYUSfvtBkKjWEkWS/XF9YvTMsw49siTY1Mfofc8prtGstVfPVsobVIuCS ERgle8R0NqFdkG01aUxWMuXUubmo0HxtE5d18= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.143.79 with SMTP id t15mr1465334fau.6.1254256933780; Tue, 29 Sep 2009 13:42:13 -0700 (PDT) In-Reply-To: <200909291953.36373.max@love2party.net> References: <20090924224935.GW473@gandalf.sssup.it> <3bbf2fe10909290839w305c85c3t1532bd7733c39a6a@mail.gmail.com> <200909291953.36373.max@love2party.net> Date: Tue, 29 Sep 2009 22:42:13 +0200 X-Google-Sender-Auth: 0680da0247ddd93f Message-ID: <3bbf2fe10909291342o4d32e381ge23e446582bb2d18@mail.gmail.com> From: Attilio Rao To: Max Laier Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org, Fabio Checconi Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 20:42:15 -0000 2009/9/29 Max Laier : > On Tuesday 29 September 2009 17:39:37 Attilio Rao wrote: >> 2009/9/25 Fabio Checconi : >> > Hi all, >> > looking at sys/sx.h I have some troubles understanding this comment: >> > >> > * A note about memory barriers. Exclusive locks need to use the same >> > * memory barriers as mutexes: _acq when acquiring an exclusive lock >> > * and _rel when releasing an exclusive lock. On the other side, >> > * shared lock needs to use an _acq barrier when acquiring the lock >> > * but, since they don't update any locked data, no memory barrier is >> > * needed when releasing a shared lock. >> > >> > In particular, I'm not understanding what prevents the following sequence >> > from happening: >> > >> > CPU A CPU B >> > >> > sx_slock(&data->lock); >> > >> > sx_sunlock(&data->lock); >> > >> > /* reordered after the unlock >> > by the cpu */ >> > if (data->buffer) >> > sx_xlock(&data->lock); >> > free(data->buffer); >> > data->buffer = NULL; >> > sx_xunlock(&data->lock); >> > >> > a = *data->buffer; >> > >> > IOW, even if readers do not modify the data protected by the lock, >> > without a release barrier a memory access may leak past the unlock (as >> > the cpu won't notice any dependency between the unlock and the fetch, >> > feeling free to reorder them), thus potentially racing with an exclusive >> > writer accessing the data. >> > >> > On architectures where atomic ops serialize memory accesses this would >> > never happen, otherwise the sequence above seems possible; am I missing >> > something? >> >> I think your concerns are right, possibly we need this patch: >> http://www.freebsd.org/~attilio/sxrw_unlockb.diff >> >> However speaking with John we agreed possibly there is a more serious >> breakage. Possibly, memory barriers would also require to ensure the >> compiler to not reorder the operation, while right now, in FreeBSD, they >> just take care of the reordering from the architecture perspective. >> The only way I'm aware of GCC offers that is to clobber memory. >> I will provide a patch that address this soon, hoping that GCC will be >> smart enough to not overhead too much the memory clobbering but just >> try to understand what's our purpose and servers it (I will try to >> compare code generated before and after the patch at least for tier-1 >> architectures). > > Does GCC really reorder accesses to volatile objects? The C Standard seems to > object: > > 5.1.2.3 - 2 > Accessing a volatile object, modifying an object, modifying a file, or calling > a function that does any of those operations are all side effects,11) which > are changes in the state of the execution environment. Evaluation of an > expression may produce side effects. At certain specified points in the > execution sequence called sequence points, all side effects of previous > evaluations shall be complete and no side effects of subsequent evaluations > shall have taken place. (A summary of the sequence points is given in annex > C.) Very interesting. I was thinking about the other operating systems which basically do 'memory clobbering' for ensuring a compiler barrier, but actually they often forsee such a barrier without the conjuction of a memory operand. I think I will need to speak a bit with a GCC engineer in order to see what do they implement in regard of volatile operands. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 20:54:28 2009 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B7C16106568D; Tue, 29 Sep 2009 20:54:28 +0000 (UTC) (envelope-from marius@nuenneri.ch) Received: from mail-ew0-f209.google.com (mail-ew0-f209.google.com [209.85.219.209]) by mx1.freebsd.org (Postfix) with ESMTP id C92038FC22; Tue, 29 Sep 2009 20:54:27 +0000 (UTC) Received: by ewy5 with SMTP id 5so4528719ewy.36 for ; Tue, 29 Sep 2009 13:54:26 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.53.205 with SMTP id g55mr1179248wec.160.1254256016832; Tue, 29 Sep 2009 13:26:56 -0700 (PDT) In-Reply-To: <3bbf2fe10909291215i2bdd73aj13c1ac433152cab4@mail.gmail.com> References: <20090924224935.GW473@gandalf.sssup.it> <3bbf2fe10909290839w305c85c3t1532bd7733c39a6a@mail.gmail.com> <200909291425.46134.jhb@freebsd.org> <3bbf2fe10909291215i2bdd73aj13c1ac433152cab4@mail.gmail.com> Date: Tue, 29 Sep 2009 22:26:56 +0200 Message-ID: From: =?ISO-8859-1?Q?Marius_N=FCnnerich?= To: Attilio Rao Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org, Fabio Checconi Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 20:54:28 -0000 On Tue, Sep 29, 2009 at 21:15, Attilio Rao wrote: > 2009/9/29 John Baldwin : >> On Tuesday 29 September 2009 11:39:37 am Attilio Rao wrote: >>> 2009/9/25 Fabio Checconi : >>> > Hi all, >>> > =A0looking at sys/sx.h I have some troubles understanding this commen= t: >>> > >>> > =A0* A note about memory barriers. =A0Exclusive locks need to use the= same >>> > =A0* memory barriers as mutexes: _acq when acquiring an exclusive loc= k >>> > =A0* and _rel when releasing an exclusive lock. =A0On the other side, >>> > =A0* shared lock needs to use an _acq barrier when acquiring the lock >>> > =A0* but, since they don't update any locked data, no memory barrier = is >>> > =A0* needed when releasing a shared lock. >>> > >>> > In particular, I'm not understanding what prevents the following sequ= ence >>> > from happening: >>> > >>> > CPU A =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 CPU B >>> > >>> > sx_slock(&data->lock); >>> > >>> > sx_sunlock(&data->lock); >>> > >>> > /* reordered after the unlock >>> > =A0 by the cpu */ >>> > if (data->buffer) >>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0sx_xlock(&data->lock); >>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0free(data->buffer); >>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0data->buffer =3D NULL; >>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0sx_xunlock(&data->lock); >>> > >>> > =A0 =A0 =A0 =A0a =3D *data->buffer; >>> > >>> > IOW, even if readers do not modify the data protected by the lock, >>> > without a release barrier a memory access may leak past the unlock (a= s >>> > the cpu won't notice any dependency between the unlock and the fetch, >>> > feeling free to reorder them), thus potentially racing with an exclus= ive >>> > writer accessing the data. >>> > >>> > On architectures where atomic ops serialize memory accesses this woul= d >>> > never happen, otherwise the sequence above seems possible; am I missi= ng >>> > something? >>> >>> I think your concerns are right, possibly we need this patch: >>> http://www.freebsd.org/~attilio/sxrw_unlockb.diff >> >> Actually, since you are only worried about reads, I think this should be >> an "acq" barrier rather than a "rel". =A0In some cases "acq" is cheaper,= so we >> should prefer the cheapest barrier that provides what we need. =A0You wo= uld >> still need to keep some language about the memory barriers since using "= acq" >> for shared unlocking is different from exclusive unlocking. > > Actually, I don't think that an acq barrier ensures enough protection > against the reordering of 'earlier' operation thus not fixing the > architecture ordering problem reported by Fabio. Also, I don't think > we just have to care about reads (or =A0I don't understand what you mean > here). > However, I'm not even sure that we have faster read barriers than the > write one. As long as it should be true in theory I don't think that's > what happen in practice. > >> The memory clobber is quite heavyweight. =A0It actually forces gcc to fo= rget any >> cached memory items in registers and reload everything again. =A0What I = really >> want is just a barrier to tell GCC to not reorder things. =A0If I read a= value >> in the program before acquiring a lock it is in theory fine to keep that >> cached across the barrier. =A0However, there isn't a way to do this sort= of >> thing with GCC currently. > > Yes, that's the only tool we have right now with GCC. I will try to > look for another way, but it sounds difficult to discover. Even if we would have a mechanism to tell GCC to not reorder the instructions the CPU itself would still be free to reorder if there are no barriers. Or am I missing something? From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 20:58:21 2009 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EB4321065672; Tue, 29 Sep 2009 20:58:21 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-fx0-f222.google.com (mail-fx0-f222.google.com [209.85.220.222]) by mx1.freebsd.org (Postfix) with ESMTP id 27AA38FC2D; Tue, 29 Sep 2009 20:58:20 +0000 (UTC) Received: by fxm22 with SMTP id 22so1288874fxm.36 for ; Tue, 29 Sep 2009 13:58:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=JOz4+6q9qPXQ73j2H0FXSmtQzV6dZR/V2ET4kGQU/uI=; b=kevww5ZYTkWQ0hFx8Qf8uZHGZ1I4jnqqHJ5RUs4bItgKpgA9Q8AatCocdj7eXKySCI kU2Ud9y6R0M7LE+tLWp2iNvo9iklYySQkoYvGOrzRejKXLMqpbFOdllosfQOJWJpYEGe I3LUfBdxjev90SC2ZID/evIuFTUSRDzsj7Yag= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=m/VFIvHLoYnjIBK4Z+JY5eKvzLNTiUnusacSlFOde0NfwyFsl6h6TXQpk/uxmfjmVs Zf86ZmfUIsc9hqWWNFbb44IoyHAibS+4cX+Vaxq6OJ0rTSrocmd+zPZfNRpU8jhUVD8t FTM3wUoWM7V9VV/kU05hTqEnBWTbz1aJo1hLY= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.64.133 with SMTP id e5mr1444413fai.31.1254257899985; Tue, 29 Sep 2009 13:58:19 -0700 (PDT) In-Reply-To: References: <20090924224935.GW473@gandalf.sssup.it> <3bbf2fe10909290839w305c85c3t1532bd7733c39a6a@mail.gmail.com> <200909291425.46134.jhb@freebsd.org> <3bbf2fe10909291215i2bdd73aj13c1ac433152cab4@mail.gmail.com> Date: Tue, 29 Sep 2009 22:58:19 +0200 X-Google-Sender-Auth: d59edbff119ef1da Message-ID: <3bbf2fe10909291358q3063f763md4ccba88c3b1d0be@mail.gmail.com> From: Attilio Rao To: =?UTF-8?Q?Marius_N=C3=BCnnerich?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org, Fabio Checconi Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 20:58:22 -0000 2009/9/29 Marius N=C3=BCnnerich : > On Tue, Sep 29, 2009 at 21:15, Attilio Rao wrote: >> 2009/9/29 John Baldwin : >>> On Tuesday 29 September 2009 11:39:37 am Attilio Rao wrote: >>>> 2009/9/25 Fabio Checconi : >>>> > Hi all, >>>> > looking at sys/sx.h I have some troubles understanding this comment= : >>>> > >>>> > * A note about memory barriers. Exclusive locks need to use the sa= me >>>> > * memory barriers as mutexes: _acq when acquiring an exclusive lock >>>> > * and _rel when releasing an exclusive lock. On the other side, >>>> > * shared lock needs to use an _acq barrier when acquiring the lock >>>> > * but, since they don't update any locked data, no memory barrier i= s >>>> > * needed when releasing a shared lock. >>>> > >>>> > In particular, I'm not understanding what prevents the following seq= uence >>>> > from happening: >>>> > >>>> > CPU A CPU B >>>> > >>>> > sx_slock(&data->lock); >>>> > >>>> > sx_sunlock(&data->lock); >>>> > >>>> > /* reordered after the unlock >>>> > by the cpu */ >>>> > if (data->buffer) >>>> > sx_xlock(&data->lock); >>>> > free(data->buffer); >>>> > data->buffer =3D NULL; >>>> > sx_xunlock(&data->lock); >>>> > >>>> > a =3D *data->buffer; >>>> > >>>> > IOW, even if readers do not modify the data protected by the lock, >>>> > without a release barrier a memory access may leak past the unlock (= as >>>> > the cpu won't notice any dependency between the unlock and the fetch= , >>>> > feeling free to reorder them), thus potentially racing with an exclu= sive >>>> > writer accessing the data. >>>> > >>>> > On architectures where atomic ops serialize memory accesses this wou= ld >>>> > never happen, otherwise the sequence above seems possible; am I miss= ing >>>> > something? >>>> >>>> I think your concerns are right, possibly we need this patch: >>>> http://www.freebsd.org/~attilio/sxrw_unlockb.diff >>> >>> Actually, since you are only worried about reads, I think this should b= e >>> an "acq" barrier rather than a "rel". In some cases "acq" is cheaper, = so we >>> should prefer the cheapest barrier that provides what we need. You wou= ld >>> still need to keep some language about the memory barriers since using = "acq" >>> for shared unlocking is different from exclusive unlocking. >> >> Actually, I don't think that an acq barrier ensures enough protection >> against the reordering of 'earlier' operation thus not fixing the >> architecture ordering problem reported by Fabio. Also, I don't think >> we just have to care about reads (or I don't understand what you mean >> here). >> However, I'm not even sure that we have faster read barriers than the >> write one. As long as it should be true in theory I don't think that's >> what happen in practice. >> >>> The memory clobber is quite heavyweight. It actually forces gcc to for= get any >>> cached memory items in registers and reload everything again. What I r= eally >>> want is just a barrier to tell GCC to not reorder things. If I read a = value >>> in the program before acquiring a lock it is in theory fine to keep tha= t >>> cached across the barrier. However, there isn't a way to do this sort = of >>> thing with GCC currently. >> >> Yes, that's the only tool we have right now with GCC. I will try to >> look for another way, but it sounds difficult to discover. > > Even if we would have a mechanism to tell GCC to not reorder the > instructions the CPU itself would still be free to reorder if there > are no barriers. Or am I missing something? Our code already takes care of that case in our barriers. Attilio --=20 Peace can only be achieved by understanding - A. Einstein From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 19:21:20 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67A4C1065693 for ; Tue, 29 Sep 2009 19:21:20 +0000 (UTC) (envelope-from tinguely@casselton.net) Received: from casselton.net (casselton.net [63.165.140.2]) by mx1.freebsd.org (Postfix) with ESMTP id F30598FC4D for ; Tue, 29 Sep 2009 19:21:19 +0000 (UTC) Received: from casselton.net (localhost [127.0.0.1]) by casselton.net (8.14.3/8.14.3) with ESMTP id n8TJ8V0O042240; Tue, 29 Sep 2009 14:08:31 -0500 (CDT) (envelope-from tinguely@casselton.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=casselton.net; s=ccnMail; t=1254251311; bh=lowk/ETMZvPnmFNlbbe7HE7cf0QlcHPxD7/zTRUMkb0=; h=Date:From:Message-Id:To:Subject:Cc:In-Reply-To; b=TLVIZdaEt3nRzGoFO/vDOKmBHUpGiCY1fhcvWGs32hJ4mgBG03+v8ejrAo3EPKXAy w4V2vj3mjyIKziZNw+NuWy0H1Kan+sngUVliZnmo8Vds0ORivmeCSkkapE/TEzLaSZ AVNp98V0HOIu2CaQs91x8UmKR3AHQTSE5xebglm4= Received: (from tinguely@localhost) by casselton.net (8.14.3/8.14.2/Submit) id n8TJ8UA2042239; Tue, 29 Sep 2009 14:08:30 -0500 (CDT) (envelope-from tinguely) Date: Tue, 29 Sep 2009 14:08:30 -0500 (CDT) From: Mark Tinguely Message-Id: <200909291908.n8TJ8UA2042239@casselton.net> To: mlfbsd@kanar.ci0.org, tom@tomjudge.com In-Reply-To: <4AC21F44.6060004@tomjudge.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.3.2 (casselton.net [127.0.0.1]); Tue, 29 Sep 2009 14:08:31 -0500 (CDT) X-Mailman-Approved-At: Tue, 29 Sep 2009 21:08:52 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: Help debugging: Fatal kernel mode data abort: 'External Linefetch Abort (P)' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 19:21:20 -0000 I don't know anything about the code other than what I read today ... It appears from you boot traces the owin[0].owin_xlate_[lo | hi] values should be fine in iq80321.c - an "VERBOSE_INIT_ARM" would confirm it. You might want to test if the "sc" pointer in iq80321.c has the same value as the global "i80321_softc" pointer. You can add those print statements to an "VERBOSE_INIT_ARM". It will tell you if something changed the global pointer or if something overwrote the owin values in the structure. If global pointer or owin was changed before the pci attach code, you can put the appropriate test into the earlier (obio, uart, itimer, iopwdtimer) attach. None of these attaches use the global "i80321_softc" pointer. --Mark. From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 21:32:33 2009 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BFB40106566B; Tue, 29 Sep 2009 21:32:33 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 8FF298FC08; Tue, 29 Sep 2009 21:32:33 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 39B1346B0D; Tue, 29 Sep 2009 17:32:33 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 851228A01D; Tue, 29 Sep 2009 17:32:32 -0400 (EDT) From: John Baldwin To: Marius =?iso-8859-1?q?N=FCnnerich?= Date: Tue, 29 Sep 2009 17:21:27 -0400 User-Agent: KMail/1.9.7 References: <20090924224935.GW473@gandalf.sssup.it> <3bbf2fe10909291215i2bdd73aj13c1ac433152cab4@mail.gmail.com> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <200909291721.27755.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Tue, 29 Sep 2009 17:32:32 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Attilio Rao , hackers@freebsd.org, Fabio Checconi Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 21:32:33 -0000 On Tuesday 29 September 2009 4:26:56 pm Marius N=FCnnerich wrote: > On Tue, Sep 29, 2009 at 21:15, Attilio Rao wrote: > > 2009/9/29 John Baldwin : > >> On Tuesday 29 September 2009 11:39:37 am Attilio Rao wrote: > >>> 2009/9/25 Fabio Checconi : > >>> > Hi all, > >>> > =A0looking at sys/sx.h I have some troubles understanding this comm= ent: > >>> > > >>> > =A0* A note about memory barriers. =A0Exclusive locks need to use t= he same > >>> > =A0* memory barriers as mutexes: _acq when acquiring an exclusive l= ock > >>> > =A0* and _rel when releasing an exclusive lock. =A0On the other sid= e, > >>> > =A0* shared lock needs to use an _acq barrier when acquiring the lo= ck > >>> > =A0* but, since they don't update any locked data, no memory barrie= r is > >>> > =A0* needed when releasing a shared lock. > >>> > > >>> > In particular, I'm not understanding what prevents the following se= quence > >>> > from happening: > >>> > > >>> > CPU A =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 CPU B > >>> > > >>> > sx_slock(&data->lock); > >>> > > >>> > sx_sunlock(&data->lock); > >>> > > >>> > /* reordered after the unlock > >>> > =A0 by the cpu */ > >>> > if (data->buffer) > >>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0sx_xlock(&data->lock); > >>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0free(data->buffer); > >>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0data->buffer =3D NULL; > >>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0sx_xunlock(&data->lock); > >>> > > >>> > =A0 =A0 =A0 =A0a =3D *data->buffer; > >>> > > >>> > IOW, even if readers do not modify the data protected by the lock, > >>> > without a release barrier a memory access may leak past the unlock = (as > >>> > the cpu won't notice any dependency between the unlock and the fetc= h, > >>> > feeling free to reorder them), thus potentially racing with an excl= usive > >>> > writer accessing the data. > >>> > > >>> > On architectures where atomic ops serialize memory accesses this wo= uld > >>> > never happen, otherwise the sequence above seems possible; am I mis= sing > >>> > something? > >>> > >>> I think your concerns are right, possibly we need this patch: > >>> http://www.freebsd.org/~attilio/sxrw_unlockb.diff > >> > >> Actually, since you are only worried about reads, I think this should = be > >> an "acq" barrier rather than a "rel". =A0In some cases "acq" is cheape= r, so we > >> should prefer the cheapest barrier that provides what we need. =A0You = would > >> still need to keep some language about the memory barriers since using= "acq" > >> for shared unlocking is different from exclusive unlocking. > > > > Actually, I don't think that an acq barrier ensures enough protection > > against the reordering of 'earlier' operation thus not fixing the > > architecture ordering problem reported by Fabio. Also, I don't think > > we just have to care about reads (or =A0I don't understand what you mean > > here). > > However, I'm not even sure that we have faster read barriers than the > > write one. As long as it should be true in theory I don't think that's > > what happen in practice. > > > >> The memory clobber is quite heavyweight. =A0It actually forces gcc to = forget any > >> cached memory items in registers and reload everything again. =A0What = I really > >> want is just a barrier to tell GCC to not reorder things. =A0If I read= a value > >> in the program before acquiring a lock it is in theory fine to keep th= at > >> cached across the barrier. =A0However, there isn't a way to do this so= rt of > >> thing with GCC currently. > > > > Yes, that's the only tool we have right now with GCC. I will try to > > look for another way, but it sounds difficult to discover. >=20 > Even if we would have a mechanism to tell GCC to not reorder the > instructions the CPU itself would still be free to reorder if there > are no barriers. Or am I missing something? No, the thing to do here for the second part is add "memory" clobbers to the existing atomic ops with barriers. It will still require barriers for them= to be enforced. =2D-=20 John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 21:32:34 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8F323106566C; Tue, 29 Sep 2009 21:32:34 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 603908FC15; Tue, 29 Sep 2009 21:32:34 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id EBA7546B2D; Tue, 29 Sep 2009 17:32:33 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 3DDFF8A01F; Tue, 29 Sep 2009 17:32:33 -0400 (EDT) From: John Baldwin To: Attilio Rao Date: Tue, 29 Sep 2009 17:31:32 -0400 User-Agent: KMail/1.9.7 References: <20090924224935.GW473@gandalf.sssup.it> <200909291953.36373.max@love2party.net> <3bbf2fe10909291342o4d32e381ge23e446582bb2d18@mail.gmail.com> In-Reply-To: <3bbf2fe10909291342o4d32e381ge23e446582bb2d18@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200909291731.32394.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Tue, 29 Sep 2009 17:32:33 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Max Laier , Fabio Checconi , freebsd-hackers@freebsd.org Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 21:32:34 -0000 On Tuesday 29 September 2009 4:42:13 pm Attilio Rao wrote: > 2009/9/29 Max Laier : > > On Tuesday 29 September 2009 17:39:37 Attilio Rao wrote: > >> 2009/9/25 Fabio Checconi : > >> > Hi all, > >> > looking at sys/sx.h I have some troubles understanding this comment: > >> > > >> > * A note about memory barriers. Exclusive locks need to use the same > >> > * memory barriers as mutexes: _acq when acquiring an exclusive lock > >> > * and _rel when releasing an exclusive lock. On the other side, > >> > * shared lock needs to use an _acq barrier when acquiring the lock > >> > * but, since they don't update any locked data, no memory barrier is > >> > * needed when releasing a shared lock. > >> > > >> > In particular, I'm not understanding what prevents the following sequence > >> > from happening: > >> > > >> > CPU A CPU B > >> > > >> > sx_slock(&data->lock); > >> > > >> > sx_sunlock(&data->lock); > >> > > >> > /* reordered after the unlock > >> > by the cpu */ > >> > if (data->buffer) > >> > sx_xlock(&data->lock); > >> > free(data->buffer); > >> > data->buffer = NULL; > >> > sx_xunlock(&data->lock); > >> > > >> > a = *data->buffer; > >> > > >> > IOW, even if readers do not modify the data protected by the lock, > >> > without a release barrier a memory access may leak past the unlock (as > >> > the cpu won't notice any dependency between the unlock and the fetch, > >> > feeling free to reorder them), thus potentially racing with an exclusive > >> > writer accessing the data. > >> > > >> > On architectures where atomic ops serialize memory accesses this would > >> > never happen, otherwise the sequence above seems possible; am I missing > >> > something? > >> > >> I think your concerns are right, possibly we need this patch: > >> http://www.freebsd.org/~attilio/sxrw_unlockb.diff > >> > >> However speaking with John we agreed possibly there is a more serious > >> breakage. Possibly, memory barriers would also require to ensure the > >> compiler to not reorder the operation, while right now, in FreeBSD, they > >> just take care of the reordering from the architecture perspective. > >> The only way I'm aware of GCC offers that is to clobber memory. > >> I will provide a patch that address this soon, hoping that GCC will be > >> smart enough to not overhead too much the memory clobbering but just > >> try to understand what's our purpose and servers it (I will try to > >> compare code generated before and after the patch at least for tier-1 > >> architectures). > > > > Does GCC really reorder accesses to volatile objects? The C Standard seems to > > object: > > > > 5.1.2.3 - 2 > > Accessing a volatile object, modifying an object, modifying a file, or calling > > a function that does any of those operations are all side effects,11) which > > are changes in the state of the execution environment. Evaluation of an > > expression may produce side effects. At certain specified points in the > > execution sequence called sequence points, all side effects of previous > > evaluations shall be complete and no side effects of subsequent evaluations > > shall have taken place. (A summary of the sequence points is given in annex > > C.) > > Very interesting. > I was thinking about the other operating systems which basically do > 'memory clobbering' for ensuring a compiler barrier, but actually they > often forsee such a barrier without the conjuction of a memory > operand. > > I think I will need to speak a bit with a GCC engineer in order to see > what do they implement in regard of volatile operands. GCC can be quite aggressive with reordering even in the face of volatile. I was recently doing a hack to export some data from the kernel to userland that used a spin loop to grab a snapshot of the contents of a structure similar to the method used in the kernel with the timehands structures. It used a volatile structure exposed from the kernel that looked something like: struct foo { volatile int gen; /* other stuff */ }; volatile struct foo *p; do { x = p->gen; /* read other stuff */ y = p->gen; } while (x != y && x != 0); GCC moved the 'y = ' up into the middle of the '/* read other stuff */'. I eventually had to add explicit "memory" clobbers to force GCC to not move the reads of 'gen' around but do them "around" all the other operations, so that the working code is: do { x = p->gen; asm volatile("" ::: "memory"); /* read other stuff */ asm volatile("" ::: "memory"); y = p->gen; } while (x != y && x != 0); -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 21:39:46 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 727DE106568D; Tue, 29 Sep 2009 21:39:46 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-bw0-f227.google.com (mail-bw0-f227.google.com [209.85.218.227]) by mx1.freebsd.org (Postfix) with ESMTP id 6704F8FC27; Tue, 29 Sep 2009 21:39:44 +0000 (UTC) Received: by bwz27 with SMTP id 27so4233101bwz.43 for ; Tue, 29 Sep 2009 14:39:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=Z9+Lo2TpUs+rG5Wl+UvI0jFs+8AzBuEjq93tt+kOjw0=; b=Dk7uRND3UAfSI1OshiWjgly/9NNZa7lrz660uzy6oGd4LKH2QCo806RCimQuQM5xLA RtyAhylEfpqlSFje/X27YCz2gtYY9R92LT/MhorxZTsskWIe5DqDJ08Srn7ZT2UdT5xQ xpS/rX8bp8wci6i4x5P9XaPUDsT9up8xDMM7M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=OOtQamT5RVTzkeyZnohiGMsc4yNglPAQ4IRDlcqJmmH3wBqXgk1u3zmVombZeVUn0Q 0cHQ0Yg6sLKFUiBlinhw8oKghz7nvO5TR4vncjH4JM+ytoD/bZTpcXDllIe06sRoONoR P89rObxgVCIggfrTeAcrkelrvcrcj8uh/cuR0= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.60.68 with SMTP id o4mr1437433fah.102.1254260383410; Tue, 29 Sep 2009 14:39:43 -0700 (PDT) In-Reply-To: <200909291731.32394.jhb@freebsd.org> References: <20090924224935.GW473@gandalf.sssup.it> <200909291953.36373.max@love2party.net> <3bbf2fe10909291342o4d32e381ge23e446582bb2d18@mail.gmail.com> <200909291731.32394.jhb@freebsd.org> Date: Tue, 29 Sep 2009 23:39:43 +0200 X-Google-Sender-Auth: fb3e3246b4db98a6 Message-ID: <3bbf2fe10909291439x21f53e34n60d63554b1dea0de@mail.gmail.com> From: Attilio Rao To: John Baldwin Content-Type: text/plain; charset=UTF-8 Cc: Ed Schouten , Max Laier , Roman Divacky , Fabio Checconi , freebsd-hackers@freebsd.org Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 21:39:46 -0000 2009/9/29 John Baldwin : > On Tuesday 29 September 2009 4:42:13 pm Attilio Rao wrote: >> 2009/9/29 Max Laier : >> > On Tuesday 29 September 2009 17:39:37 Attilio Rao wrote: >> >> 2009/9/25 Fabio Checconi : >> >> > Hi all, >> >> > looking at sys/sx.h I have some troubles understanding this comment: >> >> > >> >> > * A note about memory barriers. Exclusive locks need to use the same >> >> > * memory barriers as mutexes: _acq when acquiring an exclusive lock >> >> > * and _rel when releasing an exclusive lock. On the other side, >> >> > * shared lock needs to use an _acq barrier when acquiring the lock >> >> > * but, since they don't update any locked data, no memory barrier is >> >> > * needed when releasing a shared lock. >> >> > >> >> > In particular, I'm not understanding what prevents the following sequence >> >> > from happening: >> >> > >> >> > CPU A CPU B >> >> > >> >> > sx_slock(&data->lock); >> >> > >> >> > sx_sunlock(&data->lock); >> >> > >> >> > /* reordered after the unlock >> >> > by the cpu */ >> >> > if (data->buffer) >> >> > sx_xlock(&data->lock); >> >> > free(data->buffer); >> >> > data->buffer = NULL; >> >> > sx_xunlock(&data->lock); >> >> > >> >> > a = *data->buffer; >> >> > >> >> > IOW, even if readers do not modify the data protected by the lock, >> >> > without a release barrier a memory access may leak past the unlock (as >> >> > the cpu won't notice any dependency between the unlock and the fetch, >> >> > feeling free to reorder them), thus potentially racing with an exclusive >> >> > writer accessing the data. >> >> > >> >> > On architectures where atomic ops serialize memory accesses this would >> >> > never happen, otherwise the sequence above seems possible; am I missing >> >> > something? >> >> >> >> I think your concerns are right, possibly we need this patch: >> >> http://www.freebsd.org/~attilio/sxrw_unlockb.diff >> >> >> >> However speaking with John we agreed possibly there is a more serious >> >> breakage. Possibly, memory barriers would also require to ensure the >> >> compiler to not reorder the operation, while right now, in FreeBSD, they >> >> just take care of the reordering from the architecture perspective. >> >> The only way I'm aware of GCC offers that is to clobber memory. >> >> I will provide a patch that address this soon, hoping that GCC will be >> >> smart enough to not overhead too much the memory clobbering but just >> >> try to understand what's our purpose and servers it (I will try to >> >> compare code generated before and after the patch at least for tier-1 >> >> architectures). >> > >> > Does GCC really reorder accesses to volatile objects? The C Standard seems to >> > object: >> > >> > 5.1.2.3 - 2 >> > Accessing a volatile object, modifying an object, modifying a file, or calling >> > a function that does any of those operations are all side effects,11) which >> > are changes in the state of the execution environment. Evaluation of an >> > expression may produce side effects. At certain specified points in the >> > execution sequence called sequence points, all side effects of previous >> > evaluations shall be complete and no side effects of subsequent evaluations >> > shall have taken place. (A summary of the sequence points is given in annex >> > C.) >> >> Very interesting. >> I was thinking about the other operating systems which basically do >> 'memory clobbering' for ensuring a compiler barrier, but actually they >> often forsee such a barrier without the conjuction of a memory >> operand. >> >> I think I will need to speak a bit with a GCC engineer in order to see >> what do they implement in regard of volatile operands. > > GCC can be quite aggressive with reordering even in the face of volatile. I > was recently doing a hack to export some data from the kernel to userland > that used a spin loop to grab a snapshot of the contents of a structure > similar to the method used in the kernel with the timehands structures. It > used a volatile structure exposed from the kernel that looked something > like: > > struct foo { > volatile int gen; > /* other stuff */ > }; > > volatile struct foo *p; > > do { > x = p->gen; > /* read other stuff */ > y = p->gen; > } while (x != y && x != 0); > > GCC moved the 'y = ' up into the middle of the '/* read other stuff */'. > I eventually had to add explicit "memory" clobbers to force GCC to not > move the reads of 'gen' around but do them "around" all the other > operations, so that the working code is: > > do { > x = p->gen; > asm volatile("" ::: "memory"); > /* read other stuff */ > asm volatile("" ::: "memory"); > y = p->gen; > } while (x != y && x != 0); > I see. So probabilly clobbering memory is the only choice we have right now. I will try to make a patch which also keeps into account the possibility to skip it (or define by hand alternative approaches) for different compilers. I wonder, specifically, how llvm/clang relies with it. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 30 13:07:28 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A41DC106568D; Wed, 30 Sep 2009 13:07:28 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 629408FC29; Wed, 30 Sep 2009 13:07:28 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id F170746B0C; Wed, 30 Sep 2009 09:07:27 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id E8E848A01B; Wed, 30 Sep 2009 09:07:26 -0400 (EDT) From: John Baldwin To: Attilio Rao Date: Wed, 30 Sep 2009 07:59:28 -0400 User-Agent: KMail/1.9.7 References: <20090924224935.GW473@gandalf.sssup.it> <200909291731.32394.jhb@freebsd.org> <3bbf2fe10909291439x21f53e34n60d63554b1dea0de@mail.gmail.com> In-Reply-To: <3bbf2fe10909291439x21f53e34n60d63554b1dea0de@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200909300759.29141.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 30 Sep 2009 09:07:26 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Ed Schouten , Max Laier , Roman Divacky , Fabio Checconi , freebsd-hackers@freebsd.org Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Sep 2009 13:07:28 -0000 On Tuesday 29 September 2009 5:39:43 pm Attilio Rao wrote: > 2009/9/29 John Baldwin : > > On Tuesday 29 September 2009 4:42:13 pm Attilio Rao wrote: > >> 2009/9/29 Max Laier : > >> > On Tuesday 29 September 2009 17:39:37 Attilio Rao wrote: > >> >> 2009/9/25 Fabio Checconi : > >> >> > Hi all, > >> >> > looking at sys/sx.h I have some troubles understanding this comment: > >> >> > > >> >> > * A note about memory barriers. Exclusive locks need to use the same > >> >> > * memory barriers as mutexes: _acq when acquiring an exclusive lock > >> >> > * and _rel when releasing an exclusive lock. On the other side, > >> >> > * shared lock needs to use an _acq barrier when acquiring the lock > >> >> > * but, since they don't update any locked data, no memory barrier is > >> >> > * needed when releasing a shared lock. > >> >> > > >> >> > In particular, I'm not understanding what prevents the following sequence > >> >> > from happening: > >> >> > > >> >> > CPU A CPU B > >> >> > > >> >> > sx_slock(&data->lock); > >> >> > > >> >> > sx_sunlock(&data->lock); > >> >> > > >> >> > /* reordered after the unlock > >> >> > by the cpu */ > >> >> > if (data->buffer) > >> >> > sx_xlock(&data->lock); > >> >> > free(data->buffer); > >> >> > data->buffer = NULL; > >> >> > sx_xunlock(&data->lock); > >> >> > > >> >> > a = *data->buffer; > >> >> > > >> >> > IOW, even if readers do not modify the data protected by the lock, > >> >> > without a release barrier a memory access may leak past the unlock (as > >> >> > the cpu won't notice any dependency between the unlock and the fetch, > >> >> > feeling free to reorder them), thus potentially racing with an exclusive > >> >> > writer accessing the data. > >> >> > > >> >> > On architectures where atomic ops serialize memory accesses this would > >> >> > never happen, otherwise the sequence above seems possible; am I missing > >> >> > something? > >> >> > >> >> I think your concerns are right, possibly we need this patch: > >> >> http://www.freebsd.org/~attilio/sxrw_unlockb.diff > >> >> > >> >> However speaking with John we agreed possibly there is a more serious > >> >> breakage. Possibly, memory barriers would also require to ensure the > >> >> compiler to not reorder the operation, while right now, in FreeBSD, they > >> >> just take care of the reordering from the architecture perspective. > >> >> The only way I'm aware of GCC offers that is to clobber memory. > >> >> I will provide a patch that address this soon, hoping that GCC will be > >> >> smart enough to not overhead too much the memory clobbering but just > >> >> try to understand what's our purpose and servers it (I will try to > >> >> compare code generated before and after the patch at least for tier-1 > >> >> architectures). > >> > > >> > Does GCC really reorder accesses to volatile objects? The C Standard seems to > >> > object: > >> > > >> > 5.1.2.3 - 2 > >> > Accessing a volatile object, modifying an object, modifying a file, or calling > >> > a function that does any of those operations are all side effects,11) which > >> > are changes in the state of the execution environment. Evaluation of an > >> > expression may produce side effects. At certain specified points in the > >> > execution sequence called sequence points, all side effects of previous > >> > evaluations shall be complete and no side effects of subsequent evaluations > >> > shall have taken place. (A summary of the sequence points is given in annex > >> > C.) > >> > >> Very interesting. > >> I was thinking about the other operating systems which basically do > >> 'memory clobbering' for ensuring a compiler barrier, but actually they > >> often forsee such a barrier without the conjuction of a memory > >> operand. > >> > >> I think I will need to speak a bit with a GCC engineer in order to see > >> what do they implement in regard of volatile operands. > > > > GCC can be quite aggressive with reordering even in the face of volatile. I > > was recently doing a hack to export some data from the kernel to userland > > that used a spin loop to grab a snapshot of the contents of a structure > > similar to the method used in the kernel with the timehands structures. It > > used a volatile structure exposed from the kernel that looked something > > like: > > > > struct foo { > > volatile int gen; > > /* other stuff */ > > }; > > > > volatile struct foo *p; > > > > do { > > x = p->gen; > > /* read other stuff */ > > y = p->gen; > > } while (x != y && x != 0); > > > > GCC moved the 'y = ' up into the middle of the '/* read other stuff */'. > > I eventually had to add explicit "memory" clobbers to force GCC to not > > move the reads of 'gen' around but do them "around" all the other > > operations, so that the working code is: > > > > do { > > x = p->gen; > > asm volatile("" ::: "memory"); > > /* read other stuff */ > > asm volatile("" ::: "memory"); > > y = p->gen; > > } while (x != y && x != 0); > > > > I see. > So probabilly clobbering memory is the only choice we have right now. > I will try to make a patch which also keeps into account the > possibility to skip it (or define by hand alternative approaches) for > different compilers. > I wonder, specifically, how llvm/clang relies with it. We already allow for different compilers to defined different versions of atomic_*(). I think all you need to do for now is just add "memory" clobbers to all of the atomic operations that have either an _acq or _rel memory barrier. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 1 09:03:19 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4DCB8106566B for ; Thu, 1 Oct 2009 09:03:19 +0000 (UTC) (envelope-from ivan.anyukov@googlemail.com) Received: from mail-fx0-f222.google.com (mail-fx0-f222.google.com [209.85.220.222]) by mx1.freebsd.org (Postfix) with ESMTP id C8FB08FC22 for ; Thu, 1 Oct 2009 09:03:18 +0000 (UTC) Received: by fxm22 with SMTP id 22so2632760fxm.36 for ; Thu, 01 Oct 2009 02:03:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=jRCv23ydtH5VlmkeFKYgr3pJoG1GWZFqobKduoFq5OM=; b=Il7jfoUfrPnpUNhHTDPsZP3u5kbb60gcaHE93R6qqaaqFSOGPkNtlOVzQkOaS6sv8p /5W7kZo7fF9xMajUZeuBpeKip3UEZWWlAyb7PJ2art9dFtBfHyGRzootWxnCDK7Oe6p9 Dy6rJTRYF8Ijf5SxOt2/udmMZ5gLmZHvj8GSI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=A/XJgCfLkNGANobu8H7L+afrI7+zBL9YzHzhhYql4sIrYhyYHiVnpVJ7VJDkyAABQg losdCIYnmFw5Q5icddjkyrVDcHlFWQsVW2kOnXwpDIBo0FG4UKX1v37jKyoESLuvmwnq ggc+5EeJ95mOe7WXsanOODTAc2X5IBTV/ZayU= MIME-Version: 1.0 Received: by 10.223.23.80 with SMTP id q16mr225967fab.8.1254387797741; Thu, 01 Oct 2009 02:03:17 -0700 (PDT) Date: Thu, 1 Oct 2009 11:03:17 +0200 Message-ID: <962bea8e0910010203o26dd0b02ob3e496759d2e28dc@mail.gmail.com> From: ivan anyukov To: freebsd-hackers@freebsd.org X-Mailman-Approved-At: Thu, 01 Oct 2009 11:34:41 +0000 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: NMI with 7.2-stable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Oct 2009 09:03:19 -0000 Hi guys, I'm running 7.2-STABLE on a Thinkpad T60. When connecting a second monitor to my docking station sometimes my FreeBSD freezes. kgdb on the vmcore-file says "non-maskable interrupt trap" Some details: X.Org 1.5.3 using the radeon-Driver I think the problem appears when moving xterms from the first to the second monitor (or back). The mouse cursor looks _very_ strange then and after some minutes the whole system freezes. Does anyone know about the problem? Is it a hardware-failure for sure? A vmcore is generated, I loaded it into kgdb: This GDB was configured as "i386-marcel-freebsd"... Unread portion of the kernel message buffer: <<22>NMI>N MIIS AI SA b0b, 0E,I ESIAS A ffff < 2>R<2 >RA<2M >ApMa rpiatyr ietryr oerr,r olri,k elliyk ehlayr dhwaarrdew afraei lfuariel.ure. FFaattaall ttrraapp 1199:: nnoonn--mmaasskkaabbllee iinntteerrrruupptt ttrraapp wwhhiillee iinn kkeerrnneell mmooddee ccppuuiidd == 01;; aappiicc iidd == 0010 iinnssttrruuccttiioonn ppooiinntteerr == 00xx2200::00xxcc00eeaa7733eecc ssttaacckk ppooiinntteerr == 00xx2288::00xxcc4477aa58cc8888 ffrraammee ppooiinntteerr == 00xx2288::00xxcc4477aa58cc8888 ccooddee sseeggmmeenntt == bbaassee 00xx00,, lliimmiitt 00xxffffffffff,, ttyyppee 00xx11bb == DDPPLL 00,, pprreess 11,, ddeeff3322 11,, ggrraann 11 pprroocceessssoorr eeffllaaggss == iinntteerrrruupptt eennaabblleedd,, IIOOPPLL == 00 ccuurrrreenntt pprroocceessss == 1112 ((iiddllee:: ccppuu10)) ttrraapp nnuummbbeerr == 1199 backtrace says: #0 doadump () at pcpu.h:196 #1 0xc07c5258 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #2 0xc07c5535 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:574 #3 0xc0ab2d14 in trap_fatal (frame=0xc47a5c48, eva=0) at /usr/src/sys/i386/i386/trap.c:939 #4 0xc0ab3a9d in trap (frame=0xc47a5c48) at /usr/src/sys/i386/i386/trap.c:726 #5 0xc0a9910b in calltrap () at /usr/src/sys/i386/i386/exception.s:159 #6 0xc0ea73ec in acpi_cpu_c1 () at /usr/src/sys/modules/acpi/acpi/../../../i386/acpica/acpi_machdep.c:550 #7 0xc0ea0623 in acpi_cpu_idle () at /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/acpi_cpu.c:943 #8 0xc0aa377b in cpu_idle () at /usr/src/sys/i386/i386/machdep.c:1183 #9 0xc07e6bd4 in sched_idletd (dummy=0x0) at /usr/src/sys/kern/sched_ule.c:2681 #10 0xc07a1311 in fork_exit (callout=0xc07e690f , arg=0x0, frame=0xc47a5d38) at /usr/src/sys/kern/kern_fork.c:810 #11 0xc0a99180 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:264 Any ideas? Thanks a lot! From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 1 13:27:42 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 73DD61065672 for ; Thu, 1 Oct 2009 13:27:42 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-fx0-f222.google.com (mail-fx0-f222.google.com [209.85.220.222]) by mx1.freebsd.org (Postfix) with ESMTP id 01EDB8FC1B for ; Thu, 1 Oct 2009 13:27:41 +0000 (UTC) Received: by fxm22 with SMTP id 22so135186fxm.36 for ; Thu, 01 Oct 2009 06:27:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=I/MFBJ/TqtAe/7+q3XRdABGuCimhONP8xrYOzn7zpog=; b=NSU3q3OIVAPu5dgpHhxKddyR7P92tqKugSNMScjQfNcnaH3OVu70yUCPe9LuDkU8Kx F2GyO9Xb4fZXautVDJPIaBcmp7zhf43crbe2MtMcBa8pVzyxe1cF6RmIpkl5MglWhiy6 G4SKjBdPLzs4rKUDUVwAE7ToUW/LVxGpKpNmM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=cQ93g54WEUTmhDoJ2QYc28zQAcfKaJZj32X4GPKHhjogQ0lRfLY8uh5gKpggRMIyCz ogq0ibg8nPMXK6eJGwXDRkdf1Y7ANqI4KN8YrZ5FkgOw2QRaBKc/PRAgeAXMHg47GROl g3VNY8RMMlMdocWYaOMdIcHAmLEVXcrppTkDA= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.54.15 with SMTP id o15mr303091fag.96.1254403660753; Thu, 01 Oct 2009 06:27:40 -0700 (PDT) In-Reply-To: <962bea8e0910010203o26dd0b02ob3e496759d2e28dc@mail.gmail.com> References: <962bea8e0910010203o26dd0b02ob3e496759d2e28dc@mail.gmail.com> Date: Thu, 1 Oct 2009 15:27:40 +0200 X-Google-Sender-Auth: 7588fd37f96cd105 Message-ID: <3bbf2fe10910010627p6136cf91u42601fa9bc291cfd@mail.gmail.com> From: Attilio Rao To: ivan anyukov Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org Subject: Re: NMI with 7.2-stable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Oct 2009 13:27:42 -0000 Can you send a full dmesg? 2009/10/1 ivan anyukov : > Hi guys, > I'm running 7.2-STABLE on a Thinkpad T60. > When connecting a second monitor to my docking station sometimes my FreeBSD > freezes. > kgdb on the vmcore-file says "non-maskable interrupt trap" > Some details: > X.Org 1.5.3 using the radeon-Driver > I think the problem appears when moving xterms from the first to the second > monitor (or back). The mouse cursor looks _very_ strange then and after some > minutes the whole system freezes. > Does anyone know about the problem? Is it a hardware-failure for sure? > > A vmcore is generated, I loaded it into kgdb: > > This GDB was configured as "i386-marcel-freebsd"... > > Unread portion of the kernel message buffer: > <<22>NMI>N MIIS AI SA b0b, 0E,I ESIAS A ffff > < > 2>R<2 >>RA<2M >ApMa rpiatyr ietryr oerr,r olri,k elliyk ehlayr dhwaarrdew afraei > lfuariel.ure. > > > > FFaattaall ttrraapp 1199:: nnoonn--mmaasskkaabbllee iinntteerrrruupptt > ttrraapp wwhhiillee iinn kkeerrnneell mmooddee > > ccppuuiidd == 01;; aappiicc iidd == 0010 > > iinnssttrruuccttiioonn ppooiinntteerr == > 00xx2200::00xxcc00eeaa7733eecc > > ssttaacckk ppooiinntteerr == > 00xx2288::00xxcc4477aa58cc8888 > > ffrraammee ppooiinntteerr == > 00xx2288::00xxcc4477aa58cc8888 > > ccooddee sseeggmmeenntt == bbaassee > 00xx00,, lliimmiitt 00xxffffffffff,, ttyyppee 00xx11bb > > == DDPPLL 00,, pprreess > 11,, ddeeff3322 11,, ggrraann 11 > > pprroocceessssoorr eeffllaaggss == iinntteerrrruupptt > eennaabblleedd,, IIOOPPLL == 00 > > ccuurrrreenntt pprroocceessss == 1112 > ((iiddllee:: ccppuu10)) > > ttrraapp nnuummbbeerr == 1199 > > backtrace says: > #0 doadump () at pcpu.h:196 > #1 0xc07c5258 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 > #2 0xc07c5535 in panic (fmt=Variable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:574 > #3 0xc0ab2d14 in trap_fatal (frame=0xc47a5c48, eva=0) at > /usr/src/sys/i386/i386/trap.c:939 > #4 0xc0ab3a9d in trap (frame=0xc47a5c48) at > /usr/src/sys/i386/i386/trap.c:726 > #5 0xc0a9910b in calltrap () at /usr/src/sys/i386/i386/exception.s:159 > #6 0xc0ea73ec in acpi_cpu_c1 () at > /usr/src/sys/modules/acpi/acpi/../../../i386/acpica/acpi_machdep.c:550 > #7 0xc0ea0623 in acpi_cpu_idle () at > /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/acpi_cpu.c:943 > #8 0xc0aa377b in cpu_idle () at /usr/src/sys/i386/i386/machdep.c:1183 > #9 0xc07e6bd4 in sched_idletd (dummy=0x0) at > /usr/src/sys/kern/sched_ule.c:2681 > #10 0xc07a1311 in fork_exit (callout=0xc07e690f , arg=0x0, > frame=0xc47a5d38) > at /usr/src/sys/kern/kern_fork.c:810 > #11 0xc0a99180 in fork_trampoline () at > /usr/src/sys/i386/i386/exception.s:264 > > Any ideas? > > Thanks a lot! > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 1 15:31:34 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F0B221065697 for ; Thu, 1 Oct 2009 15:31:33 +0000 (UTC) (envelope-from ivan.anyukov@googlemail.com) Received: from mail-bw0-f227.google.com (mail-bw0-f227.google.com [209.85.218.227]) by mx1.freebsd.org (Postfix) with ESMTP id CA6618FC08 for ; Thu, 1 Oct 2009 15:31:32 +0000 (UTC) Received: by bwz27 with SMTP id 27so214727bwz.43 for ; Thu, 01 Oct 2009 08:31:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=C5zY+lzdF3dcJ4wjNlBbU2PfRdTe58g2VPtbyBtO+3U=; b=DWZAcCPe+38A2bKa0vANcOjYusJ9oTCEn7ArciuFCPMIS4T5jX9zCJhc6AQAIwni/6 /G55U9Ml3ik/5AMTxXz5Y8Bk8a+ObmSXr9mZh1DHz2QkAJza35UR8LK97RpseqcVslSU T/785DUbL+FeVkarsMSnLnBQZhO6pavFwM2+8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=g2j02Ujy2sfbQBci1Ox39+0EdcvQMT1E0IWp1R5/UClzDH0UKnq4v4U3SKEsdUNfix OV1tofG85ACsOlXaxS6kxWBuc3plJCdZ0+a8sinmwF4conT7kN02XW+9BUGxaMxeBXYx mi/8Ai6b/M4cwhfmGv4tMe6iPjKNCVJdGTx1k= MIME-Version: 1.0 Received: by 10.223.1.6 with SMTP id 6mr365496fad.103.1254411091476; Thu, 01 Oct 2009 08:31:31 -0700 (PDT) In-Reply-To: <3bbf2fe10910010627p6136cf91u42601fa9bc291cfd@mail.gmail.com> References: <962bea8e0910010203o26dd0b02ob3e496759d2e28dc@mail.gmail.com> <3bbf2fe10910010627p6136cf91u42601fa9bc291cfd@mail.gmail.com> Date: Thu, 1 Oct 2009 17:31:31 +0200 Message-ID: <962bea8e0910010831o2d39e0ecvb2fef29ed2f20954@mail.gmail.com> From: ivan anyukov To: Attilio Rao X-Mailman-Approved-At: Thu, 01 Oct 2009 15:38:23 +0000 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: NMI with 7.2-stable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Oct 2009 15:31:34 -0000 dmesg goes here: Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.2-STABLE #12: Wed Aug 26 23:21:46 CEST 2009 root@fnord:/usr/obj/usr/src/sys/GENERIC Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Core(TM) Duo CPU T2400 @ 1.83GHz (1828.76-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x6ec Stepping = 12 Features=0xbfe9fbff Features2=0xc1a9 AMD Features=0x100000 Cores per package: 2 real memory = 1609367552 (1534 MB) avail memory = 1563168768 (1490 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 ACPI Warning (tbfadt-0505): Optional field "Gpe1Block" has zero address or length: 0 102C/0 [20070320] ioapic0: Changing APIC ID to 1 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] acpi_ec0: port 0x62,0x66 on acpi0 acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 5ff00000 (3) failed Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 acpi_lid0: on acpi0 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 16 at device 1.0 on pci0 pci1: on pcib1 vgapci0: port 0x2000-0x20ff mem 0xd8000000-0xdfffffff,0xee100000-0xee10ffff irq 16 at device 0.0 on pci1 hdac0: mem 0xee400000-0xee403fff irq 17 at device 27.0 on pci0 hdac0: HDA Driver Revision: 20090329_0131 hdac0: [ITHREAD] pcib2: irq 20 at device 28.0 on pci0 pci2: on pcib2 em0: port 0x3000-0x301f mem 0xee000000-0xee01ffff irq 16 at device 0.0 on pci2 em0: Using MSI interrupt em0: [FILTER] em0: Ethernet address: 00:1e:37:cb:53:b3 pcib3: irq 21 at device 28.1 on pci0 pci3: on pcib3 wpi0: mem 0xedf00000-0xedf00fff irq 17 at device 0.0 on pci3 wpi0: Ethernet address: 00:18:de:9d:0e:8f wpi0: [ITHREAD] pcib4: irq 22 at device 28.2 on pci0 pci4: on pcib4 pcib5: irq 23 at device 28.3 on pci0 pci12: on pcib5 uhci0: port 0x1800-0x181f irq 16 at device 29.0 on pci0 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: on uhci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 2 ports with 2 removable, self powered uhci1: port 0x1820-0x183f irq 17 at device 29.1 on pci0 uhci1: [GIANT-LOCKED] uhci1: [ITHREAD] usb1: on uhci1 usb1: USB revision 1.0 uhub1: on usb1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0x1840-0x185f irq 18 at device 29.2 on pci0 uhci2: [GIANT-LOCKED] uhci2: [ITHREAD] usb2: on uhci2 usb2: USB revision 1.0 uhub2: on usb2 uhub2: 2 ports with 2 removable, self powered uhci3: port 0x1860-0x187f irq 19 at device 29.3 on pci0 uhci3: [GIANT-LOCKED] uhci3: [ITHREAD] usb3: on uhci3 usb3: USB revision 1.0 uhub3: on usb3 uhub3: 2 ports with 2 removable, self powered ehci0: mem 0xee404000-0xee4043ff irq 19 at device 29.7 on pci0 ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb4: EHCI version 1.0 usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4: on ehci0 usb4: USB revision 2.0 uhub4: on usb4 uhub4: 8 ports with 8 removable, self powered uhub5: on uhub4 uhub5: multiple transaction translators uhub5: 4 ports with 4 removable, self powered ums0: on uhub5 ums0: 3 buttons and Z dir. ukbd0: on uhub5 kbd2 at ukbd0 uhid0: on uhub5 pcib6: at device 30.0 on pci0 pci21: on pcib6 cbb0: mem 0xe4300000-0xe4300fff irq 16 at device 0.0 on pci21 cardbus0: on cbb0 pccard0: <16-bit PCCard bus> on cbb0 cbb0: [ITHREAD] isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1880-0x188f at device 31.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] atapci1: port 0x18c8-0x18cf,0x18ac-0x18af,0x18c0-0x18c7,0x18a8-0x18ab,0x18b0-0x18bf mem 0xee404400-0xee4047ff irq 16 at device 31.2 on pci0 atapci1: [ITHREAD] atapci1: AHCI called from vendor specific driver atapci1: AHCI Version 01.10 controller with 4 ports detected ata2: on atapci1 ata2: [ITHREAD] ata3: on atapci1 ata3: port not implemented ata3: [ITHREAD] ata4: on atapci1 ata4: port not implemented ata4: [ITHREAD] ata5: on atapci1 ata5: port not implemented ata5: [ITHREAD] pci0: at device 31.3 (no driver attached) acpi_tz0: on acpi0 acpi_tz1: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model Generic PS/2 mouse, device ID 0 sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled battery0: on acpi0 acpi_acad0: on acpi0 cpu0: on acpi0 est0: on cpu0 p4tcc0: on cpu0 cpu1: on acpi0 est1: on cpu1 p4tcc1: on cpu1 sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled pmtimer0 on isa0 orm0: at iomem 0xd0000-0xd0fff,0xd1000-0xd1fff,0xdc000-0xdffff,0xe0000-0xeffff pnpid ORM0000 on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 8250 or not responding sio0: [FILTER] sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ugen0: on uhub3 Timecounters tick every 1.000 msec acd0: CDRW at ata0-master UDMA33 ad4: 476940MB at ata2-master SATA150 hdac0: HDA Codec #0: Analog Devices AD1981HD hdac0: HDA Codec #1: Conexant (Unknown) pcm0: at cad 0 nid 1 on hdac0 pcm1: at cad 0 nid 1 on hdac0 SMP: AP CPU #1 Launched! GEOM_LABEL: Label for provider ad4s1a is ufsid/427627623762bv3. GEOM_LABEL: Label for provider ad4s1d is ufsid/427627623762bv3 GEOM_LABEL: Label for provider ad4s1e is ufsid/427627623762bv3. GEOM_LABEL: Label for provider ad4s1f is ufsid/427627623762bv3. Trying to mount root from ufs:/dev/ad4s1a cryptosoft0: on motherboard GEOM_ELI: Device ad4s1g.eli created. GEOM_ELI: Encryption: AES-CBC 128 GEOM_ELI: Crypto: software GEOM_LABEL: Label for provider ad4s1g.eli is ufsid/49863f. GEOM_LABEL: Label ufsid/45 removed. GEOM_LABEL: Label for provider ad4s1a is ufsid/49835. [...GEOM-stuff] drm0: on vgapci0 info: [drm] MSI enabled 1 message(s) vgapci0: child drm0 requested pci_enable_busmaster info: [drm] Initialized radeon 1.29.0 20080528 info: [drm] Setting GART location based on new memory map info: [drm] Loading R500 Microcode info: [drm] Num pipes: 1 info: [drm] writeback test succeeded in 1 usecs drm0: [ITHREAD] 2009/10/1 Attilio Rao > Can you send a full dmesg? > > 2009/10/1 ivan anyukov : > > Hi guys, > > I'm running 7.2-STABLE on a Thinkpad T60. > > When connecting a second monitor to my docking station sometimes my > FreeBSD > > freezes. > > kgdb on the vmcore-file says "non-maskable interrupt trap" > > Some details: > > X.Org 1.5.3 using the radeon-Driver > > I think the problem appears when moving xterms from the first to the > second > > monitor (or back). The mouse cursor looks _very_ strange then and after > some > > minutes the whole system freezes. > > Does anyone know about the problem? Is it a hardware-failure for sure? > > > > A vmcore is generated, I loaded it into kgdb: > > > > This GDB was configured as "i386-marcel-freebsd"... > > > > Unread portion of the kernel message buffer: > > <<22>NMI>N MIIS AI SA b0b, 0E,I ESIAS A ffff > > < > > 2>R<2 > >>RA<2M >ApMa rpiatyr ietryr oerr,r olri,k elliyk ehlayr dhwaarrdew afraei > > lfuariel.ure. > > > > > > > > FFaattaall ttrraapp 1199:: nnoonn--mmaasskkaabbllee > iinntteerrrruupptt > > ttrraapp wwhhiillee iinn kkeerrnneell mmooddee > > > > ccppuuiidd == 01;; aappiicc iidd == 0010 > > > > iinnssttrruuccttiioonn ppooiinntteerr == > > 00xx2200::00xxcc00eeaa7733eecc > > > > ssttaacckk ppooiinntteerr == > > 00xx2288::00xxcc4477aa58cc8888 > > > > ffrraammee ppooiinntteerr == > > 00xx2288::00xxcc4477aa58cc8888 > > > > ccooddee sseeggmmeenntt == bbaassee > > 00xx00,, lliimmiitt 00xxffffffffff,, ttyyppee 00xx11bb > > > > == DDPPLL 00,, pprreess > > 11,, ddeeff3322 11,, ggrraann 11 > > > > pprroocceessssoorr eeffllaaggss == iinntteerrrruupptt > > eennaabblleedd,, IIOOPPLL == 00 > > > > ccuurrrreenntt pprroocceessss == 1112 > > ((iiddllee:: ccppuu10)) > > > > ttrraapp nnuummbbeerr == 1199 > > > > backtrace says: > > #0 doadump () at pcpu.h:196 > > #1 0xc07c5258 in boot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:418 > > #2 0xc07c5535 in panic (fmt=Variable "fmt" is not available. > > ) at /usr/src/sys/kern/kern_shutdown.c:574 > > #3 0xc0ab2d14 in trap_fatal (frame=0xc47a5c48, eva=0) at > > /usr/src/sys/i386/i386/trap.c:939 > > #4 0xc0ab3a9d in trap (frame=0xc47a5c48) at > > /usr/src/sys/i386/i386/trap.c:726 > > #5 0xc0a9910b in calltrap () at /usr/src/sys/i386/i386/exception.s:159 > > #6 0xc0ea73ec in acpi_cpu_c1 () at > > /usr/src/sys/modules/acpi/acpi/../../../i386/acpica/acpi_machdep.c:550 > > #7 0xc0ea0623 in acpi_cpu_idle () at > > /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/acpi_cpu.c:943 > > #8 0xc0aa377b in cpu_idle () at /usr/src/sys/i386/i386/machdep.c:1183 > > #9 0xc07e6bd4 in sched_idletd (dummy=0x0) at > > /usr/src/sys/kern/sched_ule.c:2681 > > #10 0xc07a1311 in fork_exit (callout=0xc07e690f , arg=0x0, > > frame=0xc47a5d38) > > at /usr/src/sys/kern/kern_fork.c:810 > > #11 0xc0a99180 in fork_trampoline () at > > /usr/src/sys/i386/i386/exception.s:264 > > > > Any ideas? > > > > Thanks a lot! > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > > > > > > -- > Peace can only be achieved by understanding - A. Einstein > From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 1 16:11:16 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3DD1106566B for ; Thu, 1 Oct 2009 16:11:16 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-fx0-f222.google.com (mail-fx0-f222.google.com [209.85.220.222]) by mx1.freebsd.org (Postfix) with ESMTP id 568148FC0C for ; Thu, 1 Oct 2009 16:11:15 +0000 (UTC) Received: by fxm22 with SMTP id 22so290811fxm.36 for ; Thu, 01 Oct 2009 09:11:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=emA99zFolk7cOFxuFuH6smTHY4PDic/I0WSIQAwvQUg=; b=hYQj/SatkIDnUWa+H2kYJWkfXv8Wq6UyHkyFqh7yl1lYdrHf7VYo0YdNcYs458sdR+ i+NbHf4GVior6o1/Oad7OKZeEZixkRc4cA3se5LMUP5cW5ih5yaln04m47JUho5kEVHk glakMHk+yxL4y0PlOm6imHIY+KnxlHr4DMUYQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=CLJvuB24n0RLdf1C79fEyT+6ZZhQwTAC61XDyr4U4rh2ps2KwPf89BCqwF+Sz+sVd3 vu4GrnvENukavp1Sk8IZFbxblpejT86Ol0fdwic99bVMynz3Amaz4Ouh4OpNvHQTPha/ 4VelmoM85IhSygkFLPlll0DrS9c7WzQA+EJmE= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.62.146 with SMTP id x18mr411790fah.48.1254413475280; Thu, 01 Oct 2009 09:11:15 -0700 (PDT) In-Reply-To: <962bea8e0910010831o2d39e0ecvb2fef29ed2f20954@mail.gmail.com> References: <962bea8e0910010203o26dd0b02ob3e496759d2e28dc@mail.gmail.com> <3bbf2fe10910010627p6136cf91u42601fa9bc291cfd@mail.gmail.com> <962bea8e0910010831o2d39e0ecvb2fef29ed2f20954@mail.gmail.com> Date: Thu, 1 Oct 2009 18:11:15 +0200 X-Google-Sender-Auth: 1e5b2b736ca986ee Message-ID: <3bbf2fe10910010911v4db0fbdar9f79ea712c734a23@mail.gmail.com> From: Attilio Rao To: ivan anyukov Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org Subject: Re: NMI with 7.2-stable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Oct 2009 16:11:16 -0000 2009/10/1 ivan anyukov : > dmesg goes here: > Yes, I think it is an hw failure. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 2 20:34:30 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A0E4310656A3 for ; Fri, 2 Oct 2009 20:34:30 +0000 (UTC) (envelope-from reg@openpave.org) Received: from mx2.ucdavis.edu (mx2.ucdavis.edu [128.120.32.32]) by mx1.freebsd.org (Postfix) with ESMTP id 873DB8FC25 for ; Fri, 2 Oct 2009 20:34:30 +0000 (UTC) Received: from flint.openpave.org ([169.237.179.18]) by mx2.ucdavis.edu (8.13.7/8.13.1/it-defang-5.4.0) with ESMTP id n92KArj9007693 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 2 Oct 2009 13:10:54 -0700 (PDT) Received: from sandy.local (flint.local [169.237.179.18]) by flint.openpave.org (8.14.3/8.14.3) with ESMTP id n92KAd2N053338 for ; Fri, 2 Oct 2009 20:10:39 GMT (envelope-from reg@sandy.local) Received: (from reg@localhost) by sandy.local (8.14.3/8.14.3/Submit) id n92KAdgr053337 for freebsd-hackers@freebsd.org; Fri, 2 Oct 2009 13:10:39 -0700 (PDT) (envelope-from reg) Date: Fri, 2 Oct 2009 13:10:39 -0700 From: Jeremy Lea To: freebsd-hackers@freebsd.org Message-ID: <20091002201039.GA53034@flint.openpave.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.1 at av7 X-Virus-Status: Clean X-Scanned-By: MIMEDefang 2.57 on 128.120.32.32 Subject: Distributed SSH attack X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Oct 2009 20:34:30 -0000 Hi, This is off topic to this list, but I dont want to subscribe to -chat just to post there... Someone is currently running a distributed SSH attack against one of my boxes - one attempted login for root every minute or so for the last 48 hours. They wont get anywhere, since the box in question has no root password, and doesn't allow root logins via SSH anyway... But I was wondering if there were any security researchers out there that might be interested in the +-800 IPs I've collected from the botnet? The resolvable hostnames mostly appear to be in Eastern Europe and South America - I haven't spotted any that might be 'findable' to get the botnet software. I could switch out the machine for a honeypot in a VM or a jail, by moving the host to a new IP, and if you can think of a way of allowing the next login to succeed with any password, then you could try to see what they delivered... But I don't have a lot of time to help. Regards, -Jeremy -- FreeBSD - Because the best things in life are free... http://www.freebsd.org/ From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 2 20:45:29 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D2A3F1065692; Fri, 2 Oct 2009 20:45:29 +0000 (UTC) (envelope-from anders@FreeBSD.org) Received: from fupp.net (totem.fix.no [80.91.36.20]) by mx1.freebsd.org (Postfix) with ESMTP id 8DB608FC19; Fri, 2 Oct 2009 20:45:29 +0000 (UTC) Received: from localhost (totem.fix.no [80.91.36.20]) by fupp.net (Postfix) with ESMTP id 109BC37BEB; Fri, 2 Oct 2009 22:29:37 +0200 (CEST) Received: from fupp.net ([80.91.36.20]) by localhost (totem.fix.no [80.91.36.20]) (amavisd-new, port 10024) with LMTP id Ce2yl1x2cgur; Fri, 2 Oct 2009 22:29:36 +0200 (CEST) Received: by fupp.net (Postfix, from userid 1000) id A66CB37BEA; Fri, 2 Oct 2009 22:29:36 +0200 (CEST) Date: Fri, 2 Oct 2009 22:29:36 +0200 From: Anders Nordby To: freebsd-hackers@freebsd.org Message-ID: <20091002202936.GA836@fupp.net> Mail-Followup-To: Anders Nordby , freebsd-hackers@freebsd.org, alc@FreeBSD.org, phk@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key: http://anders.fix.no/pgp/ X-PGP-Key-FingerPrint: 1E0F C53C D8DF 6A8F EAAD 19C5 D12A BC9F 0083 5956 Cc: alc@FreeBSD.org, phk@FreeBSD.org Subject: No end to swap zone exhausted problems? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Oct 2009 20:45:29 -0000 Hi, Better post about my swap zone problems here than tear all my hair out. This concerns me: 1) How can SWAPMETA usage continue increasing, while swap space used is not? Example: http://anders.fupp.net/test/swapmeta.png. In the swap meta graph, used space is USED and free space is LIMIT-USED (FREE numbers are a bit odd, it starts with 0) from the SWAPMETA line when running vmstat -z. In the week graph, you can see that after 3 days swap space usage settles at around 90 GB, while swap meta keeps increasing. Is there a leak? 2) By default, kern.maxswzone is set to 32 MB in FreeBSD: kern.maxswzone: 33554432 This leads to vmstat -z showing: ITEM SIZE LIMIT USED FREE REQUESTS FAILURES SWAPMETA: 288, 116519, 0, 0, 0, 0 According to vmstat man page, memory numbers are printed in kilobytes. Where is the relation between maxswzone and SWAPMETA? By booting with different settings for maxswzone and checking what SWAPMETA LIMIT turns out to be, I get this table: kern.maxswzone SWAPMETA LIMIT ============== ============== 32 MB 116519 64 MB 233025 128 MB 466037 256 MB 932074 512 MB 995410 1 GB 995410 2 GB 995410 How come increasing maxswzone beyond 256 MB does not yield a larger SWAPMETA than 995410? Does it stop around 260 MB maxswzone? How can I increase SWAPMETA further to be able to swap more? When maxswzone is set to 2 GB, the number goes negative: kern.maxswzone: -2147483648 I guess my luck is out? Some additional info: - When LIMIT-USED from SWAPMETA gets to 0, the system freezes with the dreaded "swap zone exhausted, increase kern.maxswzone" message. I monitor this. It's nice to know when your system will freeze. - KMEM settings are: vm.kmem_size_max: 3865468109 vm.kmem_size_min: 0 vm.kmem_size: 2718302208 Had no luck by tuning and increasing kmem_size and kmem_size_max, it mostly leads to crashes. - I am running FreeBSD/amd64 7.2-RELEASE, and I have 8 GB physical RAM. - By lowering kern.maxusers from 386 to 16 and trimming my kernel config down to a bare minimum I got SWAPMETA LIMIT increased to a whopping 995475, an increase of ~64 KB. I need more than this. - Why do I use so much swap space? I am running Varnish with malloc for storage, and I have a rather large dataset (6 million objects for this set of servers) with mostly 1 week expire time. Performance wise it works well, better than mmapping a large file which is/was the most common method for Varnish (this gives occasional ~2 minute freezes). Regards, -- Anders. From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 2 21:38:15 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 69B941065676; Fri, 2 Oct 2009 21:38:15 +0000 (UTC) (envelope-from glarkin@FreeBSD.org) Received: from mail1.sourcehosting.net (113901-app1.sourcehosting.net [72.32.213.11]) by mx1.freebsd.org (Postfix) with ESMTP id 4708F8FC1B; Fri, 2 Oct 2009 21:38:15 +0000 (UTC) Received: from 68-189-245-235.dhcp.oxfr.ma.charter.com ([68.189.245.235] helo=cube.entropy.prv) by mail1.sourcehosting.net with esmtp (Exim 4.69 (FreeBSD)) (envelope-from ) id 1MtpVp-0003UL-Os; Fri, 02 Oct 2009 17:18:06 -0400 Received: from [127.0.0.1] (fireball.entropy.prv [192.168.1.12]) by cube.entropy.prv (Postfix) with ESMTP id B9B66356643F; Fri, 2 Oct 2009 17:18:01 -0400 (EDT) Message-ID: <4AC66E07.4030605@FreeBSD.org> Date: Fri, 02 Oct 2009 17:17:59 -0400 From: Greg Larkin Organization: The FreeBSD Project User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: Jeremy Lea References: <20091002201039.GA53034@flint.openpave.org> In-Reply-To: <20091002201039.GA53034@flint.openpave.org> X-Enigmail-Version: 0.96.0 OpenPGP: id=1C940290 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Spam-Score: 0.1 (/) Cc: freebsd-hackers@freebsd.org Subject: Re: Distributed SSH attack X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: glarkin@FreeBSD.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Oct 2009 21:38:15 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jeremy Lea wrote: > Hi, > > This is off topic to this list, but I dont want to subscribe to -chat > just to post there... Someone is currently running a distributed SSH > attack against one of my boxes - one attempted login for root every > minute or so for the last 48 hours. They wont get anywhere, since the > box in question has no root password, and doesn't allow root logins via > SSH anyway... > > But I was wondering if there were any security researchers out there > that might be interested in the +-800 IPs I've collected from the > botnet? The resolvable hostnames mostly appear to be in Eastern Europe > and South America - I haven't spotted any that might be 'findable' to > get the botnet software. > > I could switch out the machine for a honeypot in a VM or a jail, by > moving the host to a new IP, and if you can think of a way of allowing > the next login to succeed with any password, then you could try to see > what they delivered... But I don't have a lot of time to help. > > Regards, > -Jeremy > Hi Jeremy, You could set up DenyHosts and contribute to the pool of IPs that are attempting SSH logins on the Net: http://denyhosts.sourceforge.net/faq.html#4_0 It also looks like there's been quite a spike of SSH login activity recently: http://stats.denyhosts.net/stats.html Hope that helps, Greg - -- Greg Larkin http://www.FreeBSD.org/ - The Power To Serve http://www.sourcehosting.net/ - Ready. Set. Code. http://twitter.com/sourcehosting/ - Follow me, follow you -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKxm4H0sRouByUApARAtnPAKCQuivQdE1s0ZZnUO6qVWA87N8ZKgCgjyYD Tbv+hWI+KoXYsEpt0n4gW5k= =xCz7 -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 2 23:10:45 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A9251065670; Fri, 2 Oct 2009 23:10:45 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-ew0-f209.google.com (mail-ew0-f209.google.com [209.85.219.209]) by mx1.freebsd.org (Postfix) with ESMTP id 4EA1F8FC1C; Fri, 2 Oct 2009 23:10:44 +0000 (UTC) Received: by ewy5 with SMTP id 5so598285ewy.36 for ; Fri, 02 Oct 2009 16:10:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=bFNkdaS3SNvp4NVAkAcptARzgA9wV70Yj54sYVorUfc=; b=mLUL9KidwDXFAkEESAIFkkRNYTkTHVOjGC3ZeAx1SF6oAcYHuSPq083H/onEBTGprb usKDmQcZAZpyYNjVNmTKyD0nrviJ7Uzx2Y4EmS/GJetlqZOwc3mFAlDwTyqQXVeFFSrS 2m8unDG4xy1NcaE2sGgWgsfNpBwWmpmt0c8VU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; b=qfu1HFGWblJBGfWWZk/YIU+1jpZh7f57sxTDllkSd1YJQMzsHAi4k/EXgMSkAdDfGT XIByyAXrwJrTrpoQT1d4berQ8cBn/2irWNSrUSFM0EyaU5LM5yuBnWXDElAUtVBHr1R3 co/Tkg8xZhk9QCCNsMF1AnJloPt6Smsy3+Zo8= Received: by 10.216.28.21 with SMTP id f21mr459522wea.39.1254523155180; Fri, 02 Oct 2009 15:39:15 -0700 (PDT) Received: from aryeh-desktop.istudentunion.com (ool-44c0cd7a.dyn.optonline.net [68.192.205.122]) by mx.google.com with ESMTPS id t12sm4519571gvd.13.2009.10.02.15.39.12 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 02 Oct 2009 15:39:13 -0700 (PDT) Message-ID: <4AC6810D.1030106@gmail.com> Date: Fri, 02 Oct 2009 18:39:09 -0400 From: "Aryeh M. Friedman" User-Agent: Thunderbird 2.0.0.23 (X11/20090915) MIME-Version: 1.0 To: glarkin@FreeBSD.org References: <20091002201039.GA53034@flint.openpave.org> <4AC66E07.4030605@FreeBSD.org> In-Reply-To: <4AC66E07.4030605@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Jeremy Lea , freebsd-hackers@freebsd.org Subject: Re: Distributed SSH attack X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Oct 2009 23:10:45 -0000 Greg Larkin wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Jeremy Lea wrote: > >> Hi, >> >> This is off topic to this list, but I dont want to subscribe to -chat >> just to post there... Someone is currently running a distributed SSH >> attack against one of my boxes - one attempted login for root every >> minute or so for the last 48 hours. They wont get anywhere, since the >> box in question has no root password, and doesn't allow root logins via >> SSH anyway... >> >> But I was wondering if there were any security researchers out there >> that might be interested in the +-800 IPs I've collected from the >> botnet? The resolvable hostnames mostly appear to be in Eastern Europe >> and South America - I haven't spotted any that might be 'findable' to >> get the botnet software. >> >> I could switch out the machine for a honeypot in a VM or a jail, by >> moving the host to a new IP, and if you can think of a way of allowing >> the next login to succeed with any password, then you could try to see >> what they delivered... But I don't have a lot of time to help. >> >> Regards, >> -Jeremy >> >> > > Hi Jeremy, > > You could set up DenyHosts and contribute to the pool of IPs that are > attempting SSH logins on the Net: > http://denyhosts.sourceforge.net/faq.html#4_0 > > It also looks like there's been quite a spike of SSH login activity > recently: http://stats.denyhosts.net/stats.html > > Hope that helps, > Greg > - -- > Greg Larkin > > http://www.FreeBSD.org/ - The Power To Serve > http://www.sourcehosting.net/ - Ready. Set. Code. > http://twitter.com/sourcehosting/ - Follow me, follow you > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.7 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iD8DBQFKxm4H0sRouByUApARAtnPAKCQuivQdE1s0ZZnUO6qVWA87N8ZKgCgjyYD > Tbv+hWI+KoXYsEpt0n4gW5k= > =xCz7 > -----END PGP SIGNATURE----- > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > There seems to be some kind of cordinated attack because I have been seeing different backbones wink in and out (work and home are on completely diff backbones and are having roughly the same intermitten interuptions) From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 2 23:47:46 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DA8901065679; Fri, 2 Oct 2009 23:47:46 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-qy0-f192.google.com (mail-qy0-f192.google.com [209.85.221.192]) by mx1.freebsd.org (Postfix) with ESMTP id 682B38FC18; Fri, 2 Oct 2009 23:47:46 +0000 (UTC) Received: by qyk30 with SMTP id 30so1779411qyk.7 for ; Fri, 02 Oct 2009 16:47:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:date:from:to:cc :subject:in-reply-to:message-id:references:user-agent :x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; bh=agTgDxpnsCVT9Rtl3yybxzualWgXR0NsnyotDWyuGx8=; b=QpF66uaC9b/4mQmERm7lrdsOSKr9ssTs+fXmWpru37T99dUyb/aOmuy0Y/FIcIQ7OR scVwlsBx8bNYc6ldjl724lGmWSmhQj+l6lxj/KOeHuWMGKiZdVX8zhdz6Z0uC+xtnGbF HRfmJ8keTILkC3W7fRuDZlyZ57iYgXKbOBfkk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; b=HqhzBy3REJEVMX5LJGiT2olRYH+J6OJckW8KdxVv5hdVgDtWfJwePz2nMAD/g+/4g8 GfyPpolU7Y9HSR+lGqXKywN+vJ0SM+QvFtgJMifX9ELp6XRV5fl3ZUskyRYBRdsqhCJD o0IaRQpQNizjrL6RoNFRaZxLdMVNp1fFTDpG8= Received: by 10.224.60.9 with SMTP id n9mr1650916qah.31.1254526065644; Fri, 02 Oct 2009 16:27:45 -0700 (PDT) Received: from dimension.5p.local ([99.35.15.84]) by mx.google.com with ESMTPS id 26sm371387qwa.50.2009.10.02.16.27.43 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 02 Oct 2009 16:27:44 -0700 (PDT) Sender: "J. Hellenthal" Date: Fri, 2 Oct 2009 19:27:33 -0400 From: jhell To: Greg Larkin In-Reply-To: <4AC66E07.4030605@FreeBSD.org> Message-ID: References: <20091002201039.GA53034@flint.openpave.org> <4AC66E07.4030605@FreeBSD.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-OpenPGP-Key-Id: 0x89D8547E X-OpenPGP-Key-Fingerprint: 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Cc: Jeremy Lea , freebsd-hackers@freebsd.org Subject: Re: Distributed SSH attack X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Oct 2009 23:47:46 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 2 Oct 2009 17:17 -0000, glarkin wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Jeremy Lea wrote: > > Hi, > > > > This is off topic to this list, but I dont want to subscribe to -chat > > just to post there... Someone is currently running a distributed SSH > > attack against one of my boxes - one attempted login for root every > > minute or so for the last 48 hours. They wont get anywhere, since the > > box in question has no root password, and doesn't allow root logins via > > SSH anyway... > > > > But I was wondering if there were any security researchers out there > > that might be interested in the +-800 IPs I've collected from the > > botnet? The resolvable hostnames mostly appear to be in Eastern Europe > > and South America - I haven't spotted any that might be 'findable' to > > get the botnet software. > > > > I could switch out the machine for a honeypot in a VM or a jail, by > > moving the host to a new IP, and if you can think of a way of allowing > > the next login to succeed with any password, then you could try to see > > what they delivered... But I don't have a lot of time to help. > > > > Regards, > > -Jeremy > > > > Hi Jeremy, > > You could set up DenyHosts and contribute to the pool of IPs that are > attempting SSH logins on the Net: > http://denyhosts.sourceforge.net/faq.html#4_0 > > It also looks like there's been quite a spike of SSH login activity > recently: http://stats.denyhosts.net/stats.html > > Hope that helps, > Greg > - -- > Greg Larkin > > http://www.FreeBSD.org/ - The Power To Serve > http://www.sourcehosting.net/ - Ready. Set. Code. > http://twitter.com/sourcehosting/ - Follow me, follow you > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.7 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iD8DBQFKxm4H0sRouByUApARAtnPAKCQuivQdE1s0ZZnUO6qVWA87N8ZKgCgjyYD > Tbv+hWI+KoXYsEpt0n4gW5k= > =xCz7 > -----END PGP SIGNATURE----- > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > Another temporary to long term solution might be the following utilities, ports/security/sshguard-pf ports/security/expiretable This is more of a pf based solution so that's up to your policies and decision. Giving thanks to the post about DenyHosts I didn't know that existed till this point. Best regards. - -- %{----------------------------------------------------+ | dataix.net!jhell 2048R/89D8547E 2009-09-30 | | BSD since FreeBSD 4.2 Linux since Slackware 2.1 | | 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E | +----------------------------------------------------%} -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.13 (FreeBSD) iQEcBAEBAgAGBQJKxoxtAAoJEJBXh4mJ2FR+BLQIAIm4nAh8TinDB/QOI6RX2xxO CSv46ZxoRlr2uv3FF5LmIVhPt0tskSrO+WLP0Xjm2ORB05tiFRpbzMBRawH41J1p 0USI90j+y9UzXinGRX9vt3GAofRkfuQuXXMUMAwTCZY1+EyzOP/K0dfRTSTj24LH 386epgCU3FA8S9UqKSPSdpQNxf+Yq/urd6ykfOTtcMUh/m2bakYIgwtVb4zOe+34 lpTlsXxuPcv9WtcOkqkj8LhZgFYKTRajfiw/G8cCnHqlaKuSDSH1hPEu7ePUAC5o wj6TZWh186astBg2WtfIke5zKKQz2ELyT5a3GvhWxR4/l9QWN5F0ZX7TuzaWK1M= =vtNQ -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 3 08:37:51 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E10F106568B for ; Sat, 3 Oct 2009 08:37:51 +0000 (UTC) (envelope-from jruohone@gmail.com) Received: from mail-ew0-f209.google.com (mail-ew0-f209.google.com [209.85.219.209]) by mx1.freebsd.org (Postfix) with ESMTP id EFFB78FC08 for ; Sat, 3 Oct 2009 08:37:50 +0000 (UTC) Received: by ewy5 with SMTP id 5so808469ewy.36 for ; Sat, 03 Oct 2009 01:37:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:received:date:from:to :subject:message-id:reply-to:mail-followup-to:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=ND/vV9/d8rnJsms4ZMUfGSR2LXysWF4BNoA0UXm/pys=; b=Pmzt3OfBgcQyRIlDNU8aghKvKIWjiYJRoLfehbFr0z3e5tmsGaUTWqDgZ6clzGT9Ly +gjg6UPwo8CI//FPwtk+R2onHDX86goVzESVngUSfJOjM2S3iGOJxMSspQ0bOHJ4wQth w3cfxk0jGqp8fMlmKnrOTNVASH//xL9rmxEpU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:subject:message-id:reply-to:mail-followup-to :references:mime-version:content-type:content-disposition :in-reply-to:user-agent; b=v6Tlo5gAZ92HqhaHdJiXkomo2b+HsbSK/wOJ4nsVIWEvACDx6DrICSwPHX6C+oeJXa iGE3+KWCTUt/dUtFEQMgc6yKjwESas1Gt0L79s4fQv55bCbpK7JbLkGXid5apjlzhkZ0 S5H3r6uOUrcMjtL+x1EfVZCUCAnezqyfQiAW0= Received: by 10.211.160.19 with SMTP id m19mr721679ebo.2.1254557618683; Sat, 03 Oct 2009 01:13:38 -0700 (PDT) Received: from marx.net.bit (nblzone-210-250.nblnetworks.fi [83.145.210.250]) by mx.google.com with ESMTPS id 28sm1453966eyg.20.2009.10.03.01.13.37 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sat, 03 Oct 2009 01:13:37 -0700 (PDT) Sender: a b Received: by marx.net.bit (Postfix, from userid 1000) id 02A791446F; Sat, 3 Oct 2009 11:13:35 +0300 (EEST) Date: Sat, 3 Oct 2009 11:13:35 +0300 From: Jukka Ruohonen To: freebsd-hackers@freebsd.org Message-ID: <20091003081335.GA19914@marx.net.bit> Mail-Followup-To: freebsd-hackers@freebsd.org References: <20091002201039.GA53034@flint.openpave.org> <4AC66E07.4030605@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4AC66E07.4030605@FreeBSD.org> User-Agent: Mutt/1.4.2.3i Subject: Re: Distributed SSH attack X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: jruohonen@iki.fi List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Oct 2009 08:37:51 -0000 On Fri, Oct 02, 2009 at 05:17:59PM -0400, Greg Larkin wrote: > You could set up DenyHosts and contribute to the pool of IPs that are > attempting SSH logins on the Net: > http://denyhosts.sourceforge.net/faq.html#4_0 While I am well aware that a lot of people use DenyHosts or some equivalent tool, I've always been somewhat skeptical about these tools. Few issues: 1. Firewalls should generally be as static as is possible. There is a reason why high securelevel prevents modifications to firewalls. 2. Generally you do not want some parser to modify your firewall rules. Parsing log entries created by remote unauthenticated users as root is never a good idea. 3. Doing (2) increases the attack surface. 4. There have been well-documented cases where (3) has opened opportunities for both remote and local DoS. Two cents, as they say, Jukka. From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 3 11:43:59 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78BE61065670 for ; Sat, 3 Oct 2009 11:43:59 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) by mx1.freebsd.org (Postfix) with ESMTP id 19DC38FC15 for ; Sat, 3 Oct 2009 11:43:58 +0000 (UTC) Received: from gidgate.gid.co.uk (80-46-130-69.static.dsl.as9105.com [80.46.130.69]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id n93BhvSV079245; Sat, 3 Oct 2009 12:43:58 +0100 (BST) (envelope-from rb@gid.co.uk) Received: from [194.32.164.28] ([194.32.164.6]) by gidgate.gid.co.uk (8.13.8/8.13.8) with ESMTP id n93Bhp4S010199; Sat, 3 Oct 2009 12:43:51 +0100 (BST) (envelope-from rb@gid.co.uk) Message-Id: From: Bob Bishop To: jruohonen@iki.fi In-Reply-To: <20091003081335.GA19914@marx.net.bit> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v936) Date: Sat, 3 Oct 2009 12:43:50 +0100 References: <20091002201039.GA53034@flint.openpave.org> <4AC66E07.4030605@FreeBSD.org> <20091003081335.GA19914@marx.net.bit> X-Mailer: Apple Mail (2.936) Cc: freebsd-hackers@freebsd.org Subject: Re: Distributed SSH attack X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Oct 2009 11:43:59 -0000 Hi, On 3 Oct 2009, at 09:13, Jukka Ruohonen wrote: > While I am well aware that a lot of people use DenyHosts or some > equivalent > tool, I've always been somewhat skeptical about these tools. Few > issues: > > 1. Firewalls should generally be as static as is possible. There is > a reason > why high securelevel prevents modifications to firewalls. > > 2. Generally you do not want some parser to modify your firewall > rules. > Parsing log entries created by remote unauthenticated users as > root is > never a good idea. > > 3. Doing (2) increases the attack surface. > > 4. There have been well-documented cases where (3) has opened > opportunities > for both remote and local DoS. > > Two cents, as they say, > > Jukka. Blackhole routes can be added as an alternative to tweaking firewall rules. The other objections (esp. 3) still apply of course, but these attacks are such a PITA (noise in the logs if nothing else) that one has to do something. -- Bob Bishop rb@gid.co.uk From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 3 10:03:30 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE15C106568D for ; Sat, 3 Oct 2009 10:03:30 +0000 (UTC) (envelope-from kraduk@googlemail.com) Received: from mail-fx0-f222.google.com (mail-fx0-f222.google.com [209.85.220.222]) by mx1.freebsd.org (Postfix) with ESMTP id 39C018FC0C for ; Sat, 3 Oct 2009 10:03:29 +0000 (UTC) Received: by fxm22 with SMTP id 22so1840766fxm.36 for ; Sat, 03 Oct 2009 03:03:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=832nfsIRUCDkB358iU+fR7ZoBjbH/WfLG2NhdmO8yZo=; b=kh/Pls8faa0B0A3GcKw+UcWA6oA9k2obqwhp3QaoZJ2+AB4RTj1jmZQAlE381Jbted Uj/k/MYrb/iFh0EiM/DJcj9KfHZ2Lw+RU+OR8pcskiZ/zS2aBi3R4Cqh0F7RjubxAscb I0vC/MNvbGSY4ISZKR4XmlSMtccLKK/IEijNM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=bckNswAhOiWX6UVhUBAjd4OA3jR6zdSIWArQRPytD+aEGOiQw6Imwfk/NBm0vVTUPY dVCldzRcaVnPbvLNmRS4dzSOSE1ffMGo8Uh9OHJJpcXcbaOsuy1y/Alj7swZSL0OTBSF EHKYbbNxLCAxYpEr7DMP+QKEbfllwkHTCX6Q8= MIME-Version: 1.0 Received: by 10.239.139.39 with SMTP id r39mr323744hbr.116.1254564209180; Sat, 03 Oct 2009 03:03:29 -0700 (PDT) In-Reply-To: <20091003081335.GA19914@marx.net.bit> References: <20091002201039.GA53034@flint.openpave.org> <4AC66E07.4030605@FreeBSD.org> <20091003081335.GA19914@marx.net.bit> Date: Sat, 3 Oct 2009 11:03:29 +0100 Message-ID: From: krad To: jruohonen@iki.fi, freebsd-hackers@freebsd.org X-Mailman-Approved-At: Sat, 03 Oct 2009 13:21:11 +0000 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Re: Distributed SSH attack X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Oct 2009 10:03:30 -0000 2009/10/3 Jukka Ruohonen > On Fri, Oct 02, 2009 at 05:17:59PM -0400, Greg Larkin wrote: > > You could set up DenyHosts and contribute to the pool of IPs that are > > attempting SSH logins on the Net: > > http://denyhosts.sourceforge.net/faq.html#4_0 > > While I am well aware that a lot of people use DenyHosts or some equivalent > tool, I've always been somewhat skeptical about these tools. Few issues: > > 1. Firewalls should generally be as static as is possible. There is a > reason > why high securelevel prevents modifications to firewalls. > > 2. Generally you do not want some parser to modify your firewall rules. > Parsing log entries created by remote unauthenticated users as root is > never a good idea. > > 3. Doing (2) increases the attack surface. > > 4. There have been well-documented cases where (3) has opened opportunities > for both remote and local DoS. > > Two cents, as they say, > > Jukka. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > simplest this to do is disable password auth, and use key based. From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 3 14:27:09 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF40310656AD for ; Sat, 3 Oct 2009 14:27:09 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id 223D28FC16 for ; Sat, 3 Oct 2009 14:27:08 +0000 (UTC) Received: from inchoate.gsoft.com.au (ppp121-45-212-218.lns11.adl2.internode.on.net [121.45.212.218]) (authenticated bits=0) by cain.gsoft.com.au (8.13.8/8.13.8) with ESMTP id n93ER58I040589 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sat, 3 Oct 2009 23:57:06 +0930 (CST) (envelope-from doconnor@gsoft.com.au) From: "Daniel O'Connor" To: freebsd-hackers@freebsd.org Date: Sat, 3 Oct 2009 23:56:54 +0930 User-Agent: KMail/1.9.10 References: <20091002201039.GA53034@flint.openpave.org> <20091003081335.GA19914@marx.net.bit> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart3670453.vm8lei8oIr"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200910032357.02207.doconnor@gsoft.com.au> X-Spam-Score: -2.212 () BAYES_00,RDNS_DYNAMIC X-Scanned-By: MIMEDefang 2.63 on 203.31.81.10 Cc: jruohonen@iki.fi, krad Subject: Re: Distributed SSH attack X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Oct 2009 14:27:09 -0000 --nextPart3670453.vm8lei8oIr Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Sat, 3 Oct 2009, krad wrote: > simplest this to do is disable password auth, and use key based. Your logs are still full of crap though. I find sshguard works well, and I am fairly sure you couldn't spoof a=20 valid TCP connection through pf sanitising so it would be difficult=20 (nigh-impossible?) for someone to cause you to block a legit IP. If you can, changing the port sshd runs on is by far the simplest work=20 around. Galling as it is to have to change stuff to work around=20 malicious assholes.. =2D-=20 Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C --nextPart3670453.vm8lei8oIr Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (FreeBSD) iD8DBQBKx1825ZPcIHs/zowRAhrZAKCH5XISuIqQoL//bD0vow5OPTfQWQCcDDlr FQaFma9NtcLmp4JqiTVhtaw= =oUaU -----END PGP SIGNATURE----- --nextPart3670453.vm8lei8oIr-- From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 3 20:45:37 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 47D6C106566B; Sat, 3 Oct 2009 20:45:37 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-fx0-f222.google.com (mail-fx0-f222.google.com [209.85.220.222]) by mx1.freebsd.org (Postfix) with ESMTP id 602A48FC16; Sat, 3 Oct 2009 20:45:36 +0000 (UTC) Received: by fxm22 with SMTP id 22so2071492fxm.36 for ; Sat, 03 Oct 2009 13:45:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=pZyvUZIwWegkrHiWS9DBRQ3UgxnMUAI8uiZkIZxVEiE=; b=F+dcdzJiMR5zoPjt7tHUNz3xiWcX+VCvBl5vqbrGHC+kxgetTJA4HDW7B/s0VEBgw1 IL40ViLwqkjU5pr/HXsnv5jOgxszqPFnFwZ3a0Ltf8tyK431weysUdZVH092RNGbPGpD NtWUAc1SHLtY04J8qYGBKURt/vvh1WIs3Tvxs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=o7W63YrAfX0FBoZqAxB/e5SdY5tg3A8BJ0WxZoBfp2fka4d3kO1Fuohz5stlNEzMYF MDQtiXR5wS0Ag/npvN+H2ImiHLcVilRtC4D8FdErhkavESZK03wynpJTmnXRwmoMXvqS rKXphtTVtnxUvMHTw/5IvPEe0fbmfe1ser2Mg= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.2.79 with SMTP id 15mr1414119fai.106.1254602735382; Sat, 03 Oct 2009 13:45:35 -0700 (PDT) In-Reply-To: <200909291731.32394.jhb@freebsd.org> References: <20090924224935.GW473@gandalf.sssup.it> <200909291953.36373.max@love2party.net> <3bbf2fe10909291342o4d32e381ge23e446582bb2d18@mail.gmail.com> <200909291731.32394.jhb@freebsd.org> Date: Sat, 3 Oct 2009 22:45:35 +0200 X-Google-Sender-Auth: 857e18463470939a Message-ID: <3bbf2fe10910031345n5ea54f40i1ca4dd6a1d42ff13@mail.gmail.com> From: Attilio Rao To: John Baldwin Content-Type: text/plain; charset=UTF-8 Cc: Max Laier , Fabio Checconi , freebsd-hackers@freebsd.org Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Oct 2009 20:45:37 -0000 2009/9/29 John Baldwin : > On Tuesday 29 September 2009 4:42:13 pm Attilio Rao wrote: >> 2009/9/29 Max Laier : >> > On Tuesday 29 September 2009 17:39:37 Attilio Rao wrote: >> >> 2009/9/25 Fabio Checconi : >> >> > Hi all, >> >> > looking at sys/sx.h I have some troubles understanding this comment: >> >> > >> >> > * A note about memory barriers. Exclusive locks need to use the same >> >> > * memory barriers as mutexes: _acq when acquiring an exclusive lock >> >> > * and _rel when releasing an exclusive lock. On the other side, >> >> > * shared lock needs to use an _acq barrier when acquiring the lock >> >> > * but, since they don't update any locked data, no memory barrier is >> >> > * needed when releasing a shared lock. >> >> > >> >> > In particular, I'm not understanding what prevents the following sequence >> >> > from happening: >> >> > >> >> > CPU A CPU B >> >> > >> >> > sx_slock(&data->lock); >> >> > >> >> > sx_sunlock(&data->lock); >> >> > >> >> > /* reordered after the unlock >> >> > by the cpu */ >> >> > if (data->buffer) >> >> > sx_xlock(&data->lock); >> >> > free(data->buffer); >> >> > data->buffer = NULL; >> >> > sx_xunlock(&data->lock); >> >> > >> >> > a = *data->buffer; >> >> > >> >> > IOW, even if readers do not modify the data protected by the lock, >> >> > without a release barrier a memory access may leak past the unlock (as >> >> > the cpu won't notice any dependency between the unlock and the fetch, >> >> > feeling free to reorder them), thus potentially racing with an exclusive >> >> > writer accessing the data. >> >> > >> >> > On architectures where atomic ops serialize memory accesses this would >> >> > never happen, otherwise the sequence above seems possible; am I missing >> >> > something? >> >> >> >> I think your concerns are right, possibly we need this patch: >> >> http://www.freebsd.org/~attilio/sxrw_unlockb.diff >> >> >> >> However speaking with John we agreed possibly there is a more serious >> >> breakage. Possibly, memory barriers would also require to ensure the >> >> compiler to not reorder the operation, while right now, in FreeBSD, they >> >> just take care of the reordering from the architecture perspective. >> >> The only way I'm aware of GCC offers that is to clobber memory. >> >> I will provide a patch that address this soon, hoping that GCC will be >> >> smart enough to not overhead too much the memory clobbering but just >> >> try to understand what's our purpose and servers it (I will try to >> >> compare code generated before and after the patch at least for tier-1 >> >> architectures). >> > >> > Does GCC really reorder accesses to volatile objects? The C Standard seems to >> > object: >> > >> > 5.1.2.3 - 2 >> > Accessing a volatile object, modifying an object, modifying a file, or calling >> > a function that does any of those operations are all side effects,11) which >> > are changes in the state of the execution environment. Evaluation of an >> > expression may produce side effects. At certain specified points in the >> > execution sequence called sequence points, all side effects of previous >> > evaluations shall be complete and no side effects of subsequent evaluations >> > shall have taken place. (A summary of the sequence points is given in annex >> > C.) >> >> Very interesting. >> I was thinking about the other operating systems which basically do >> 'memory clobbering' for ensuring a compiler barrier, but actually they >> often forsee such a barrier without the conjuction of a memory >> operand. >> >> I think I will need to speak a bit with a GCC engineer in order to see >> what do they implement in regard of volatile operands. > > GCC can be quite aggressive with reordering even in the face of volatile. I > was recently doing a hack to export some data from the kernel to userland > that used a spin loop to grab a snapshot of the contents of a structure > similar to the method used in the kernel with the timehands structures. It > used a volatile structure exposed from the kernel that looked something > like: > > struct foo { > volatile int gen; > /* other stuff */ > }; > > volatile struct foo *p; > > do { > x = p->gen; > /* read other stuff */ > y = p->gen; > } while (x != y && x != 0); > > GCC moved the 'y = ' up into the middle of the '/* read other stuff */'. > I eventually had to add explicit "memory" clobbers to force GCC to not > move the reads of 'gen' around but do them "around" all the other > operations, so that the working code is: > > do { > x = p->gen; > asm volatile("" ::: "memory"); > /* read other stuff */ > asm volatile("" ::: "memory"); > y = p->gen; > } while (x != y && x != 0); The situation was not so desperate as I first thought. Actually, only ia32 and amd64 seems affected by the missing of memory clobbering because it is already done for all the other platform when using all the memory barriers. On ia32 and amd64 too, the impact is more limited than what I expected. atomic_cmpset_* already clobbers the memory and only atomic_store_rel_* is not adeguately protected among the atomics used in our locking primitives, thus I would really expect a limited performance penalty if any. What was not really protected where the functions defined through ATOMIC_ASM() and that was the larger part to fix. I spoke briefly about the compiler support with Christian Lattner (@Apple, LLVM leader) and he mentioned he was aware of the aggressive behaviour of GCC pointing me in the direction that what the C Standard really mandates is that read/write operations with non-volatile operands can be reordered across a volatile operand but that the read/write of volatile operands are strong ordered in regard of eachother. This however means that we have to fix the 'memory clobber' for GCC-simil compilers and also offer a support for the other that let them specify a memory barrier. I then wrote this patch: http://www.freebsd.org/~attilio/atomic.h.diff That should address all the concern raised and also forsee the possibility for other compiler to specify memory barriers semantics differently from normal atomic. rdivacky@ kindly already tested the patch on LLVM/CLANG reporting no problems. I still didn't compare pickly the produced binaries, but I see a little increase in the binary sizes probabilly caming from the .text growing. Attilio -- Peace can only be achieved by understanding - A. Einstein