From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 19 21:38:53 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 772FE47E for ; Sat, 19 Oct 2013 21:38:53 +0000 (UTC) (envelope-from electreg@list.ru) Received: from fallback3.mail.ru (fallback3.mail.ru [94.100.176.58]) by mx1.freebsd.org (Postfix) with ESMTP id EBEAF2277 for ; Sat, 19 Oct 2013 21:38:52 +0000 (UTC) Received: from f133.i.mail.ru (f133.i.mail.ru [94.100.178.193]) by fallback3.mail.ru (mPOP.Fallback_MX) with ESMTP id 0A4C9FBAE463 for ; Sun, 20 Oct 2013 01:38:42 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=list.ru; s=mail; h=References:In-Reply-To:Content-Type:Message-ID:Reply-To:Date:Mime-Version:Subject:Cc:To:From; bh=RRcKYPc+3C7e0s2ZAgjIp+tKj2wOL/XMZ0Kzel+j3G0=; b=XjvUy9vHF6xyw4yx0H8oF3XkteT/WiIvVAJ75rRom/jVmD3IayvUSKfbjQMxNTkAssTQuNCCLBPOlkL12cV+vYe2bso2OWvV5TjSxear4VOm6V1N1dxf7n3OGBqzqR2tduqQpBiP3iNklkRc+wXJosR2NaGKBhNp2Mkiz2Og6aQ=; Received: from mail by f133.i.mail.ru with local (envelope-from ) id 1VXeED-0003jz-Ce; Sun, 20 Oct 2013 01:38:33 +0400 Received: from [94.51.58.93] by e.mail.ru with HTTP; Sun, 20 Oct 2013 01:38:33 +0400 From: =?UTF-8?B?QWxleGV5IEVnb3Jvdg==?= To: =?UTF-8?B?RGFycmVuIFBpbGdyaW0=?= Subject: =?UTF-8?B?UmU6IGRldGVybWluZSBkcml2ZSdzIFNBUyBwb3J0?= Mime-Version: 1.0 X-Mailer: Mail.Ru Mailer 1.0 X-Originating-IP: [94.51.58.93] Date: Sun, 20 Oct 2013 01:38:33 +0400 X-Priority: 3 (Normal) Message-ID: <1382218713.206890674@f133.i.mail.ru> X-Mras: Ok X-Spam: undefined In-Reply-To: <5262E2DD.6030703@bluerosetech.com> References: <1382192148.119437035@f257.i.mail.ru> <5262E2DD.6030703@bluerosetech.com> X-Mailman-Approved-At: Sun, 20 Oct 2013 05:14:55 +0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: =?UTF-8?B?QWxleGV5IEVnb3Jvdg==?= List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Oct 2013 21:38:53 -0000 ID4gcGVyaGFwcyB5b3UgY2FuIHRyYWNlIHRoZSBjYWJsZXMgYW5kIGdldCBhIFBDQiBkaWFncmFt PwpJIG5lZWQgdG8gZG8gdGhpcyAib24gdGhlIGZseSIgdmlhIHNvbWUgc2hlbGwgc2NyaXB0IHRv IGJlIGFibGUgdG8gdXNlIGl0IGluIGRldmQgcnVsZXMuCk9uIExpbnV4IHdlIHVzZSAvc3lzLyog ZW50cmllcyB0byBkbyB0aGlzIC0gdW5mb3J0dW5hdGVseSBpdCdzIHdvcmtpbmcgb24gTGludXgg b25seS4KCg== From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 20 06:28:47 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 882BDD4 for ; Sun, 20 Oct 2013 06:28:47 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-pa0-x22c.google.com (mail-pa0-x22c.google.com [IPv6:2607:f8b0:400e:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 61AA02838 for ; Sun, 20 Oct 2013 06:28:47 +0000 (UTC) Received: by mail-pa0-f44.google.com with SMTP id fb1so3830881pad.31 for ; Sat, 19 Oct 2013 23:28:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=vBWwW41X4OYQu85WRv94cH30Z3Vhe5MkYRmn/Ogutzg=; b=tFp/BwW6i22r6riniWEUrlIIRxw9F9+Oznu0ZOkm3Dg+Y4M88z++RzUaHxHm1ZWKVM eiHTMEUcXHmIUY6LN1QT84AKMmVnra0UWiDKJaoLf6LE6stnbt3+yHVuVNpmtSm/HBD4 6zhHCq4aOr7BY1iSzL25zH+QSBbUCb0OIXVER1nookeZGvzt138Iaouhd9rcwFMhWmdT Q4JaFH10Mt4Uvc9RLtfpxE3dClFlSmGrytnv96fuVdBfWxSiCYLFuot2uTVvfS2Q+CjF 3Y/kdPeoP+La+1unJXZTTIkmoGJUDhVOy0RaVpP1R6+TbRIKAVyD3xvnLZEhEh8LrJW/ ZJLw== MIME-Version: 1.0 X-Received: by 10.66.26.236 with SMTP id o12mr255519pag.136.1382250526307; Sat, 19 Oct 2013 23:28:46 -0700 (PDT) Received: by 10.70.92.79 with HTTP; Sat, 19 Oct 2013 23:28:46 -0700 (PDT) In-Reply-To: <1382218713.206890674@f133.i.mail.ru> References: <1382192148.119437035@f257.i.mail.ru> <5262E2DD.6030703@bluerosetech.com> <1382218713.206890674@f133.i.mail.ru> Date: Sun, 20 Oct 2013 01:28:46 -0500 Message-ID: Subject: Re: determine drive's SAS port From: Adam Vande More To: Alexey Egorov Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Hackers , Darren Pilgrim X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Oct 2013 06:28:47 -0000 On Sat, Oct 19, 2013 at 4:38 PM, Alexey Egorov wrote: > > perhaps you can trace the cables and get a PCB diagram? > I need to do this "on the fly" via some shell script to be able to use it > in devd rules. > On Linux we use /sys/* entries to do this man linsysfs -- however you shouldn't need to do this in the first place. -- Adam From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 20 16:09:47 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3EE787BB for ; Sun, 20 Oct 2013 16:09:47 +0000 (UTC) (envelope-from feld@FreeBSD.org) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 13A35208E for ; Sun, 20 Oct 2013 16:09:46 +0000 (UTC) Received: from compute4.internal (compute4.nyi.mail.srv.osa [10.202.2.44]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 2A45721773 for ; Sun, 20 Oct 2013 12:09:46 -0400 (EDT) Received: from web3 ([10.202.2.213]) by compute4.internal (MEProxy); Sun, 20 Oct 2013 12:09:46 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:mime-version :content-transfer-encoding:content-type:in-reply-to:references :subject:date; s=smtpout; bh=D2QC3sMkf4s/zV9lyfoaDICfqS8=; b=o52 xTJuIIoikcNabOr4crBWeKrd7IhPlnjLXwfcflZgrmO0PlvvrgpZihLj0eDrDKup ODITFvg/MHXcAxIPiO3wy2Yj9p6rgIvX8nXsBb7JyRCYOGFupYknzfs5Wv5GBpW9 gkPYHjE2dzW0p/z/a8ahbgTsnxxI+WSEjwMXfQ+Q= Received: by web3.nyi.mail.srv.osa (Postfix, from userid 99) id 09EC2101B77; Sun, 20 Oct 2013 12:09:46 -0400 (EDT) Message-Id: <1382285385.3666.36239857.6EFC4FB5@webmail.messagingengine.com> X-Sasl-Enc: hnpW2yDsyTzbl0OjDPhIv4zYK7CLmeV8RY6Ts6UxSme9 1382285385 From: Mark Felder To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-443dd2cc In-Reply-To: <1382192148.119437035@f257.i.mail.ru> References: <1382192148.119437035@f257.i.mail.ru> Subject: Re: determine drive's SAS port Date: Sun, 20 Oct 2013 11:09:45 -0500 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Oct 2013 16:09:47 -0000 On Sat, Oct 19, 2013, at 9:15, Alexey Egorov wrote: > Hello all, > > I have a server with LSI HBA card, and when I remove drive I can see > following messages in log: > Perhaps use GPT label to indicate what its physical location is? From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 20 17:01:47 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 95764505 for ; Sun, 20 Oct 2013 17:01:47 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5984822EA for ; Sun, 20 Oct 2013 17:01:47 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1VXvy3-0005Mw-Ez; Sun, 20 Oct 2013 16:35:03 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r9KGYxj1033049; Sun, 20 Oct 2013 10:34:59 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1+LZRVO8lMpeTBtaigiQHAA Subject: Re: determine drive's SAS port From: Ian Lepore To: Alexey Egorov In-Reply-To: <1382192148.119437035@f257.i.mail.ru> References: <1382192148.119437035@f257.i.mail.ru> Content-Type: text/plain; charset="UTF-8" Date: Sun, 20 Oct 2013 10:34:59 -0600 Message-ID: <1382286899.92499.110.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by damnhippie.dyndns.org id r9KGYxj1033049 Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Oct 2013 17:01:47 -0000 On Sat, 2013-10-19 at 18:15 +0400, Alexey Egorov wrote: > Hello all, >=20 > I have a server with LSI HBA card, and when I remove drive I can see fo= llowing messages in log: >=20 > (da0:mps0:0:5:0): lost device - 0 outstanding, 2 refs > (da0:mps0:0:5:0): removing device entry >=20 > Is there a way to determine physical port (number "5" in "(da0:mps0:0:5= :0)=EF=BB=BF=EF=BB=BF") when drive is inserted? (I need this to be able t= o create device symlinks based on physical port). >=20 > Thanks in advance.=20 I don't have hardware like that to play with, but when I plug in an eSata drive I get devd events like this: root@dpnand:/root # cat /var/run/devd.pipe=20 !system=3DDEVFS subsystem=3DCDEV type=3DCREATE cdev=3Dpass2 !system=3DDEVFS subsystem=3DCDEV type=3DCREATE cdev=3Dada0 !system=3DDEVFS subsystem=3DCDEV type=3DCREATE cdev=3Dad0 !system=3DDEVFS subsystem=3DCDEV type=3DCREATE cdev=3Dada0p11 !system=3DDEVFS subsystem=3DCDEV type=3DCREATE cdev=3Dad0p11 !system=3DDEVFS subsystem=3DCDEV type=3DCREATE cdev=3Ddiskid/DISK-10MS109= LT74Z !system=3DDEVFS subsystem=3DCDEV type=3DCREATE cdev=3Dufsid/51fabc51ea1a9= 23b !system=3DDEVFS subsystem=3DCDEV type=3DCREATE cdev=3Dgptid/c057d696-fae3= -11e2-b79c-5404a6f2f88a !system=3DDEVFS subsystem=3DCDEV type=3DCREATE cdev=3Ddiskid/DISK-10MS109= LT74Zp11 ^C The pass2 dev appears in a camcontrol devlist, like this: root@dpnand:/root # camcontrol devlist at scbus0 target 0 lun 0 (pass2,ada0) at scbus2 target 0 lun 0 (pass0,da0) at scbus2 target 0 lun 1 (pass1,da1) Would the camcontrol bus/target/lun output give you the number you're looking for? It's a pity there isn't a devd event with more info in it (similar to what you get when usb devices come and go), but perhaps with some scripting you can make the connection between events and devices. -- Ian From owner-freebsd-hackers@FreeBSD.ORG Sun Oct 20 21:57:03 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id BE29C20C; Sun, 20 Oct 2013 21:57:03 +0000 (UTC) (envelope-from pali.gabor@gmail.com) Received: from mail-oa0-x236.google.com (mail-oa0-x236.google.com [IPv6:2607:f8b0:4003:c02::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 69F8422F9; Sun, 20 Oct 2013 21:57:03 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id o20so1805435oag.27 for ; Sun, 20 Oct 2013 14:57:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=6aozCwXJc4sQECUSnECYx/1MpKb0zcuWXw/wrgbmylI=; b=Qe4kqDJsvB3nyIwLTgAX6GRmqHoJ/FUilAAC0yWC0eECNNUMpuSKAJgQfFi3r0YE1d GS9y+fQayhW2F1f9a7K3558TgJ/ngkpYQR9n6e60TJeXd6Lfj3+XDyWJU7v2bojYxsHO /TXl92a9PGyNQvg+Od0LeuA6nZwt5V4WIlzFunyIaDtXHsYnVL17EYi7IU4ftpZmND0/ QMqmmIAbHu/Hs0eCiRPptAqRp0yFbWJ7uouKXgBSsORFxYxnTCtcT+Nekt8lk2ptc5zL ptGHc5H887pI97+rrsjKKPBrIXKqgLmnEaKs8zaEBjdyHmGapQo+YiQ81QCgSgDeHO39 yPIQ== MIME-Version: 1.0 X-Received: by 10.60.43.131 with SMTP id w3mr4504439oel.10.1382306222583; Sun, 20 Oct 2013 14:57:02 -0700 (PDT) Sender: pali.gabor@gmail.com Received: by 10.182.22.44 with HTTP; Sun, 20 Oct 2013 14:57:02 -0700 (PDT) Date: Sun, 20 Oct 2013 23:57:02 +0200 X-Google-Sender-Auth: THSdfTj42tCW4vbgQoNrD9x1iQw Message-ID: Subject: FreeBSD Quarterly Status Report, July-September, 2013 From: Gabor Pali To: hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: stable@freebsd.org, current@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Oct 2013 21:57:03 -0000 FreeBSD Quarterly Status Report, July-September 2013 Introduction This report covers FreeBSD-related projects between July and September 2013. This is the third of four reports planned for 2013. We have had another very active three months in the FreeBSD world, including two Developer Summits (BSDCam and EuroBSDcon) that will be covered in separate status reports. FreeBSD continues to push hard on security, with improvements to both the performance and reliability of the random number generation, and more compartmentalisation in programs in the base system. For developers, there is work on a new modern debugger. There is also a significant amount of of modernization in the support for Objective-C and Ada via ports, making FreeBSD a first-rate platform for developing in either language, in addition to the existing C++11 and C11 support already present in the base system. Server users will be pleased to see improvements in the iSCSI stack and scalability allowing over a million I/O operations per second on commodify hardware, while desktop users will see improvements in X support for new GPUs and for possible X replacements. Thanks to all the reporters for the excellent work! This report contains 30 entries and we hope you enjoy reading it. The deadline for submissions covering between October and December, 2013 is January 14th, 2014. __________________________________________________________________ FreeBSD Team Reports * FreeBSD Core Team * FreeBSD Port Management Team * FreeBSD Postmaster Team * FreeBSD Release Engineering Team Projects * Static Code Analysis Kernel * AES-NI Improvements for GELI * Atomic "close-on-exec" * Continuation of the Newcons Project * GEOM Direct Dispatch and Fine-Grained CAM Locking * Native iSCSI Stack * Reworking random(4) * SDIO Driver * VirtIO Network Multiqueue * VMware VMXNET3 Driver Architectures * FreeBSD on Cubieboard2 * FreeBSD/EC2 * FreeBSD/pseries * FreeBSD/sparc64 * Superpages for ARMv7 Userland Programs * Capsicum * LLDB Debugger Port Ports * FreeBSD Ada Ports * FreeBSD Python Ports * GNOME/FreeBSD * GNUstep on FreeBSD * X.Org on FreeBSD Documentation * FreeBSD Documentation Project Primer Edit * The entities Documentation Branch Google Summer of Code * Download Manager Service for the Ports Collection Miscellaneous * The FreeBSD Foundation __________________________________________________________________ AES-NI Improvements for GELI URL: http://svnweb.freebsd.org/changeset/base/255187 Contact: John-Mark Gurney An enhancement to the AES-NI implementation for OpenCrypto, the kernel's cryptography framework, has been committed that significantly improves AES-XTS and AES-CBC decryption performance. This gives geli(8) around a three times performance boost on gnop(8) using AES-XTS compared to the old code. These improvements are available to users of the OpenCrypto framework and crypto(4). __________________________________________________________________ Atomic "close-on-exec" URL: https://wiki.freebsd.org/AtomicCloseOnExec Contact: Jilles Tjoelker If threads or signal handlers call fork() and exec(), file descriptors may be passed undesirably to child processes, which may lead to hangs (if a pipe is not closed), exceeding the file descriptor limit, and security problems (if the child process has lower privilege). One solution is various new APIs that set the "close-on-exec" flag atomically with allocating a file descriptor. Some existing software will use the new features if present or will even refuse to compile without them. With mkostemp(), dup3(), and a change to modes of fopen() and freopen(), everything proposed in Austin Group issue #411 has now been implemented. For all POSIX-specified functions that allocate file descriptors, it is possible to request that the new descriptor be set close-on-exec atomically. Additionally, many file descriptors used internally by libc and libutil now have the close-on-exec bit set. __________________________________________________________________ Capsicum Contact: Pawel Jakub Dawidek Capsicum is the FreeBSD sandboxing subsystem, which presents programmers with a capability module allowing fine-grained delegation of rights to less-privileged processes. Casper is a friendly daemon that provides services to sandboxed processes, allowing policy-based access to privileged services such as DNS resolution. The work on Capsicum and related projects (such as Casper, libnv, etc.) is progressing nicely. An overhaul of the cap_rights_t was committed to FreeBSD head and will be included in 10.0. This allows us to have more capability rights on file descriptors than the previous limit of 64 rights, which was almost reached. This change is not backward compatible, so it was very important to get it into 10.0. libnv, used for communication between Casper services and consumers, but which will hopefully be used more widely, is finalized and comes with a nice set of regression tests. The number of applications sandboxed using the Capsicum framework is increasing. We have around 10 of them already in base and more that are not yet committed. This project is being sponsored by the FreeBSD Foundation. Open tasks: 1. Finish documentation of Casper and its services. 2. Implement regression tests for Casper services. 3. Finish documentation for libnv. 4. Start making libc more sandbox-friendly, that is, modifying functions such as strerror(3), strsignal(3), localtime(3), login_get*(), getservent(3), getprotent(3), and getrpcent(3) which currently open files on first use, which might be too late if we are already in a capability-mode sandbox. 5. Rethink the system.filesystem Casper service to allow for easy compartmentalization of various command-line tools that operate on multiple files. __________________________________________________________________ Continuation of the Newcons Project URL: http://svn.freebsd.org/base/user/ed/newcons/ Contact: Aleksandr Rybalko The Newcons project is aimed to replace the old syscons(4)-based virtual terminals. The main objectives are: support Unicode characters, and move away from the dependency on fixed VGA and VESA graphics modes and built-in BIOS services. This project was originally started by Ed Schouten, and it already featured the following features (among many others) in 2013: * Unicode fonts with Latin, Cyrillic and some more simple character sets. * Unicode output support. * Graphics mode support. * Text mode support. * sysmouse(4) support, without copy/paste. And these have been extended by the following items recently: * History, that is, the ability to scroll through the terminal history. The old, separate history buffer has been removed. * The history is implemented by a circular buffer which has no risk of overflow, and scrolling appears "unlimited". * VT_PROCESS mode, a way to hold the terminal and prevent terminal switching. For example, X.Org uses this feature to prevent the user from switching to a non-X terminal. * drm2/fb_helper, the KMS driver. This binds Newcons to framebuffers created the DRM-enabled video drivers in the kernel (such as i915kms and radeonkms). * Dynamic attachment of VT drivers, vt_allocate() to allow attaching console video drivers at a later point where framebuffer owner can manage the initialization. This is for KMS and devices without early graphics support. Supported startup modes for KMS: * Start without VT graphics drivers, then load KMS. * Start with VGA, then load KMS. * Preload KMS, then the KMS driver will be attached to the output. * Preload KMS, start with VGA, then KMS driver will replace the VGA output. This project is being sponsored by The FreeBSD Foundation. Many thanks to Ed Schouten, who started the Newcons project and did most of the work. Open tasks: 1. Implement a Generic Framebuffer interface, a simple interface to offer direct access to the framebuffer from the userland (via /dev/fb*) and automatic management of virtual terminals by Newcons. 2. Mouse support, copy/paste using sysmouse(4). 3. Improve locking. 4. Bug fixes. 5. Integrate into FreeBSD head. 6. Integrate into FreeBSD 10.0. 7. Implement mapping non-ASCII characters to Unicode on keyboard input. 8. Adapt existing screen savers. 9. Last but not least, testing is welcome! __________________________________________________________________ Download Manager Service for the Ports Collection URL: https://wiki.freebsd.org/SummerOfCode2013/IntellegentDownloadManage= r URL: https://wiki.freebsd.org/IdeasPage/IDMS Contact: Ambarisha Bhatlapenumarthi Contact: Xin Li This is a Google Summer of Code 2013 project that aims to replace the fetch(1)-based method for getting distribution files, such as source tarballs, for the third-party applications (ports) with an intelligent Download Manager Service (see links for more information). All the modules highlighted in the project wiki have been completed (see links). Specifically: * A service that receives and serves download requests. It samples download speeds from different mirrors and uses this information to pick the best mirror on the next request. It can migrate jobs between mirrors if it realizes that a complete download from a different mirror would be faster than proceeding with the mirror it is currently using. * A status dump feature has also been added to the client (dmget) which dumps the information about active downloads, speeds from mirrors, etc. Open tasks: 1. The implementation (especially job migration and dumping status) has not been tested thoroughly. Test the code, write more unit and regression tests. __________________________________________________________________ FreeBSD Ada Ports URL: http://www.dragonlace.net Contact: John Marino A few years ago, Ada-based ports almost completely disappeared from the Ports Collection. This was not surprising, as FSF GNAT, the only open-source Ada compiler, ceased to build correctly on any BSD flavor. Previously-built bootstrap compilers would not run on modern FreeBSD, and certainly not on amd64. The first step, see the link for details, was to patch GCC in order to fix GNAT not only on FreeBSD, but DragonFly, NetBSD, and OpenBSD as well. New bootstraps for both i386 and amd64 platforms were produced during this effort. Ada compilers on FreeBSD now pass 100% of the ACATS and GCC testsuites. With the introduction of the first new Ada compiler port, the GCC 4.6-based lang/gnat-aux, the GNAT Programming Studio (a multilanguage integrated development environment), XML/Ada, and GtkAda were among the first Ada ports resurrected. With the latest compiler, lang/gcc-aux based on GCC 4.7, a cohesive Ada framework was created with the new USES=3D framework. Currently around 2= 0 ports are part of this framework including Florist, ASIS, GPRbuild, QtAda, AdaControl, AdaBrowse, PolyOrb, and AWS (Ada Web Server). The GNAT AUX compiler is also still in use to serve as a basis for the GNATDroid ports which are FreeBSD-to-Android Ada+C cross-compilers. However, these will soon be integrated into the Ada Framework. At this point, it looks like FreeBSD (shared with DragonFly via DPorts) has taken the crown from Debian as the recognized best Ada development platform. The FreeBSD versions of the software are more recent and the Ports Collection has ports not available on Debian, such as LibSparkCrypto, the Matreshka library, and the Ahven unit tester. Future work potentially includes converting GCC AUX to GCC 4.8 to acquire better Ada 2012 support, importing Spark 2014 into ports when it arrives and to continue to add new Ada ports to the framework. __________________________________________________________________ FreeBSD Core Team Contact: FreeBSD Core Team In the third quarter of 2013, the Core Team focused on officially launching pkg.freebsd.org, the Project's official pkg(8) repository, in cooperation with the Port Management Team, the Security Team, and the Cluster Administration Team. At the same time, there are plans to gradually deprecate the use of the old pkg_add(1), allowing pkg(8) to be the default binary package management solution for FreeBSD, arriving with 10.0-RELEASE. Thomas Abthorpe has been appointed to the role of liaison between the Core Team and the Ports Management Team, in order to make the collaboration more effective. David Chisnall has joined the group that publishes the Quarterly Status reports and compiled a special status report on the results of the BSDCan 2013 Developer Summit. David also took the lead role on the organization of an off-season developer summit in Cambridge, UK, which was finally held at the end of August. For the items discussed in Cambridge, preparation of a detailed report is still in progress. There were src commit bits issued for 5 new developers and most of the src commits being idle more than 12 months have been taken into safekeeping as result of a major cleanup to the repository access file in July, performed by Gavin Atkinson. __________________________________________________________________ FreeBSD Documentation Project Primer Edit URL: http://www.freebsd.org/doc/en/books/fdp-primer/book.html Contact: Warren Block The FreeBSD Documentation Project Primer had not changed at the same rate as the documents themselves. Some sections were outdated and others were verbose and confusing, while information on new changes to the documentation were not described at all. In July, Warren gave the entire FDP Primer a fairly intense edit for simplicity and clarity. Chapters and sections were moved into a more logical order, and information was updated to be a better guide to the current state. Markup examples were added and revised. Style guidelines were also extended and updated. The Primer is now far more consistent and usable. As always, there is still room for improvement, and additions or corrections are encouraged. Open tasks: 1. An introductory chapter on writing manual pages with mdoc(7) would be an excellent addition. __________________________________________________________________ FreeBSD on Cubieboard2 URL: http://svnweb.freebsd.org/changeset/base/254056 Contact: Ganbold Tsagaankhuu Initial support of Allwinner A20 SoC is committed to head. The A20 SoC on Cubieboard2 is pin-to-pin compatible with the A10 in Cubieboard1 and FreeBSD supports the following peripherals: * USB EHCI * GPIO Open tasks: 1. Get the EMAC Ethernet driver working. Need more help from network driver experts. 2. Add more drivers. __________________________________________________________________ FreeBSD Port Management Team URL: http://www.FreeBSD.org/ports/ URL: http://www.freebsd.org/doc/en/articles/contributing-ports/ URL: http://portsmon.freebsd.org/index.html URL: http://www.freebsd.org/portmgr/index.html URL: http://blogs.freebsdish.org/portmgr/ URL: http://www.twitter.com/freebsd_portmgr/ URL: http://www.facebook.com/portmgr URL: http://lists.freebsd.org/mailman/listinfo/freebsd-pkg-fallout Contact: FreeBSD Port Management Team The ports tree contains approximately 24,400 ports, while the PR count exceeds 1,900. In the third quarter, we added four new committers and took in six commit bits for safekeeping. A significant amount of effort has gone into tweaking and manipulating the infrastructure to modernize and update it, in preperation for pkg(8) replacing the old pkg_add(1) infrastructure, as well as preparing for FreeBSD 10.0 with Clang as default compiler, libc++ as the default C++ standard library, and iconv(1) integrated into libc. Automated procedures for quality assurance have been implemented, notably pkg-fallout. All porters are encouraged to subscribe to the associated mailing list (see links), and do their part to fix ports for pkg(8) and Clang readiness. Many iterations of tests were run to ensure that as many packages as possible would be available for the 9.2 release. Open tasks: 1. Most ports PRs are assigned, we now need to focus on testing, committing, and closing. __________________________________________________________________ FreeBSD Postmaster Team URL: http://lists.freebsd.org/mailman/listinfo/freebsd-fortran URL: http://lists.freebsd.org/mailman/listinfo/freebsd-pkg-fallout URL: http://lists.freebsd.org/mailman/listinfo/freebsd-users-jp Contact: FreeBSD Postmaster Team In the third quarter of 2013, the FreeBSD Postmaster Team has implemented the following items that may be interest of the general public: * Created the freebsd-fortran list, requested by Anton Shterenlikht. * Created the freebsd-pkg-fallout list, requested by Baptiste Daroussin. * Created the freebsd-users-jp list, requested by Hiroki Sato * Retired the freebsd-mozilla list, requested by Florian Smeets. * Worked with the FreeBSD Cluster Administrators to enable TLS support on incoming and outgoing mail servers. * Started discussions and exploration of current and possible future mail and spam filtering. * Started the process for retiring the aic7xxx mailing list. Completion of this is scheduled for 12 October 2013. __________________________________________________________________ FreeBSD Python Ports URL: https://wiki.FreeBSD.org/Python URL: irc://freebsd-python@irc.freenode.net Contact: FreeBSD Python Team We are currently working on cleaning up the lang/python* ports to improve their compatibility with the original upstream build behaviour and to reduce the need for FreeBSD-specific build patches. A first step was made in September by reducing the flags injected into the different Python interpreter versions. The first tasks have been completed to support the installation of packages for different Python ports. A new metaport structure has replaced the original Python port behaviour, and will be enhanced over the next months to enable improved installation support of packages for different Python versions at the same time. The Python ports framework was enhanced with automated packaging list creation and replacement macros, which improve the compatibility with multiple Python versions and reduce the packaging list sizes. PyPy was heavily enhanced over the last couple of months. Major updates to the port solved integration issues and a new pypy-devel port for snapshots and previews was added. Since the PyPy 3 release, there is a new pypy3-devel port available to provide not only compatibility for Python 2.x specific scripts, but also for those using the 3.x language specification. IronPython found its way into the FreeBSD ports tree, providing an implementation of the Python language based on .NET and Mono. Open tasks: 1. Develop a high-level and lightweight Python Ports Policy. 2. Chase the unification of Distribute (devel/py-distribute) and Setuptools (devel/py-setuptools*). 3. Add support for granular dependencies (for example >=3D1.0 or < 2.0)= . 4. Look at what adding pip (Python Package Index) support looks like. 5. More tasks can be found on the Team's wiki page (see links). __________________________________________________________________ FreeBSD Release Engineering Team URL: http://www.FreeBSD.org/releases/9.2R/schedule.html URL: http://www.FreeBSD.org/releases/10.0R/schedule.html URL: http://ftp.FreeBSD.org/pub/FreeBSD/snapshots/VM-IMAGES/ URL: http://ftp.FreeBSD.org/pub/FreeBSD/snapshots/ISO-IMAGES/ Contact: FreeBSD Release Engineering Team The FreeBSD Release Engineering Team has completed the 9.2-RELEASE process. The release cycle changed with a last-minute addition of 9.2-RC4. The 9.2-RELEASE was announced September 30, four weeks behind the original schedule. The FreeBSD 10.0-RELEASE cycle has started, and testing is strongly encouraged. For testing purposes, both installation images and virtual machine images exist on the FreeBSD Project FTP servers. Open tasks: 1. Test 10.0-CURRENT and report problems. __________________________________________________________________ FreeBSD/EC2 URL: http://www.daemonology.net/freebsd-on-ec2/ URL: https://aws.amazon.com/marketplace/pp/B00AA25MLK/ Contact: Colin Percival FreeBSD images are available for use in EC2 for 8.3-RELEASE, 8.4-RELEASE, 9.0-RELEASE, 9.1-RELEASE, and 9.2-RELEASE. In 9.2-RELEASE, FreeBSD runs in EC2 using an unpatched source tree, but it needs the XENHVM kernel configuration. Starting from FreeBSD 10.0-ALPHA3, the GENERIC kernel configuration now contains all the XENHVM bits needed to allow FreeBSD to run in EC2 natively. Consequently, FreeBSD 10.0 will be the first release for which FreeBSD/EC2 is purely "bits off the ISO". This also means that starting with 10.0 it will be possible to use freebsd-update(8) for all base system updates -- in earlier releases it was necessary to recompile the XENHVM kernel manually. Due to FreeBSD's use of HVM virtualization, running on "old" EC2 instance types (m1, m2, c1, t1) requires that FreeBSD pretends to be Windows, which unfortunately results in paying the higher "windows" EC2 instance prices. On "new" EC2 instances (cc1, cc2, cg1, cr1, hi1, hs1, and m3) FreeBSD can run as a "unix" image at the lower rate. Open tasks: 1. Test FreeBSD 10.0-ALPHAs/BETAs/RCs as they become available. Plenty of new Xen code has been committed recently and there are probably bugs to find before the release. 2. Keep nagging Amazon to provide more instance types which FreeBSD can run on without paying a "Windows tax". 3. Provide some mechanism for instance configuration via EC2 user-data. This might involve using cloud-init, or it might be a new system. __________________________________________________________________ FreeBSD/pseries URL: http://svnweb.freebsd.org/changeset/base/255643 Contact: Andreas Tobler Contact: Nathan Whitehorn Starting with FreeBSD 10.0-ALPHA4, the projects/pseries branch has been merged into FreeBSD head. This allows FreeBSD/powerpc64 to run in an IBM POWER logical partition and on certain classes of older IBM-type PowerPC hardware. Open tasks: 1. Test, possibly on real hardware. Most testing and development was conducted with the emulated LPAR target in QEMU. Please send any testing reports to the freebsd-ppc mailing list. __________________________________________________________________ FreeBSD/sparc64 Contact: Marius Strobl There are several things going on with the FreeBSD/sparc64 port. After having fixed all remaining problems and starting with 9.2-RELEASE, releases for this architecture are cross-built on the FreeBSD Project cluster. As one might already have noticed, this means that from now on, sparc64 install sets and images including those for ALPHA, BETA, and RC builds, are available alongside those for the other platforms supported by FreeBSD. Since August 2013, automatically cross-built monthly FreeBSD/sparc64 snapshots are distributed via the official project mirrors. Hopefully, this can soon be extended further with freebsd-update(8) support for sparc64. The X.Org ports have been fixed to work on sparc64 when built with the WITH_NEW_XORG knob. However, it still needs to be evaluated whether the recently committed update to Mesa 9.1.6 has introduced any breakage. __________________________________________________________________ GEOM Direct Dispatch and Fine-Grained CAM Locking URL: http://svnweb.freebsd.org/base/projects/camlock/ URL: http://people.freebsd.org/~mav/camlock_patches/ Contact: Alexander Motin Last year's high-performance storage vendor summit reported a performance bottleneck in the FreeBSD block storage subsystem, limiting peak performance to around 300-500K IOPS. While that is still more than enough for average systems, detailed investigation has shown a number of places that require radical improvement. The unmapped I/O support implemented early this year has already improved I/O performance by about 30% and moved more focus toward GEOM and CAM subsystems scalability. Fixing these issues was the goal of this project. The existing GEOM design assumed most I/O handling was to be done by only two kernel threads (g_up() and g_down()). That simplified locking in some cases, but limited potential SMP scalability and created additional scheduler overhead. This project introduces the concept of direct I/O dispatch into GEOM for cases where it is known to be safe and efficient. That implies marking some GEOM consumers and providers with one or two new flags, declaring situations when a direct function call can be used instead of normal request queuing. That permits avoiding any context switches inside GEOM for the most widely used topologies, simultaneously processing multiple I/Os from multiple calling threads. Having GEOM pass through multiple concurrent calls down to the underlying layers exposed major lock congestion in CAM. In the existing CAM design, all devices connected to the same ATA/SCSI controller share a single lock, which can be quite busy due to multiple controller hardware accesses and/or code logic. Experiments have shown that applying only the above GEOM direct dispatch changes burns up to 60% of system CPU time or even more in attempts to obtain these locks by multiple callers, killing any benefits of GEOM direct dispatch. To overcome this scaling limitation, a new fine-grained CAM locking design was implemented. It implies splitting the big per-SIM locks into several smaller ones: per-LUN locks, per-bus locks, queue locks, etc. After these changes, the remaining per-SIM lock protects only the controller driver internals, reducing lock congestion down to an acceptable level and keeping compatibility with existing drivers. Together, the GEOM and CAM changes double the peak I/O rate, reaching up to 1,000,000 IOPS on contemporary hardware. The changes were tested by a number of people and will be committed into FreeBSD head and merged to stable/10 after the end of the FreeBSD 10.0 release cycle. The project is sponsored by iXsystems, Inc. Open tasks: 1. More reviews, more stability and performance tests. __________________________________________________________________ GNOME/FreeBSD URL: http://www.FreeBSD.org/gnome/ Contact: FreeBSD GNOME Team Glib 2.36 and Gtk 3.8 were imported into the ports tree. The GNOME Team is currently working on improving the quality of GNOME 3.6. The version of multimedia/cheese shipped with GNOME 3 is now able to use devd(8) to find the camera through multimedia/webcamd. Several build improvements have been made to the www/webkit-gtk3 port, however it still is rather fragile. MATE, a desktop environment forked from the now-unmaintained codebase of GNOME 2, is about ready to go in. GNOME 2 will be removed at some point in the near future. How or when this will happen is not yet clear. Open tasks: 1. Test the update. Contact the maintainers if it is suspected that a port does not work with the newer version of devel/glib20. 2. Update the FreeBSD GNOME website with recent changes in the ports tree, add new items in preparation for GNOME 3 and Mate, etc. 3. Continue working on GNOME 3.6, stability and missing features. 4. Import MATE into the ports tree. __________________________________________________________________ GNUstep on FreeBSD Contact: David Chisnall GNUstep is the open source implementation of the Objective-C APIs based on the OpenStep specification that Apple brands as Cocoa. The similarities between the FreeBSD and OS X libc make FreeBSD an attractive target platform for porting OS X applications, with the addition of GNUstep. The GNUstep ports in FreeBSD have now been updated to the latest releases and now build with the GNUstep Objective-C runtime and Clang 3.3, with the non-fragile ABI by default. This means that all of the modern features of Objective-C are supported, including Automatic Reference Counting (ARC) and recent syntax improvements. The devel/gnustep meta-port will install all of the core GNUstep libraries, ready for development. The x11/gnustep-app meta-port will install all of the GNUstep-based applications and libraries currently in the ports tree. Many of these are old and not well-tested with later GNUstep release, so consider them experimental at present. We are currently working on updating them, including moving from some abandoned upstream locations to the GNUstep Applications Project (GAP), which has taken over maintenance of a number of older GNUstep programs. __________________________________________________________________ LLDB Debugger Port URL: https://wiki.freebsd.org/lldb Contact: Ed Maste LLDB is the debugger project in the LLVM family. It supports the Mac OS X, Linux, and FreeBSD platforms. A number of improvements have been made to the port since the previous status update. Unit test failures have been triaged and have defects entered in LLDB's bug tracker. In combination with the lldb buildbot this allows for the quick identification of new failures introduced by other ongoing development. Core file support has also been added. An LLDB snapshot has been imported into the FreeBSD base system and is available as of SVN revision 255722. It is not yet built by default but may be enabled by adding WITH_LLDB=3D to src.conf(5). This project is sponsored by DARPA/AFRL in collaboration with SRI International and the University of Cambridge. Open tasks: 1. Support live debugging of multithreaded processes. 2. Fix amd64 watchpoints. 3. Add support for remote debugging (gdbserver, debugserver). 4. Add support for kernel debugging. 5. Verify i386 and arm architectures. 6. Implement MIPS target support. 7. Verify cross-debugging. 8. Investigate and fix test suite failures. __________________________________________________________________ Native iSCSI Stack URL: https://wiki.freebsd.org/Native%20iSCSI%20target Contact: Edward Tomasz Napiera=C5=82a Due to the quickly approaching time of 10.0-RELEASE, the priorities for the native iSCSI stack shifted somewhat, from performance optimizations to making sure the new stack is reliable, feature-complete, and is able to interoperate correctly with various implementations. Plenty of time was invested into testing and debugging, mostly on the initiator side, to make sure it works correctly with other targets, such as Solaris COMSTAR, and behaves properly in edge conditions like connection problems. Nevertheless, some fundamental optimizations, such as Immediate Data support, were implemented. The documentation has improved, and there will be a new section added to the FreeBSD Handbook describing the use of the new stack. The new stack was committed to head and will ship as part of 10.0-RELEASE. There is ongoing work on fixing issues reported by early adopters. This project is being sponsored by The FreeBSD Foundation. Open tasks: 1. Fix newly reported issues. 2. Improve performance. __________________________________________________________________ Reworking random(4) Contact: Mark Murray Contact: Arthur Mesh Contact: Dag-Erling Sm=C3=B8rgrav Random numbers require a lot more thought and preparation than would naively appear to be the case. For simulations, number sequences that are repeatable but sufficiently disordered are often needed to achieve required experimental duplication ability, and many programmers are familiar with these. For cryptography, it is essential that an attacker not be able to predict or guess the output sequence, thus giving a source of security-critical secret material for uses such as passwords or "key material". FreeBSD's random number generator, available as the pseudo-file /dev/random produces unpredictable numbers intended for cryptographic use, and is thus a Cryptograpically-Secured Pseudo-Random Number Generator, or CSPRNG. The security is given by careful design of the output generator (based on a block cipher) and input entropy accumulation queues. The latter uses hashes to accumulate stochastic information harvested from various places in the kernel to provide highly unpredictable input to the generator. The algorithm for doing this, Yarrow, by Schneier et al, may be found by web search. FreeBSD's CSPRNG also allowed for certain stochastic sources, deemed to be "high-quality", to directly supply the random(4) device without going through Yarrow. With recent revelations over possible government surveillance and involvement in the selection of these "high-quality" sources, it is felt that they can no longer be trusted, and must therefore also be processed though Yarrow. The matter was discussed at various levels of formality at the Cambridge Developer Summit in August, and at EuroBSDcon 2013 in September. This work is now done, and the random(4) CSPRNG is now brought to a more paranoid, modern standard of distrust with regard to its entropy sources. Infrastructure work was also done to facilitate certain entropy-source choices for the convenience of the system administrators. Future work is now going ahead with the implementation of the Fortuna algorithm by Ferguson and Schneier as an upgrade or alternative to Yarrow. Initially a choice will be presented, and decisions on the future of the CSPRNG processing algorithms in use will be made in the future as needs arise. Open tasks: 1. Implement FIPS 800-90b support. 2. A full, in-depth review of entropy. __________________________________________________________________ SDIO Driver URL: https://wiki.freebsd.org/SDIO URL: https://github.com/kibab/freebsd/tree/kibab-dplug Contact: Ilya Bakulin SDIO is an interface designed as an extension of the existing SD card standard, to allow connecting different peripherals to the host with the standard SD controller. Peripherals currently sold at the general market include WLAN/BT modules, cameras, fingerprint readers, and barcode scanners. The driver is implemented as an extension to the existing MMC bus, adding a lot of new SDIO-specific bus methods. A prototype of the driver for the Marvell SDIO WLAN/BT (Avastar 88W8787) module is also being developed, using the existing Linux driver as the reference. SDIO card detection and initialization already work, most needed bus methods are implemented and tested. There is an ongoing work to design a good locking model for the stack. The WiFi driver is able to load firmware onto the card and initialize it. Open tasks: 1. SDIO stack: Design a locking model, define how the interrupts should be processed (on SDIO controller level, MMC stack level and by child drivers). 2. Marvell SDIO WiFi: connect to the FreeBSD network stack, write the code to implement required functions (such as sending and receiving data, network scanning, and so on). 3. Implement detach path. It cannot be tested on the DreamPlug used for development, because the DreamPlug does not have an external SDIO-capable slot. __________________________________________________________________ Static Code Analysis URL: http://scan.coverity.com/ URL: http://scan.freebsd.your.org/ URL: http://clang-analyzer.llvm.org/ Contact: Ulrich Spoerlein With our own (old and unstable) instance of Coverity Prevent gone, we have now fully transitioned to the Scan project run by Coverity (see links), which Open Source projects can use to learn about possible defects in their source code. We also continue to run our code base through the Static Analyzer that is shipped with Clang/LLVM. It cannot track the state of the code over time, but has the benefit that everyone can use it without any special setup. See the home page at the links section for more information on the Clang Static Analyzer project in general, and head over to the FreeBSD Clang Static Analyzer Scan page (see links) to see those possible defects (no signup required). We are looking for a co-admin for both of these projects to increase the bus-factor and the chance of survival for these services. Fame and fortune await! Open tasks: 1. Maybe turn on email reports for new defects to the internal list of FreeBSD developers. 2. Find co-admin. 3. Fix the defects reported by Coverity and Clang. __________________________________________________________________ Superpages for ARMv7 URL: http://static.usenix.org/events/osdi02/tech/full_papers/navarro/nav= arro.pdf URL: http://wiki.freebsd.org/ARMSuperpages URL: http://blogs.arm.com/software-enablement/1079-transparent-superpage= s-for-freebsd-on-arm URL: https://wiki.freebsd.org/201309DevSummit?action=3DAttachFile&do=3Dv= iew&target=3Dsemihalf-superpages_armv7.pdf URL: http://svnweb.freebsd.org/changeset/base/254918 Contact: Zbigniew Bodek Contact: Grzegorz Bernacki Contact: Rafa=C5=82 Jaworowski The ARM architecture is becoming more and more prevalent, with increasing usage beyond the mobile and embedded space. Among the more interesting industry trends emerging in the recent months, there has been the concept of "ARM server". Top-tier companies like Dell and HP have already started to develop such systems. Key to the success of FreeBSD in these new areas is dealing with the sophisticated features of the platform, for example adding support for superpages. The objective of this project is to enable FreeBSD/arm to utilize superpages, allowing efficient use of TLB translations (by enlarging TLB coverage), leading to improved performance in many applications and scalability. This is intended to work on ARMv7-based processors, however compatibility with ARMv6 will be preserved. The following steps have been made since the last status report: * The pmap(9) module has been adjusted to fully utilize superpages. * Found and fixed minor bugs in superpage management. * Implemented the pmap_advise() routine. * Performed extensive testing and benchmarking: + Giga Updates Per Second (GUPS) benchmark: 34% lower memory access latency and 34% higher updates ratio. + LMbench: 38% lower memory latency. + Self-hosted buildworld: 20% shorter, using GCC. * Final integration into FreeBSD head. This project is jointly sponsored by The FreeBSD Foundation and Semihalf. Open tasks: 1. Adjust pmap to resolve the demotion issue caused by the continuous active queue scanning in VM. 2. Support for 64KB page size. 3. Move pv_flags to page table entry descriptors. __________________________________________________________________ The entities Documentation Branch URL: http://svnweb.freebsd.org/changeset/doc/42226 Contact: Ren=C3=A9 Ladan The entities project branch has been successfully merged into the main documentation branch per revision 42226 of the doc repository (see link). The purpose of this branch was to remove the duplicated definitions of authors in both authors.ent and developers.ent. The latter file has been removed after migrating its contents to the former file. While most changes are not visible to end users, the Committer's Guide was changed to accomodate for changes related to adding a new committer. Translators were also informed of the update. The largest hurdle mentioned in the last report, processing the element, was solved with the help of G=C3=A1bor K=C3=B6vesd=C3=A1n. __________________________________________________________________ The FreeBSD Foundation URL: http://www.FreeBSDFoundation.org/ Contact: Deb Goodkin The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Most of our funding is used to support FreeBSD development projects, conferences and developer summits, purchase equipment to grow and improve the FreeBSD infrastructure, and provide legal support for the Project. We listened to our donors who asked us to have more fundraising efforts throughout the year. This quarter we had the second of three fundraising campaigns planned for 2013. We started the quarter having raised $365,291. By the end of the quarter, we raised $410,000 for the year. These early donations have made a significant impact on our fundraising efforts this year. Some of the highlights from this past quarter include: * Projects completed last quarter: + ARM Superpages + Documentation project infrastructure enhancements * Projects in progress: + Native iSCSI kernel stack + Newcons console driver * Projects that started last quarter: + Capsicum Integration + Network Stack Layer 2 Modernization * Platinum Sponsor for EuroBSDcon, had six Foundation representatives attend the conference and the Developer Summit, sponsored 7 developers to attend the conference, and sponsored the Developer Summit. * Sponsored the Cambridge Developer Summit, and sponsored 2 developers to attend this event. * Attended Indianapolis LinuxFest July 27, FOSSCON in Philadelphia August 10, Ohio LinuxFest in Columbus September 14, and LinuxCon in New Orleans September 16-17, to promote FreeBSD. * Met with the FreeBSD Core Team to discuss their goals and to discuss areas that we can help. * Met with the Documentation Team to talk about helping them update their website as well as what other areas we can help them with. * Recognized Dag-Erling Sm=C3=B8rgrav at EuroBSDcon for his contributi= ons to FreeBSD. * Became a sponsor of vBSDCon, a new conference in Washington, DC. * Hired Glen Barber as a full-time employee to do system administration work and to help with release engineering. * Hired Cinthy Tanko as a part-time administrative assistant to help with day-to-day Foundation activities. * Purchased hardware to be placed in our NYI colo to support the building and distribution of new style packages in advance of FreeBSD 10. * Provided teleconferencing services to the Core Team to support their monthly conferences. __________________________________________________________________ VirtIO Network Multiqueue URL: http://svnweb.freebsd.org/changeset/base/255112 Contact: Bryan Venteicher The VirtIO network driver, vtnet(4), is used by FreeBSD systems running on hypervisors including bhyve(4) and Linux's KVM. It recently gained support for multiple queues, along with a significant cleanup and support for a few additional features. __________________________________________________________________ VMware VMXNET3 Driver URL: http://lists.freebsd.org/pipermail/freebsd-current/2013-August/0434= 94.html URL: http://svnweb.freebsd.org/base/head/sys/dev/vmware/vmxnet3/ Contact: Bryan Venteicher A port of the OpenBSD vmx(4) ethernet driver for VMware virtual machines has been committed. The driver can be used in place of the VMware Tools vmxnet3 driver, which currently does not support 10.0-RELEASE (or anything past 9.0-RELEASE). Open tasks: 1. Performance improvements, multiqueue support. 2. Merge to stable/9. __________________________________________________________________ X.Org on FreeBSD URL: https://wiki.freebsd.org/Graphics URL: https://wiki.freebsd.org/Xorg URL: http://trillian.chruetertee.ch/ports/browser/trunk URL: https://wiki.freebsd.org/AMD_GPU URL: https://github.com/dumbbell/freebsd/tree/kms-drm-update-38 Contact: FreeBSDX11 Team Mesa 9.1 (libGL and dri) was updated in ports. This includes experimental ports for libEGL and libgles2: they are dependencies of the experimental ports for Wayland and Weston. The radeonkms driver was committed to FreeBSD head in the end of August and will be part of 10.0-RELEASE. It received several fixes since the initial commit and now seems quite stable. However, one missing major feature is support for suspend/resume: the GPU almost always locks up during resume on the test computer. Thanks to the update of Mesa and the update of x11-drivers/xf86-video-ati to 7.2.0 in the ports tree, every pieces are in place to allow users to use recent AMD video cards (up to HD7000, maybe some HD8000). The driver will now only receive bug fixes and focus will move on the update of the DRM generic code and the i915 driver. The generic DRM code, shared by the i915kms and radeonkms video drivers is quite old now. Work has started to update and sync it with that of Linux 3.8. This code is available on GitHub. The expected benefits are: * Fixes in the framebuffer code, which would help the future deployment of Newcons. * Preliminary support for minor devices (that is, control versus render nodes). * Support for setmaster and dropmaster, which allows to run multiple X sessions. Fran=C3=A7ois Tigeot from DragonFly is also working on updates to their = DRM code, and the X11 team is planning to share the effort. An experimental devd(8) backend was added to the x11-servers/xorg-server port. This allows X.Org to use devd(8) to detect and configure input devices (for example, keyboards and mices) dynamically. Our current wiki articles are used to describe projects and report status. However, they lack some consistency and links between them. We started to think about reorganizing them to: * Improve the coordination between the ports and the kernel efforts. * Make the information more accessible. Nothing is visible yet on the wiki. Open tasks: 1. Keep tracking Mesa 9.2 or later and xorg-server 1.14. Both are currently blocked, but it is good to keep track of what upstream is doing. 2. Test and report successes and failures for AMD GPUs. 3. Wayland builds now. Work is being done on Weston to see if there are any run-time issues. Weston is the reference compositor for Wayland. 4. Improve the devd(8) backend for x11-servers/xorg-server, so the HAL option can be removed completely. __________________________________________________________________ From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 21 03:58:01 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4DAECBA3 for ; Mon, 21 Oct 2013 03:58:01 +0000 (UTC) (envelope-from sean_bruno@yahoo.com) Received: from nm3-vm2.bullet.mail.ne1.yahoo.com (nm3-vm2.bullet.mail.ne1.yahoo.com [98.138.91.19]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E04EE271F for ; Mon, 21 Oct 2013 03:58:00 +0000 (UTC) Received: from [98.138.90.53] by nm3.bullet.mail.ne1.yahoo.com with NNFMP; 21 Oct 2013 03:55:06 -0000 Received: from [98.138.226.127] by tm6.bullet.mail.ne1.yahoo.com with NNFMP; 21 Oct 2013 03:55:06 -0000 Received: from [127.0.0.1] by smtp206.mail.ne1.yahoo.com with NNFMP; 21 Oct 2013 03:55:06 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1382327706; bh=N6HtowDlgMKlTEa5zz4G4tktmNkN4QcFDivlozPOGqY=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Subject:From:Reply-To:To:Content-Type:Date:Message-ID:Mime-Version:X-Mailer; b=Jy+vawFewt2S5bVhpQVkJ5Jv4nmt9eL5PkOlfv9kMx1N9kzBCmbWCd1+fpDPRpBJnE+dfOxKo1tAnCixXFBUSifAnbnwX4PEimgPn5+lA6COUnfedmb6+oIuuoPdbjZ8Q865t67rDdAzEC+fYySHE/rktuUasvEPbsAxM5QG3GQ= X-Yahoo-Newman-Id: 319148.53035.bm@smtp206.mail.ne1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: WUw4q7MVM1l6ZZ0SJJ6UkmaJ7bOqoNB652Iif_YMVkqN6v9 mMKJnHEnJwNHLjcPdZE6gvArxB1mRvdUobwoFUfFM2lYW4ZXmkfiyQxmYIkL RCl2i.dNZVadE1OkQhcWLICDx.bXZV4nMhtLxnxrij9Eb9LUa4aX08PWysBA IrtC5m6E7Udmqr2ThlVqt.pD4hjPbdD.VMtKLTiZgHEBaGMrr3SHEll7XfLm w5B_DlYtuXpXshE_cO6CHfK072TG4kZ2bL_yiHZeRyhtTpfK0CqzRFX9xeHE aROJxrfhilrKKDtOXcc.nJxCiLPzfgOXBgM9Pa34mhEe73aaooM2gPiMTmg3 WJPdSaaCIq_oODSUz7y2WAhLnztq60in7TuBQ.EG1bpKBbxlnUsD7O2q6xA4 9MESHhZp0TEgN9yjlsHiQ5X6TXW5aPnxvwoZ6KdlJRXzdBVulkVJX5eoYXow j13bbD_2JPH_J67PluLzWlB089H.byiWlKGqUZLggOg_JOY2xBvhsiU09Z1v fHrD6ZN0QC1oC_oJXDjr1Ln4qGXBL5TlOOEr11n4JKJtmz4UK6PDi2AeY4P1 DCC3xOj1w1iygl0LpujUWGuLTXlNL75vgcQgddOJWWJmqzOk9jpwjbxeHoLn j X-Yahoo-SMTP: u5BKR6OswBC_iZJVfGRoMkTIpc8pEA4- X-Rocket-Received: from [192.168.100.108] (sean_bruno@63.138.121.126 with ) by smtp206.mail.ne1.yahoo.com with SMTP; 20 Oct 2013 20:55:06 -0700 PDT Subject: gperf -- #define for if (0) ; else for From: Sean Bruno To: "freebsd-hackers@freebsd.org" Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-SA48uZOt/F4H/9FUwr1p" Date: Sun, 20 Oct 2013 23:55:05 -0400 Message-ID: <1382327705.2610.9.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Oct 2013 03:58:01 -0000 --=-SA48uZOt/F4H/9FUwr1p Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I don't even know why this is a thing in our code base. Its generating a lot of clang noise due to -Wdangling-else /usr/src/gnu/usr.bin/gperf/../../../contrib/gperf/src/search.cc:417:15: warning: add explicit braces to avoid dangling else [-Wdangling-else] for (int i3 =3D imax; i3 >=3D 0; i3--) ^ /usr/src/gnu/usr.bin/gperf/../../../contrib/gperf/src/search.cc:39:22: note: expanded from macro 'for' #define for if (0) ; else for ^ I see no reason to continue this and propose the removal of the line in contrib/gperf/src/search.cc=20 37 38 /* Assume ISO C++ 'for' scoping rule. */ 39 #define for if (0) ; else for --=-SA48uZOt/F4H/9FUwr1p Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQEcBAABAgAGBQJSZKWZAAoJEBkJRdwI6BaH6FcH/A/LCXTq/1M1VBCJtsxLzFJ0 U2W6PpjdgDjwtsWdJCHoiFpyp8LCuYKfKRYkbk79dljWBwnMksA6RFfCop8qVcXM K+d7EwRjLyaFAenatAs/i0V7MAed7ajJERFejoCP271X+VzOg0Z1bhpGMiU8tC2Z hEz2dR7f1KLkkBGutQ/XV5GQB1fI0AcA1NyNTcfVXHpNe+NgBh6Lavd4+LDf9uQ5 1UTR88ZqsE6OeP+4LWf60vlPWZTHlmyjsiqM0LJ0nIJF9vazw5frI+BpOhf7FdB4 MQhA34ip2HtnlKqVA1DPoUCedQjyQ0nkG7yns2PgvFeTvVoGvEhvz4GHfLGdOm0= =8JJa -----END PGP SIGNATURE----- --=-SA48uZOt/F4H/9FUwr1p-- From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 21 04:18:29 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 7F09D1A9; Mon, 21 Oct 2013 04:18:29 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-5.mit.edu (dmz-mailsec-scanner-5.mit.edu [18.7.68.34]) by mx1.freebsd.org (Postfix) with ESMTP id 25B5A284B; Mon, 21 Oct 2013 04:18:28 +0000 (UTC) X-AuditID: 12074422-b7f5a8e000000a34-58-5264a9e20e9f Received: from mailhub-auth-4.mit.edu ( [18.7.62.39]) by dmz-mailsec-scanner-5.mit.edu (Symantec Messaging Gateway) with SMTP id 18.19.02612.2E9A4625; Mon, 21 Oct 2013 00:13:22 -0400 (EDT) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id r9L4DMwG023027; Mon, 21 Oct 2013 00:13:22 -0400 Received: from multics.mit.edu (system-low-sipb.mit.edu [18.187.2.37]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id r9L4DKTT021969 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 21 Oct 2013 00:13:21 -0400 Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id r9L4DKgj010120; Mon, 21 Oct 2013 00:13:20 -0400 (EDT) Date: Mon, 21 Oct 2013 00:13:19 -0400 (EDT) From: Benjamin Kaduk To: sbruno@freebsd.org Subject: Re: gperf -- #define for if (0) ; else for In-Reply-To: <1382327705.2610.9.camel@localhost> Message-ID: References: <1382327705.2610.9.camel@localhost> User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrNIsWRmVeSWpSXmKPExsUixG6nrvtoZUqQwec/ehbbN/9jtOjpPcHo wOQx49N8lgDGKC6blNSczLLUIn27BK6M4zsPsRd0cVQc+HCCvYFxBlsXIyeHhICJxLKGy+wQ tpjEhXvrgeJcHEIC+xglDvdPZoJwNjJKrOy+ApU5xCRx/+8kRgingVHi/8KVYP0sAtoSG35t YAax2QRUJGa+2Qi2QwRo7rR/dxhBbGYBe4l/bf9ZQGxhAWOJ/rO/WUFsTgEDiZ3zjoDN4RVw kNg37zxYvZCAvsSpmauYQGxRAR2J1funsEDUCEqcnPmEBWKmpcS5P9fZJjAKzkKSmoUktYCR aRWjbEpulW5uYmZOcWqybnFyYl5eapGuqV5uZoleakrpJkZwiLoo7WD8eVDpEKMAB6MSD2+A S0qQEGtiWXFl7iFGSQ4mJVHev8uAQnxJ+SmVGYnFGfFFpTmpxYcYJTiYlUR45ywAyvGmJFZW pRblw6SkOViUxHlvcdgHCQmkJ5akZqemFqQWwWRlODiUJHh1gLEoJFiUmp5akZaZU4KQZuLg BBnOAzScF6SGt7ggMbc4Mx0if4rRmOPf2g/fGDmOvAOSQix5+XmpUuK8KiClAiClGaV5cNNg aeYVozjQc8K8TiBVPMAUBTfvFdAqJqBVGhpJIKtKEhFSUg2MiQcnlm7ZaJm54v4S2X8Z24Tt vySIP7VrYrWs+Nx4Y4rjh3aJzhL7/hdVvCtnZJVGPHFtjdV/YNtrtF+51POT8L3/e//eK9B3 Wlo4+9HbijtpuksNtbKXOX7Med+09Or7x3uj3r1jYje9+ihaZH3eGrWigzIh7HYqD24/2vKO neFJfdrRs29dlViKMxINtZiLihMBsnXebg4DAAA= Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Oct 2013 04:18:29 -0000 On Sun, 20 Oct 2013, Sean Bruno wrote: > I don't even know why this is a thing in our code base. Its generating > a lot of clang noise due to -Wdangling-else > > /usr/src/gnu/usr.bin/gperf/../../../contrib/gperf/src/search.cc:417:15: > warning: add explicit braces to avoid dangling else [-Wdangling-else] > for (int i3 = imax; i3 >= 0; i3--) > ^ > /usr/src/gnu/usr.bin/gperf/../../../contrib/gperf/src/search.cc:39:22: > note: expanded from macro 'for' > #define for if (0) ; else for > ^ > > I see no reason to continue this and propose the removal of the line in > contrib/gperf/src/search.cc > > 37 > 38 /* Assume ISO C++ 'for' scoping rule. */ > 39 #define for if (0) ; else for StackOverflow (!) [1] suggests that they're a workaround for a bug in old versions of Visual Studio. Someone hand Sean the danish axe, please. -Ben [1] http://stackoverflow.com/questions/984878/what-is-the-possible-use-for-define-for-if-false-else-for From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 21 06:38:09 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B2416A10 for ; Mon, 21 Oct 2013 06:38:09 +0000 (UTC) (envelope-from adutkowski@gmail.com) Received: from mail-oa0-x22f.google.com (mail-oa0-x22f.google.com [IPv6:2607:f8b0:4003:c02::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7F1592DDA for ; Mon, 21 Oct 2013 06:38:09 +0000 (UTC) Received: by mail-oa0-f47.google.com with SMTP id i18so1436363oag.20 for ; Sun, 20 Oct 2013 23:38:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=27UkU567cAqwHQY1APm6iQT3vVp8YP35ph0SF7/q74s=; b=gnxuOm4kJQy8ciudjkwvri3rHbQ5+CoBkb43aNO7dqPLkPpIuFG3iX3ddf+0Z4Fnbo 7X1Zih1b9Kwya9SQGCECh8R39CXma8Sgfdd/+fZzjUmmDXfdwTtc8u/dFEo20qXvpWVK KqwtMsAwiEU+DGmGrw5A5sINQ9IcpkgzjeNeXzLhP/reEDwOtx5cjgJbbggO6OJmE1y/ tNTpIbK3Hd/ogH9lJT1tII+iOO79dhRk2sZS56YBYI3GvA0zbpW+OXsv4EDNNq/n8YN+ uEYKXE4eNWOKIMn1gqCz7xPUzrNdsKH43leffklkbVuKPEFLV/nz07kCr0klM21vrajU czYw== MIME-Version: 1.0 X-Received: by 10.182.121.137 with SMTP id lk9mr19385310obb.32.1382337488647; Sun, 20 Oct 2013 23:38:08 -0700 (PDT) Sender: adutkowski@gmail.com Received: by 10.76.132.9 with HTTP; Sun, 20 Oct 2013 23:38:08 -0700 (PDT) Date: Mon, 21 Oct 2013 08:38:08 +0200 X-Google-Sender-Auth: nklCvpQxzlAwatX8-LlZg-SKPsw Message-ID: Subject: modyfing cross toolchain compilation From: Aleksander To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Oct 2013 06:38:09 -0000 hello! Few days ago, when playing with linux-ppc, I have identified a bug with PowerPC toolchain. It turns out that PowerPC e500 core is not compilant with generic ppc ISA - it lacks of 'lwsync' instruction. This instruction is used in libstdc++.so library, so I need to modify toolchain build process by adding -me500v2 option, when building libraries. Can you tell me how to do it, so I can test this potential soluton? -- regards aleek From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 21 10:30:05 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 570FEFFF; Mon, 21 Oct 2013 10:30:05 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (muon.cran.org.uk [93.89.92.64]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 144752BC9; Mon, 21 Oct 2013 10:30:04 +0000 (UTC) Received: from muon.cran.org.uk (localhost [127.0.0.1]) by muon.cran.org.uk (Postfix) with ESMTP id A9319E6DF2; Mon, 21 Oct 2013 11:29:56 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=cran.org.uk; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; s=mail; bh=F6YMoaTnPo/L crFSeGJSxX5LdkQ=; b=S6tRdzQ+uPdbBjcWEKGHHXP08nRikY2yDwadL+HTTq9N LgwbkmRGrc2qtFqOVrjZ8bJPAEne50i7Sp2eA1ifItUu9IcWPPqUN7mU2BulBIzh gvzkWzG354AIyP787pAk2DmtxTmeK6C3HId0erIPvuw4cAHBtoUeVDqVC7MyrMg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=cran.org.uk; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; q=dns; s=mail; b=DtGUrE oUt8ATc81lnR5LDe19K2y+ChoE8egKqU9Ej31sBo8/WqxRWVlyMVG6n8o1QUmTq/ qAJ8nG0nROEfwChIIWfwXgbAtZYATIoK4gxu4/TIQbcnTJU+tFI8L7qO3hpsyg3L khs5a6JIrLnrPUmV6UNc5UndVl2kbCtJAa/y0= Received: from [192.168.2.103] (unknown [93.89.81.205]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id 7B08DE6DC4; Mon, 21 Oct 2013 11:29:56 +0100 (BST) Message-ID: <52650223.2020707@cran.org.uk> Date: Mon, 21 Oct 2013 11:29:55 +0100 From: Bruce Cran User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Benjamin Kaduk , sbruno@freebsd.org Subject: Re: gperf -- #define for if (0) ; else for References: <1382327705.2610.9.camel@localhost> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Oct 2013 10:30:05 -0000 On 10/21/2013 5:13 AM, Benjamin Kaduk wrote: > >> 37 >> 38 /* Assume ISO C++ 'for' scoping rule. */ >> 39 #define for if (0) ; else for > > StackOverflow (!) [1] suggests that they're a workaround for a bug in > old versions of Visual Studio. http://msdn.microsoft.com/en-us/library/b80153d8%28v=vs.90%29.aspx also documents it. Visual C++ 6 was released in 1998, which was unfortunately the same year as the first ISO C++ standard. -- Bruce Cran From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 21 13:30:39 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A2BF7451; Mon, 21 Oct 2013 13:30:39 +0000 (UTC) (envelope-from rank1seeker@gmail.com) Received: from mail-ee0-x22e.google.com (mail-ee0-x22e.google.com [IPv6:2a00:1450:4013:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 199DB2855; Mon, 21 Oct 2013 13:30:38 +0000 (UTC) Received: by mail-ee0-f46.google.com with SMTP id c13so3594588eek.5 for ; Mon, 21 Oct 2013 06:30:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:to:cc:subject:date:in-reply-to:references; bh=iG+TvQuRVihfm/O6YXLr+oCFjKxvJiG9W9PnnnnueqI=; b=hNqY7yX/HzNcmU/UdXk9kyGd0s8w9w0lHkxQO7CgMPQh3jHAG5ZB3cboJGqb8yOUlc s6f6jsLHME8hEjGWj9SckL8YuQ6HOwKLOMk/VG3AdHP7r90Mv7L2EYFHhLqNa7hG5Prn DodRti0NrtSYmw12n8CQ1PnfOIzwPSnR/acMALhi7Y4zIGaJzeUcwYB5gfKbKLWQ95Dp oQwWY9XADU3DgxmaPp/ahtNPxXZvx8WfxegcXWYolqqMre1kC9xi7sUbROP9e9lVttWT z6Cml0SDRUbZV8NJ7/4qZDZ13yBNz+1AYKqt7z0KPHPy8uvid5tKvkMQF9k+Pr8rqf3+ oAvA== X-Received: by 10.15.81.137 with SMTP id x9mr1953361eey.77.1382362237463; Mon, 21 Oct 2013 06:30:37 -0700 (PDT) Received: from DOMYPC ([82.193.208.225]) by mx.google.com with ESMTPSA id b42sm43828623eem.9.2013.10.21.06.30.35 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 21 Oct 2013 06:30:36 -0700 (PDT) Message-ID: <20131021.133036.045.1@DOMY-PC> From: rank1seeker@gmail.com To: "John Baldwin" Subject: Re: UFS related panic (daily <-> find) Date: Mon, 21 Oct 2013 15:30:36 +0200 In-Reply-To: References: <20130719.174511.786.3@DOMY-PC> <201310071212.05281.jhb@freebsd.org> <20131016.104912.479.1@DOMY-PC> <201310161650.52354.jhb@freebsd.org> X-Mailer: POP Peeper (3.8.1.0) Cc: Adam Vande More , hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Oct 2013 13:30:39 -0000 > > Same drill as before, see what instruction this is. Actually, this looks > > to > > be in the same location as your last panic, so a NULL pointer is 0x1 > > instead > > of 0x0 again. In my experience, this would still indicate failing RAM to > > me, > > memtest86+ notwithstanding (memtest86+ is single threaded AFAIK, so it may > > not stress the hardware quite the same, e.g. if the error is heat related, > > etc.). > > > memtest* cannot conclusively diagnose a dimm as good. Usually the only > practical solution is to swap modules with known good ones. > 0xc082c552 : cmp %ecx,0x24(%eax) PREVIOUS we talked about 0xc083bd42 : cmp %ecx,0x24(%eax) CURRENT ONE Lattest (few days ago): -- #7 0xc06d5f11 in cache_lookup_times (dvp=0xc921b470, vpp=0xe7c24ae8, cnp=0xe7c24afc, tsp=0x0, ticksp=0x0) at /usr/src/sys/kern/vfs_cache.c:548 548 numchecks++; (kgdb) p ncp $1 = (struct namecache *) 0x1 (kgdb) p *ncp Cannot access memory at address 0x1 -- Now, after all this I recompiled kernel and world and there was no crash. How can it be, when it is far more stresing dan daily's 'find'?! Why does exactly daily's 'find' AND not every time, but each 10th or 20th time triggers this? I see addresses 0xc08* and 0xc06* appearing each time, so as I have four DDR1 (400) modules, each of 256 MB = 1GB, can those addresses aid me in targeting failing module? If I can't use memtest86+-4.20, to determine failing module, then what is a use of it at all? Test RAM speed perhaps? I might try to pull out module 2 and 4 (dual channel) and wait for a month to see if find will crash machine. Then another month ... Well you get a point. There must be other solution. Thanks for your help. Domagoj From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 21 13:45:10 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 847C2B5A; Mon, 21 Oct 2013 13:45:10 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from smtpauth3.wiscmail.wisc.edu (wmauth3.doit.wisc.edu [144.92.197.226]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 55893293E; Mon, 21 Oct 2013 13:45:06 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from avs-daemon.smtpauth3.wiscmail.wisc.edu by smtpauth3.wiscmail.wisc.edu (Oracle Communications Messaging Server 7u4-27.01(7.0.4.27.0) 64bit (built Aug 30 2012)) id <0MV000H00RAU4A00@smtpauth3.wiscmail.wisc.edu>; Mon, 21 Oct 2013 07:44:58 -0500 (CDT) X-Spam-PmxInfo: Server=avs-3, Version=6.0.3.2322014, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2013.10.21.123314, SenderIP=0.0.0.0 X-Spam-Report: AuthenticatedSender=yes, SenderIP=0.0.0.0 Received: from wanderer.tachypleus.net (adsl-76-208-69-44.dsl.mdsnwi.sbcglobal.net [76.208.69.44]) by smtpauth3.wiscmail.wisc.edu (Oracle Communications Messaging Server 7u4-27.01(7.0.4.27.0) 64bit (built Aug 30 2012)) with ESMTPSA id <0MV00008ZREW5V20@smtpauth3.wiscmail.wisc.edu>; Mon, 21 Oct 2013 07:44:58 -0500 (CDT) Message-id: <526521C8.2030003@freebsd.org> Date: Mon, 21 Oct 2013 07:44:56 -0500 From: Nathan Whitehorn User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 To: Aleksander , freebsd-hackers@freebsd.org Subject: Re: modyfing cross toolchain compilation References: In-reply-to: X-Enigmail-Version: 1.5.2 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Oct 2013 13:45:10 -0000 On 10/21/13 01:38, Aleksander wrote: > hello! > > Few days ago, when playing with linux-ppc, I have identified a bug with > PowerPC toolchain. It turns out that PowerPC e500 core is not compilant > with generic ppc ISA - it lacks of 'lwsync' instruction. This instruction > is used in libstdc++.so library, so I need to modify toolchain build > process by adding -me500v2 option, when building libraries. Can you tell > me how to do it, so I can test this potential soluton? > Which core is this? sync (and lwsync) are supposed to be supported on all PowerPC processors, since they are part of Book II and is *supposed* to be transparently interpreted as "sync" if not supported since the L field of the encoding was reserved previous to the introduction of lwsync. So I think your CPU is broken. It seems like GCC does have some hacks for this and may do the right thing via adding some #defines if you set CPUTYPE=8450 in make.conf, but it doesn't look like the relevant glue in config/rs6000 is hooked up for that at present. Without those, since the instruction is supposed to be valid, it will happily compile and assemble it. So an immediate solution is to just change the instruction in libstdc++ to "sync". Longer term, probably the Book-E kernel should trap lwsync illegal instruction faults and interpret them as "sync" (or do nothing, actually, since a trap is as good as a sync) on such broken cores. I doubt this is the last piece of code is userland or ports that assumes that Book-II instructions are valid. This keeps binary compat with compliant cores. We should also fix up GCC so it sets TARGET_E500 if the right CPUTYPE is set so that natively compiled executables don't hit the trap. -Nathan From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 21 16:42:24 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id CA3CB2A1; Mon, 21 Oct 2013 16:42:24 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4174B2475; Mon, 21 Oct 2013 16:42:24 +0000 (UTC) Received: by mail-wi0-f179.google.com with SMTP id hm4so4331406wib.0 for ; Mon, 21 Oct 2013 09:42:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=PTPCk3QmAYk3QxpI4TgjMn8vyXjIjXw9zjQWdshS8ss=; b=jucpwoRm/tMJyLl7Azd1nlGtG1TRDHDN12NOsqt60fKWJbBBEL/7RscUnazV86Wxnz sgxZ0P2CH7mMakJ2xeEk/v7mEP2fcgR8UlhtsCw859sdFwZjFzHYRV/dZwnBbBpK0hwe kILH7ZkTgbMb9sW1XzgiKdBT0zgRS+GV3acNHB0A0SR/q77S1kGMOOTsGFUXRnF1PnZ6 g2yasOOhLNkw9v5j3VXOX95/4hLH/BS/SQwinLrTPoVcdukrE6h7AWRXoN0MGSvI4g9d aB9BTFqKc2/+NJg4RZvpprhdvQ82FyvCQO3mPgS8UN+8lFCCKkjRpZ1+IKWf2VZbFXVG hx5g== MIME-Version: 1.0 X-Received: by 10.181.11.163 with SMTP id ej3mr10373668wid.47.1382373742563; Mon, 21 Oct 2013 09:42:22 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.171.35 with HTTP; Mon, 21 Oct 2013 09:42:22 -0700 (PDT) In-Reply-To: <1382286899.92499.110.camel@revolution.hippie.lan> References: <1382192148.119437035@f257.i.mail.ru> <1382286899.92499.110.camel@revolution.hippie.lan> Date: Mon, 21 Oct 2013 10:42:22 -0600 X-Google-Sender-Auth: zuwx3rYZJy-iB5emjY_xyLJvdXc Message-ID: Subject: Re: determine drive's SAS port From: Alan Somers To: Ian Lepore Content-Type: text/plain; charset=ISO-8859-1 Cc: Alexey Egorov , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Oct 2013 16:42:25 -0000 On Sun, Oct 20, 2013 at 10:34 AM, Ian Lepore wrote: > On Sat, 2013-10-19 at 18:15 +0400, Alexey Egorov wrote: >> Hello all, >> >> I have a server with LSI HBA card, and when I remove drive I can see following messages in log: >> >> (da0:mps0:0:5:0): lost device - 0 outstanding, 2 refs >> (da0:mps0:0:5:0): removing device entry >> >> Is there a way to determine physical port (number "5" in "(da0:mps0:0:5:0)") when drive is inserted? (I need this to be able to create device symlinks based on physical port). >> >> Thanks in advance. > > I don't have hardware like that to play with, but when I plug in an > eSata drive I get devd events like this: > > root@dpnand:/root # cat /var/run/devd.pipe > !system=DEVFS subsystem=CDEV type=CREATE cdev=pass2 > !system=DEVFS subsystem=CDEV type=CREATE cdev=ada0 > !system=DEVFS subsystem=CDEV type=CREATE cdev=ad0 > !system=DEVFS subsystem=CDEV type=CREATE cdev=ada0p11 > !system=DEVFS subsystem=CDEV type=CREATE cdev=ad0p11 > !system=DEVFS subsystem=CDEV type=CREATE cdev=diskid/DISK-10MS109LT74Z > !system=DEVFS subsystem=CDEV type=CREATE cdev=ufsid/51fabc51ea1a923b > !system=DEVFS subsystem=CDEV type=CREATE cdev=gptid/c057d696-fae3-11e2-b79c-5404a6f2f88a > !system=DEVFS subsystem=CDEV type=CREATE cdev=diskid/DISK-10MS109LT74Zp11 > ^C > > The pass2 dev appears in a camcontrol devlist, like this: > > root@dpnand:/root # camcontrol devlist > at scbus0 target 0 lun 0 (pass2,ada0) > at scbus2 target 0 lun 0 (pass0,da0) > at scbus2 target 0 lun 1 (pass1,da1) > > Would the camcontrol bus/target/lun output give you the number you're > looking for? It's a pity there isn't a devd event with more info in it > (similar to what you get when usb devices come and go), but perhaps with > some scripting you can make the connection between events and devices. > > -- Ian > > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" If your disk is connected to a SAS expander, then you can use either SES or SMP commands to figure out the port to which a drive is connected. But it sounds like your disks are connected directly to the HBA? If that's the case, the problem is much harder. I don't think that there is a standard way to get it, and the FreeBSD kernel doesn't currently help much. mps(4) accepts some custom ioctls to look up configuration information. To learn how to use them, ask your LSI FAE for the "Fusion-MPT 2.0 Message Passing Interface (MPI) Specification Guide". The best solution would be to update the kernel to report physical path information for da disks. I think that information is currently reported only for disks connected to a SES enclosure. -Alan From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 21 22:48:17 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B59FDAF8 for ; Mon, 21 Oct 2013 22:48:17 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8D93B2C6E for ; Mon, 21 Oct 2013 22:48:17 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 8F47DB94A; Mon, 21 Oct 2013 18:48:16 -0400 (EDT) From: John Baldwin To: rank1seeker@gmail.com Subject: Re: UFS related panic (daily <-> find) Date: Mon, 21 Oct 2013 18:47:00 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <20130719.174511.786.3@DOMY-PC> <20131021.133036.045.1@DOMY-PC> In-Reply-To: <20131021.133036.045.1@DOMY-PC> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201310211847.00875.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 21 Oct 2013 18:48:16 -0400 (EDT) Cc: Adam Vande More , hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Oct 2013 22:48:17 -0000 On Monday, October 21, 2013 9:30:36 am rank1seeker@gmail.com wrote: > > > Same drill as before, see what instruction this is. Actually, this > looks > > > to > > > be in the same location as your last panic, so a NULL pointer is 0x1 > > > instead > > > of 0x0 again. In my experience, this would still indicate failing RAM > to > > > me, > > > memtest86+ notwithstanding (memtest86+ is single threaded AFAIK, so it > may > > > not stress the hardware quite the same, e.g. if the error is heat > related, > > > etc.). > > > > > > memtest* cannot conclusively diagnose a dimm as good. Usually the only > > practical solution is to swap modules with known good ones. > > > > > 0xc082c552 : cmp %ecx,0x24(%eax) > PREVIOUS we talked about > 0xc083bd42 : cmp %ecx,0x24(%eax) > CURRENT ONE Different instruction pointer doesn't matter. The error is in the memory that %eax is loaded from in a prior instruction. > Now, after all this I recompiled kernel and world and there was no crash. > How can it be, when it is far more stresing dan daily's 'find'?! Because it might have shuffled where the bad memory cell now lives by having the kernel text + data laid out differently in RAM? > I see addresses 0xc08* and 0xc06* appearing each time, so as I have four > DDR1 (400) modules, each of 256 MB = 1GB, can those addresses aid me in > targeting failing module? The virtual addresses (0xc*) do not matter. They are not physical addresses which are what you would need. > If I can't use memtest86+-4.20, to determine failing module, then what is a > use of it at all? > Test RAM speed perhaps? Swap out your dimms. That's really the only test, esp. if you have a reproducible crash. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 21 22:53:02 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6BDE5D07 for ; Mon, 21 Oct 2013 22:53:02 +0000 (UTC) (envelope-from sean_bruno@yahoo.com) Received: from nm23-vm4.bullet.mail.gq1.yahoo.com (nm23-vm4.bullet.mail.gq1.yahoo.com [98.136.217.83]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 38D382CF6 for ; Mon, 21 Oct 2013 22:53:01 +0000 (UTC) Received: from [98.137.12.59] by nm23.bullet.mail.gq1.yahoo.com with NNFMP; 21 Oct 2013 22:50:22 -0000 Received: from [98.136.164.71] by tm4.bullet.mail.gq1.yahoo.com with NNFMP; 21 Oct 2013 22:50:22 -0000 Received: from [127.0.0.1] by smtp233.mail.gq1.yahoo.com with NNFMP; 21 Oct 2013 22:50:22 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1382395822; bh=ZZOhUrdITcngnYerBGfEChISRjwIaf/kwcZMDyOT7R0=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Subject:From:Reply-To:To:Cc:In-Reply-To:References:Content-Type:Date:Message-ID:Mime-Version:X-Mailer; b=k8ApdgXYd9iAAYwIyc6q5mELWxihyz7iWpoBPNwt1/re2MNynx6EN+X/XkwsQ8ee8XnuGmZtTKYIy1fxrRJ9fSat9UZPbbJ7BfEgTVLYxBeg5YAei6Td4AA3toyzqVlLAqwP9U6hn8bSkTYuRIUVC2qJ6h9rD4m6sEw2ytmswn4= X-Yahoo-Newman-Id: 235955.23168.bm@smtp233.mail.gq1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: 7MdT_SIVM1kdYLBJ0Yv1PwIImwGbc9aTW3hfyOJ3ezSCZYN yjDVNCp9j_r3h4Cdqfap0XWoWTL_T9bS_0RtTbnkbUkiOwzOISP7tRIxSa8x aGJbfKxPdqcDVl0ZosKHwlrE5qkN143O0PA_H5bXqLMio2FvadcdeYLp_fqX VhmxlHLwFCJUZ490.jJNLP9EItwrveOkVle0zE3MmlsqIkhyoIFCxk7g.7fL A43sZ06gBAsY3MJUkzw61WkOSETfrmlnIl3vfuQ82yydoQWkS7rzGrgY5cj_ tTn1KwRqR0E83l92CU_8likapNeO0ghvsTEpYzCCHJXW87GLxBurWFxJx4Vt DFh7ZTvXg5Yri0bT0GxThRwK.weEgLnpdQ9V8h7Y612IZTmuAbBwAtJgnbSL urfwTFdZUuns_1PkAGycs8caHaLCiZUhhdiEyoJUbfUllA3zXf2QeDt8hV09 DmQmZktzZK9Y4yg2Uc6Ju0q9fAEgKLga9ci4LjEDDAKTqWJ0BIlbr8o26GAS k9wHDdhC7gH3W3djVp0Sg16t6E7kFSDv202XUDa1lSVSh.MJxld0dZ8OzB4G j533d47I6.iDcV6azvfSzgTrcFWBYVk9Tu5KH8JfKIYX4UgMVPav2vJ5JEJd 7dgD6ZFCv5Jb442VU63G16EGiwET2HD_t0BPQ1iabnqz0DzyNt43ZqG39RCp SPDEUZRmpgg-- X-Yahoo-SMTP: u5BKR6OswBC_iZJVfGRoMkTIpc8pEA4- X-Rocket-Received: from [192.168.1.3] (sean_bruno@96.47.64.130 with ) by smtp233.mail.gq1.yahoo.com with SMTP; 21 Oct 2013 22:50:22 +0000 UTC Subject: Re: gperf -- #define for if (0) ; else for From: Sean Bruno To: Bruce Cran In-Reply-To: <52650223.2020707@cran.org.uk> References: <1382327705.2610.9.camel@localhost> <52650223.2020707@cran.org.uk> Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-BMfSLVO6WJOEvHDsj1vT" Date: Mon, 21 Oct 2013 18:50:20 -0400 Message-ID: <1382395820.4447.1.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Cc: "freebsd-hackers@freebsd.org" , Benjamin Kaduk X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Oct 2013 22:53:02 -0000 --=-BMfSLVO6WJOEvHDsj1vT Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable On Mon, 2013-10-21 at 11:29 +0100, Bruce Cran wrote: > On 10/21/2013 5:13 AM, Benjamin Kaduk wrote: > > > >> 37 > >> 38 /* Assume ISO C++ 'for' scoping rule. */ > >> 39 #define for if (0) ; else for > > > > StackOverflow (!) [1] suggests that they're a workaround for a bug in= =20 > > old versions of Visual Studio. >=20 > http://msdn.microsoft.com/en-us/library/b80153d8%28v=3Dvs.90%29.aspx also= =20 > documents it. Visual C++ 6 was released in 1998, which was=20 > unfortunately the same year as the first ISO C++ standard. >=20 Hrm, it looks like gperf is in ports ... should we consider just removing it from the base system in the first place? sean --=-BMfSLVO6WJOEvHDsj1vT Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQEcBAABAgAGBQJSZa+sAAoJEBkJRdwI6BaHEpwH/AjmftkdZZkYet9wXT6DYO3w KWnbQ86t14hjwsoNVzoKjbMOkbKKjYTdvogA1SXweCZtC39wY9hPDOqgR7bihkLZ TZFReNAMgw0lwUkqjTp2aGi8aF+3Z/CMjW2sWm1JXnN+ryGWDJ68g7YGYCQhUf9j zBTpvZfBXmkyGVLzFLPSku+8yVpp4swDvP9iSdAhE8LdPmPzu//FcCNLy0C+oNpr VEnGPebQitofv1/9AO79l/0W5JDmj4iyeVLGRLjyg2tM8Db7O+Q9qh2pET3MjbrC NUNUJzIzApxu03B0PFbJulFalAZXFUQPc0FBaPI03t7A6dh3SpXzMq9pOEW/3M4= =/1bW -----END PGP SIGNATURE----- --=-BMfSLVO6WJOEvHDsj1vT-- From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 22 06:37:02 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 89AC79F1; Tue, 22 Oct 2013 06:37:02 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mail-we0-x22e.google.com (mail-we0-x22e.google.com [IPv6:2a00:1450:400c:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F2E0E21F4; Tue, 22 Oct 2013 06:37:01 +0000 (UTC) Received: by mail-we0-f174.google.com with SMTP id u56so7550124wes.19 for ; Mon, 21 Oct 2013 23:37:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=voHFBnfvQO2s3PGVh1hzSxJ55KHvP+J7tZJ192Zcmp8=; b=E/t6+9f+URSAqa35XtspZHPl2SHAS22R8+2zYSxpnPQsVk4mcu1TKciGJpJT1xp5SR qXlM5Gxy63Qjj0B+2syqz+YpyP3/n9aEL4bnFErmoznc2XVF3Dnwyjt+VqJ1INs5IFyv e8udE0hNckNwBa2q7+Vcq0rjBWJVrNBKJj47Oor6hrm+fcNoQ1GA8i0VzO2sIjhq6MkX pfOnvdhW2P5BHZzI+mGSRSw+6EsuXmcF11Pp5KptidUNw5GkbbTVHvlBbv8TDQB5UL3U WhJBPDmgBmx6B/RyI2iOS+zoFLNl4m4R9am/9qtcCDYeZAMdlFhZ6ASfQtt4PmkfVagL 86nw== X-Received: by 10.194.109.68 with SMTP id hq4mr17047897wjb.12.1382423819917; Mon, 21 Oct 2013 23:36:59 -0700 (PDT) Received: from ithaqua.etoilebsd.net (ithaqua.etoilebsd.net. [37.59.37.188]) by mx.google.com with ESMTPSA id c10sm3000961wie.11.2013.10.21.23.36.58 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 21 Oct 2013 23:36:59 -0700 (PDT) Sender: Baptiste Daroussin Date: Tue, 22 Oct 2013 08:36:56 +0200 From: Baptiste Daroussin To: sbruno@freebsd.org Subject: Re: gperf -- #define for if (0) ; else for Message-ID: <20131022063655.GI20973@ithaqua.etoilebsd.net> References: <1382327705.2610.9.camel@localhost> <52650223.2020707@cran.org.uk> <1382395820.4447.1.camel@localhost> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="/GPgYEyhnw15BExa" Content-Disposition: inline In-Reply-To: <1382395820.4447.1.camel@localhost> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Bruce Cran , "freebsd-hackers@freebsd.org" , Benjamin Kaduk X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Oct 2013 06:37:02 -0000 --/GPgYEyhnw15BExa Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Oct 21, 2013 at 06:50:20PM -0400, Sean Bruno wrote: > On Mon, 2013-10-21 at 11:29 +0100, Bruce Cran wrote: > > On 10/21/2013 5:13 AM, Benjamin Kaduk wrote: > > > > > >> 37 > > >> 38 /* Assume ISO C++ 'for' scoping rule. */ > > >> 39 #define for if (0) ; else for > > > > > > StackOverflow (!) [1] suggests that they're a workaround for a bug in= =20 > > > old versions of Visual Studio. > >=20 > > http://msdn.microsoft.com/en-us/library/b80153d8%28v=3Dvs.90%29.aspx al= so=20 > > documents it. Visual C++ 6 was released in 1998, which was=20 > > unfortunately the same year as the first ISO C++ standard. > >=20 >=20 >=20 > Hrm, it looks like gperf is in ports ... should we consider just > removing it from the base system in the first place? >=20 Yes that is the plan :), remember that gperf is a dependency for gcc 4.2, s= o as long as we have gcc 4.2 we need gperf in base, we can probably turn it off depending on the gcc option (but that needs some work on the ports tree.) If you are volunteering at turning of gperf building, please prepare a patch against head, create a PR with [exp-run] in the subject and as portmgr for a exp, best would also be working with someone so that an infrastructure to d= epend on gperf is created in the ports tree (aka a USES=3Dgperf). regards, Bapt --/GPgYEyhnw15BExa Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlJmHQcACgkQ8kTtMUmk6EwtBACfSYg32etX1alYBlx0HZjsxu+k gjAAoJ2I5RmD2FIKizuhe6+ReZ2kEkq+ =/AeR -----END PGP SIGNATURE----- --/GPgYEyhnw15BExa-- From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 22 11:42:47 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 375DCBAD for ; Tue, 22 Oct 2013 11:42:47 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B959D23DB for ; Tue, 22 Oct 2013 11:42:46 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id eh20so1639149lab.22 for ; Tue, 22 Oct 2013 04:42:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Qf41qs3gbrmTvKefGiuwOzd5uDPKq66V9TmZCQJ97j4=; b=yPNDSgw1iA/n/ozS9favJvbQgmjKt8nVX2Dh5Guwzi3q/TyfY+gyyfzlq1xjEXClam LO87rr4D6agsoUCUPPRRml3aYhwAYBdTPSXbWKdqe5ONr5X1288KZn+O+VHPMLW08tzY uGay6ew9W+8J2p6PRkQAn2J53+zi2zzAujOBkexgCiHkinACM1ImoR8haFJl/gz4FYW0 +JHE5T7tTr+5E6iv4Gk03X6+f2Fqp/zzhQIQIQJpN9UNB00DU8Tp3IoDCTuycXyn5TQ9 0MeVXJwC2xO4ham5uteOicvLvM9bNCJxD6spqH0QYqLph/YXu8o4oYGQt+y9kjgTDlNf FKng== MIME-Version: 1.0 X-Received: by 10.152.120.5 with SMTP id ky5mr17859890lab.18.1382442164199; Tue, 22 Oct 2013 04:42:44 -0700 (PDT) Received: by 10.112.165.69 with HTTP; Tue, 22 Oct 2013 04:42:44 -0700 (PDT) In-Reply-To: <20131021.133036.045.1@DOMY-PC> References: <20130719.174511.786.3@DOMY-PC> <201310071212.05281.jhb@freebsd.org> <20131016.104912.479.1@DOMY-PC> <201310161650.52354.jhb@freebsd.org> <20131021.133036.045.1@DOMY-PC> Date: Tue, 22 Oct 2013 12:42:44 +0100 Message-ID: Subject: Re: UFS related panic (daily <-> find) From: Tom Evans To: rank1seeker@gmail.com Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Oct 2013 11:42:47 -0000 On Mon, Oct 21, 2013 at 2:30 PM, wrote: > If I can't use memtest86+-4.20, to determine failing module, then what is a > use of it at all? > Test RAM speed perhaps? You can't use it to prove that a stick is not faulty, but you can use it to prove that it is. 5 hours is perhaps not long enough to exercise the issue, or perhaps there is a different, longer, test suite you can choose. Cheers Tom From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 22 14:28:17 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B08AA1FC for ; Tue, 22 Oct 2013 14:28:17 +0000 (UTC) (envelope-from claudiu.vasadi@gmail.com) Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 80D982ECC for ; Tue, 22 Oct 2013 14:28:17 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id at1so1883756iec.29 for ; Tue, 22 Oct 2013 07:28:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=jR+Juet+dy6kHvaclR5I7rhIGIf6QF1o+u1vdMHEKMY=; b=TV2SuKTpipT2XEDPGwI0TwpBrdysm87oIuiLBrDX6S8Ca17H96d5a4BF23Xx1D8uIP gxRPQe1ZFRjcAYGCcUEz3DMziovs1vo4FqAuyE3Q21vGg7m3JRuwqgXN3dVhXMftGbM+ 85k80IE6DdNtl4ATgGnkFvqkKQETgGQUfeURb7PwQU9u/vIoZTwc9AtzRZasOLCJwaYh M3stxMb6dEZO5q6YJBEtzxcsArNhmcogBW2g/q45IrVYIT7aIvrtnWmIh4CTV7HDR/Jm KrGwm8ykhPZIiUUCc/uj4mdPz8ripHVPwgLaQxIp4dfkG0dAVQsYt9ILovkVcMh+Fk2S DsKA== MIME-Version: 1.0 X-Received: by 10.50.122.40 with SMTP id lp8mr13848822igb.24.1382452096719; Tue, 22 Oct 2013 07:28:16 -0700 (PDT) Received: by 10.64.134.169 with HTTP; Tue, 22 Oct 2013 07:28:16 -0700 (PDT) Date: Tue, 22 Oct 2013 16:28:16 +0200 Message-ID: Subject: 9.2-STABLE r255918 with GENERIC and iwn - core dump From: claudiu vasadi To: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Oct 2013 14:28:17 -0000 Hi everyone, I have a Lenovo Thinkpad T420s with Intel core i7 @ 2.70GHz, 8GB RAM, Intel SSD 160GB and iwn0: mem 0xf4200000-0xf4201fff irq 17 at device 0.0 on pci3 Today, while connecting to different AP's, I noticed at one point that I was not getting an IP although the wifi card was associated. Within "wifimgr", I did a "Save and Reconnect" and then got a core dump. Bellow, the bt: GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0xffffff801e5f7000 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80a10431 stack pointer = 0x28:0xffffff8000276980 frame pointer = 0x28:0xffffff8000276a20 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (swi4: clock) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: #0 0xffffffff80948a06 at kdb_backtrace+0x66 #1 0xffffffff8090e50e at panic+0x1ce #2 0xffffffff80cf3440 at trap_fatal+0x290 #3 0xffffffff80cf37a1 at trap_pfault+0x211 #4 0xffffffff80cf3d54 at trap+0x344 #5 0xffffffff80cdd093 at calltrap+0x8 #6 0xffffffff808dfddd at intr_event_execute_handlers+0xfd #7 0xffffffff808e15cd at ithread_loop+0x9d #8 0xffffffff808dc82f at fork_exit+0x11f #9 0xffffffff80cdd5be at fork_trampoline+0xe Uptime: 8h20m28s Dumping 952 out of 8106 MB:..2% (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) ..11% (CTRL-C to abort) (CTRL-C to abort) ..21%..31% (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) ..41% (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) ..51% (CTRL-C to abort) (CTRL-C to abort) ..61% (CTRL-C to abort) ..71% (CTRL-C to abort) ..81%..91% Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /boot-mount/boot/kernel/zfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /boot-mount/boot/kernel/opensolaris.ko.symbols...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from /boot-mount/boot/kernel/geom_eli.ko.symbols...done. done. Loaded symbols for /boot/kernel/geom_eli.ko Reading symbols from /boot/kernel/crypto.ko...Reading symbols from /boot-mount/boot/kernel/crypto.ko.symbols...done. done. Loaded symbols for /boot/kernel/crypto.ko Reading symbols from /boot/kernel/linux.ko...Reading symbols from /boot-mount/boot/kernel/linux.ko.symbols...done. done. Loaded symbols for /boot/kernel/linux.ko Reading symbols from /boot/kernel/drm.ko...Reading symbols from /boot-mount/boot/kernel/drm.ko.symbols...done. done. Loaded symbols for /boot/kernel/drm.ko Reading symbols from /boot/modules/nvidia.ko...done. Loaded symbols for /boot/modules/nvidia.ko Reading symbols from /boot/kernel/mmc.ko...Reading symbols from /boot-mount/boot/kernel/mmc.ko.symbols...done. done. Loaded symbols for /boot/kernel/mmc.ko Reading symbols from /boot/kernel/mmcsd.ko...Reading symbols from /boot-mount/boot/kernel/mmcsd.ko.symbols...done. done. Loaded symbols for /boot/kernel/mmcsd.ko Reading symbols from /boot/kernel/acpi_call.ko...done. Loaded symbols for /boot/kernel/acpi_call.ko Reading symbols from /boot/kernel/umodem.ko...Reading symbols from /boot-mount/boot/kernel/umodem.ko.symbols...done. done. Loaded symbols for /boot/kernel/umodem.ko Reading symbols from /boot/modules/vboxnetflt.ko...done. Loaded symbols for /boot/modules/vboxnetflt.ko Reading symbols from /boot/modules/vboxdrv.ko...done. Loaded symbols for /boot/modules/vboxdrv.ko Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from /boot-mount/boot/kernel/netgraph.ko.symbols...done. done. Loaded symbols for /boot/kernel/netgraph.ko Reading symbols from /boot/kernel/ng_ether.ko...Reading symbols from /boot-mount/boot/kernel/ng_ether.ko.symbols...done. done. Loaded symbols for /boot/kernel/ng_ether.ko Reading symbols from /boot/modules/vboxnetadp.ko...done. Loaded symbols for /boot/modules/vboxnetadp.ko #0 doadump (textdump=) at pcpu.h:234 234 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump (textdump=) at pcpu.h:234 #1 0xffffffff8090dfe6 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:449 #2 0xffffffff8090e4e7 in panic (fmt=0x1
) at /usr/src/sys/kern/kern_shutdown.c:637 #3 0xffffffff80cf3440 in trap_fatal (frame=0xc, eva=) at /usr/src/sys/amd64/amd64/trap.c:879 #4 0xffffffff80cf37a1 in trap_pfault (frame=0xffffff80002768d0, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:795 #5 0xffffffff80cf3d54 in trap (frame=0xffffff80002768d0) at /usr/src/sys/amd64/amd64/trap.c:463 #6 0xffffffff80cdd093 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:232 #7 0xffffffff80a10431 in ieee80211_tx_mgt_timeout (arg=0xffffff801e5f7000) at /usr/src/sys/net80211/ieee80211_output.c:2487 #8 0xffffffff809246e8 in softclock (arg=) at /usr/src/sys/kern/kern_timeout.c:518 #9 0xffffffff808dfddd in intr_event_execute_handlers (p=, ie=0xfffffe0007221b00) at /usr/src/sys/kern/kern_intr.c:1272 #10 0xffffffff808e15cd in ithread_loop (arg=0xfffffe0007209460) at /usr/src/sys/kern/kern_intr.c:1285 #11 0xffffffff808dc82f in fork_exit (callout=0xffffffff808e1530 , arg=0xfffffe0007209460, frame=0xffffff8000276b00) at /usr/src/sys/kern/kern_fork.c:990 #12 0xffffffff80cdd5be in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:606 #13 0x0000000000000000 in ?? () One thing to keep in mind is that since I started using geli+ZFS (installed with PC-BSD 9.1 cd), I always got "Cannot reset interface wlan0 - exit status 1" with "wifimgr" whichever action i did (ex: reconect, rescan, up/down,etc). I would appreciate some help in debugging this. -- Best regards, Claudiu Vasadi From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 22 14:20:09 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D19F111C; Tue, 22 Oct 2013 14:20:09 +0000 (UTC) (envelope-from electreg@list.ru) Received: from fallback5.mail.ru (fallback5.mail.ru [94.100.176.59]) by mx1.freebsd.org (Postfix) with ESMTP id 843812E4E; Tue, 22 Oct 2013 14:20:09 +0000 (UTC) Received: from f87.i.mail.ru (f87.i.mail.ru [128.140.169.159]) by fallback5.mail.ru (mPOP.Fallback_MX) with ESMTP id 05B57FC69E59; Tue, 22 Oct 2013 18:19:52 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=list.ru; s=mail; h=References:In-Reply-To:Content-Type:Message-ID:Reply-To:Date:Mime-Version:Subject:Cc:To:From; bh=qwLrgvUOZPZUTH5xv1WszyR1L5CUf56Wy9f82FEaGkE=; b=SDLNdrPDo1aoFgTy9CMNh5vIT5PfuJtdyZNAZgCj8PoDkC6huUX6xDVTrvKLpIeb6BEyqxpBd+6el2u5EdYc4iegjFAyzuTSWVCyPHRjEnaSOtKUvFzJHoBH+pZX/fEANCVDtbe+4zocCPO1aQWamperZ/396nmVA2Ouo//AMnc=; Received: from mail by f87.i.mail.ru with local (envelope-from ) id 1VYcoC-0005Lj-6E; Tue, 22 Oct 2013 18:19:44 +0400 Received: from [88.205.176.131] by e.mail.ru with HTTP; Tue, 22 Oct 2013 18:19:44 +0400 From: =?UTF-8?B?QWxleGV5IEVnb3Jvdg==?= To: =?UTF-8?B?SWFuIExlcG9yZQ==?= Subject: =?UTF-8?B?UmU6IGRldGVybWluZSBkcml2ZSdzIFNBUyBwb3J0?= Mime-Version: 1.0 X-Mailer: Mail.Ru Mailer 1.0 X-Originating-IP: [88.205.176.131] Date: Tue, 22 Oct 2013 18:19:44 +0400 X-Priority: 3 (Normal) Message-ID: <1382451584.885779403@f87.i.mail.ru> X-Mras: Ok X-Spam: undefined In-Reply-To: <1382286899.92499.110.camel@revolution.hippie.lan> References: <1382192148.119437035@f257.i.mail.ru> <1382286899.92499.110.camel@revolution.hippie.lan> X-Mailman-Approved-At: Tue, 22 Oct 2013 15:18:50 +0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: =?UTF-8?B?QWxleGV5IEVnb3Jvdg==?= List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Oct 2013 14:20:09 -0000 ID7CoFdvdWxkIHRoZSBjYW1jb250cm9sIGJ1cy90YXJnZXQvbHVuIG91dHB1dCBnaXZlIHlvdSB0 aGUgbnVtYmVyIHlvdSdyZWxvb2tpbmcgZm9yPwoKVGhhbmtzISAidGFyZ2V0IiBpbiBjYW1jb250 cm9sIG91dHB1dCBpcyBleGFjdGx5IHdoYXQgSSdtIGxvb2tpbmcgZm9yLsKg From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 22 15:41:03 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 05294CDB for ; Tue, 22 Oct 2013 15:41:03 +0000 (UTC) (envelope-from leonerd@leonerd.org.uk) Received: from cel.leonerd.org.uk (cel.leonerd.org.uk [IPv6:2001:8b0:3f7::2]) by mx1.freebsd.org (Postfix) with ESMTP id 8A79F235B for ; Tue, 22 Oct 2013 15:41:02 +0000 (UTC) Received: from shy.leonerd.org.uk (8.9.7.8.3.c.e.f.f.f.2.8.9.a.e.8.3.4.0.0.7.f.3.0.0.b.8.0.1.0.0.2.ip6.arpa [IPv6:2001:8b0:3f7:43:8ea9:82ff:fec3:8798]) by cel.leonerd.org.uk (Postfix) with ESMTPSA id B52A71BDA9 for ; Tue, 22 Oct 2013 16:41:00 +0100 (BST) Date: Tue, 22 Oct 2013 16:41:00 +0100 From: Paul "LeoNerd" Evans To: freebsd-hackers@FreeBSD.org Subject: BPF "loop" instruction to help IPv6 filtering Message-ID: <20131022164100.001122f6@shy.leonerd.org.uk> X-Mailer: Claws Mail 3.9.2 (GTK+ 2.24.21; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA256; boundary="Sig_/1v3qyOWaN0a0N=hGLHIl2Pt"; protocol="application/pgp-signature" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Oct 2013 15:41:03 -0000 --Sig_/1v3qyOWaN0a0N=hGLHIl2Pt Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hi all, [I tried this suggestion at Linux a while ago, but was met with complete silence. Trying my luck here instead hoping I can nudge support from here first] BPF deliberately avoids being fully Turing-powerful in order to guarantee that its filter programs definitely terminate, to ensure timely handling of packet filtering in the kernel. This is good. However, I find that trying to filter IPv6 packets means that the program has to take a number of non-ideal steps that make it less useful to IPv6. In contrast to IPv4 packets, IPv6 packets do not have a fixed header with an area for protocol-level options, followed eventually by the "upper level" protocol like TCP or UDP at an offset easily calculable in fixed space. Instead, an IPv6 packet contains a "linked list" structure, where the overall IPv6 header gives the protocol number of the "next" protocol on the packet. That will either be TCP/UDP/similar, or it will be some sort of wrapper header, with its own "next" field. It is further complicated by each header having its own format, there not being a standard "next" place to look without understanding the specific protocol. All very messy. Anyhow, the current way BPF-using programs have to filter for IPv6 is to unroll the implied "while" loop this would create a fixed number of times (e.g. tcpdump usually does 5), and expands the entire skip program in each unrolling, leading to quite a large amount of code. I would like to propose a new BPF instruction, "LOOP", which can apply a limited kind of while() loop to the BPF program, making it easier to filter on IPv6 packets. It would operate in conjunction with the 'X' register, in the following way: LOOP A, label -- add the accumulator to 'X', and jump backwards to the given instruction if the new value of X is still shorter than the overall packet length. LOOP k, label -- add the immediate constant k to X, and jump backwards as before. Statically, the program can be checked to ensure that a constant 'k' is non-zero, and that no instruction alters the value of 'X' between the LOOP and its target label -- if these checks fail then the program is rejected. Dynamically while running, if the LOOP A instruction is run with the accumulator zero, then the program should "RET 0" (similar to a divide by zero). I believe these semantics ensure that the program is still guaranteed to terminate eventually, as each LOOP instruction must make a non-zero progress to the index, X; and furthermore it cannot proceed further than the length of the packet. How this would be used in practice would involve creating a switch-type block, inspecting the "next protocol" field of the current IPv6 header, being at the X index. If the type is the required one (e.g. TCP), then the packet can be accepted (or further tested). If not, then the switch-type block can increment X according to the size of the header it is currently looking at, ensuring it now points at the next header type. This allows an efficient IPv6-parsing program, because it doesn't have to be loop-unrolled a static number of times, either consuming too much program space, or lacking the ability to handle deeply-nested constructs.=20 Does this sound sane? --=20 Paul "LeoNerd" Evans leonerd@leonerd.org.uk ICQ# 4135350 | Registered Linux# 179460 http://www.leonerd.org.uk/ --Sig_/1v3qyOWaN0a0N=hGLHIl2Pt Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (GNU/Linux) iF4EAREIAAYFAlJmnIwACgkQ4y9efBOfZ0ihSgD/W2WqvPXP/nBpiyMxPtBFfk/e nUFRb1SXmJIZ9UtLqR0A/Rk4aDDM9riwxfdHDrQtFeyv5H5s3/0Ko+Q9RWwRL+H4 =9/SL -----END PGP SIGNATURE----- --Sig_/1v3qyOWaN0a0N=hGLHIl2Pt-- From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 22 16:07:58 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 76718E0E; Tue, 22 Oct 2013 16:07:58 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-qe0-x22a.google.com (mail-qe0-x22a.google.com [IPv6:2607:f8b0:400d:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2381E24EA; Tue, 22 Oct 2013 16:07:58 +0000 (UTC) Received: by mail-qe0-f42.google.com with SMTP id gc15so4948904qeb.15 for ; Tue, 22 Oct 2013 09:07:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=TZHOiAE8hKZ6ZUK7K6mKLCodxLzm/YBOJu8ZiPZ+REM=; b=cpPPyU0CxWOx+NbkzCmgnMHC6lvIFqg69NdVOs9VxsZxvJ9eZVrD+XFQG05KXtSCII VD3c15osOXxJSifaa3Gi9RklKnijHvLfhyCD8l3/MnAnTQ0c3T6r+6EYXDhX/NX3Oc92 ogcYukQs547mQ1hImmWIMCqtk2eaJl0weLwgh2Fdb+uGRVWYTVpSQHFfQtQBTt7s4Voa GNFq2PclZqtjSbwAPb+QnkX8jwzy68uuT1DPyniPV57qiY+x4xgkTErXtGM3oCgQRVaV k18zcUBlfeqJ++Wgpo76tScAWQ9BGqHjK2hiUhihvln62uIvdjSoL2G1NxlF8aMbgS3g CPlQ== MIME-Version: 1.0 X-Received: by 10.49.127.179 with SMTP id nh19mr29993888qeb.1.1382458077212; Tue, 22 Oct 2013 09:07:57 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.207.66 with HTTP; Tue, 22 Oct 2013 09:07:57 -0700 (PDT) In-Reply-To: References: Date: Tue, 22 Oct 2013 09:07:57 -0700 X-Google-Sender-Auth: zSqfefMoreg7Ukj_65a3ocClEv4 Message-ID: Subject: Re: 9.2-STABLE r255918 with GENERIC and iwn - core dump From: Adrian Chadd To: claudiu vasadi , "freebsd-wireless@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Oct 2013 16:07:58 -0000 I know what's causing this! It's because when the management frame completes, there's a callback mbuf tag (M_TXCB) that causes the driver to call the net80211 TX completion callback. Now, because some drivers call the net80211 tx completion callback from within their driver locks, it causes locking issues. So, someone (I don't know or really care who) made it so whenever a TX completion occurs, the net80211 code will schedule a callout to occur. This means the callout occurs outside of the driver locks, solving that issue. This has a bunch of problems. * Firstly, if you have multiple management frames coming in, only the most recent will be acknowledged. Tsk. There's only one callout, and it's per vap. * Secondly, no node reference is taken before scheduling the callout, so if the node is destroyed (eg because the BSS is freed during a channel scan or reset) and the callout still occurs, it'll dereference a bad node. This is the crash cause. * Thirdly, the cancellation occurs in the VAP state change path. It doesn't know about the node(s) that just received TX completions. Since the callback is per vap, there's no way to figure out which node needs dereferencing.. so things blow up. The solution is just to undo this brain damaged solution and require that drivers call the TX completion callback with no driver locks held. That's on my TODO list but it'll take a little more time. Now that 10 has branched I'll be happy to just flip that switch in -HEAD and deal with the locking fallout. Thanks, -adrian On 22 October 2013 07:28, claudiu vasadi wrote: > Hi everyone, > > I have a Lenovo Thinkpad T420s with Intel core i7 @ 2.70GHz, 8GB RAM, Intel > SSD 160GB and iwn0: mem > 0xf4200000-0xf4201fff irq 17 at device 0.0 on pci3 > > Today, while connecting to different AP's, I noticed at one point that I > was not getting an IP although the wifi card was associated. Within > "wifimgr", I did a "Save and Reconnect" and then got a core dump. > > Bellow, the bt: > > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you > are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address = 0xffffff801e5f7000 > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff80a10431 > stack pointer = 0x28:0xffffff8000276980 > frame pointer = 0x28:0xffffff8000276a20 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 12 (swi4: clock) > trap number = 12 > panic: page fault > cpuid = 0 > KDB: stack backtrace: > #0 0xffffffff80948a06 at kdb_backtrace+0x66 > #1 0xffffffff8090e50e at panic+0x1ce > #2 0xffffffff80cf3440 at trap_fatal+0x290 > #3 0xffffffff80cf37a1 at trap_pfault+0x211 > #4 0xffffffff80cf3d54 at trap+0x344 > #5 0xffffffff80cdd093 at calltrap+0x8 > #6 0xffffffff808dfddd at intr_event_execute_handlers+0xfd > #7 0xffffffff808e15cd at ithread_loop+0x9d > #8 0xffffffff808dc82f at fork_exit+0x11f > #9 0xffffffff80cdd5be at fork_trampoline+0xe > Uptime: 8h20m28s > Dumping 952 out of 8106 MB:..2% (CTRL-C to abort) (CTRL-C to abort) > (CTRL-C to abort) (CTRL-C to abort) ..11% (CTRL-C to abort) (CTRL-C to > abort) ..21%..31% (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) > (CTRL-C to abort) (CTRL-C to abort) ..41% (CTRL-C to abort) (CTRL-C to > abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to > abort) (CTRL-C to abort) (CTRL-C to abort) ..51% (CTRL-C to abort) > (CTRL-C to abort) ..61% (CTRL-C to abort) ..71% (CTRL-C to abort) > ..81%..91% > > Reading symbols from /boot/kernel/zfs.ko...Reading symbols from > /boot-mount/boot/kernel/zfs.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/zfs.ko > Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from > /boot-mount/boot/kernel/opensolaris.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/opensolaris.ko > Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from > /boot-mount/boot/kernel/geom_eli.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/geom_eli.ko > Reading symbols from /boot/kernel/crypto.ko...Reading symbols from > /boot-mount/boot/kernel/crypto.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/crypto.ko > Reading symbols from /boot/kernel/linux.ko...Reading symbols from > /boot-mount/boot/kernel/linux.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/linux.ko > Reading symbols from /boot/kernel/drm.ko...Reading symbols from > /boot-mount/boot/kernel/drm.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/drm.ko > Reading symbols from /boot/modules/nvidia.ko...done. > Loaded symbols for /boot/modules/nvidia.ko > Reading symbols from /boot/kernel/mmc.ko...Reading symbols from > /boot-mount/boot/kernel/mmc.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/mmc.ko > Reading symbols from /boot/kernel/mmcsd.ko...Reading symbols from > /boot-mount/boot/kernel/mmcsd.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/mmcsd.ko > Reading symbols from /boot/kernel/acpi_call.ko...done. > Loaded symbols for /boot/kernel/acpi_call.ko > Reading symbols from /boot/kernel/umodem.ko...Reading symbols from > /boot-mount/boot/kernel/umodem.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/umodem.ko > Reading symbols from /boot/modules/vboxnetflt.ko...done. > Loaded symbols for /boot/modules/vboxnetflt.ko > Reading symbols from /boot/modules/vboxdrv.ko...done. > Loaded symbols for /boot/modules/vboxdrv.ko > Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from > /boot-mount/boot/kernel/netgraph.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/netgraph.ko > Reading symbols from /boot/kernel/ng_ether.ko...Reading symbols from > /boot-mount/boot/kernel/ng_ether.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/ng_ether.ko > Reading symbols from /boot/modules/vboxnetadp.ko...done. > Loaded symbols for /boot/modules/vboxnetadp.ko > #0 doadump (textdump=) at pcpu.h:234 > 234 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) bt > #0 doadump (textdump=) at pcpu.h:234 > #1 0xffffffff8090dfe6 in kern_reboot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:449 > #2 0xffffffff8090e4e7 in panic (fmt=0x1
) at > /usr/src/sys/kern/kern_shutdown.c:637 > #3 0xffffffff80cf3440 in trap_fatal (frame=0xc, eva=) > at /usr/src/sys/amd64/amd64/trap.c:879 > #4 0xffffffff80cf37a1 in trap_pfault (frame=0xffffff80002768d0, > usermode=0) at /usr/src/sys/amd64/amd64/trap.c:795 > #5 0xffffffff80cf3d54 in trap (frame=0xffffff80002768d0) at > /usr/src/sys/amd64/amd64/trap.c:463 > #6 0xffffffff80cdd093 in calltrap () at > /usr/src/sys/amd64/amd64/exception.S:232 > #7 0xffffffff80a10431 in ieee80211_tx_mgt_timeout (arg=0xffffff801e5f7000) > at /usr/src/sys/net80211/ieee80211_output.c:2487 > #8 0xffffffff809246e8 in softclock (arg=) at > /usr/src/sys/kern/kern_timeout.c:518 > #9 0xffffffff808dfddd in intr_event_execute_handlers (p= out>, ie=0xfffffe0007221b00) > at /usr/src/sys/kern/kern_intr.c:1272 > #10 0xffffffff808e15cd in ithread_loop (arg=0xfffffe0007209460) at > /usr/src/sys/kern/kern_intr.c:1285 > #11 0xffffffff808dc82f in fork_exit (callout=0xffffffff808e1530 > , arg=0xfffffe0007209460, > frame=0xffffff8000276b00) at /usr/src/sys/kern/kern_fork.c:990 > #12 0xffffffff80cdd5be in fork_trampoline () at > /usr/src/sys/amd64/amd64/exception.S:606 > #13 0x0000000000000000 in ?? () > > > One thing to keep in mind is that since I started using geli+ZFS (installed > with PC-BSD 9.1 cd), I always got "Cannot reset interface wlan0 - exit > status 1" with "wifimgr" whichever action i did (ex: reconect, rescan, > up/down,etc). > > > I would appreciate some help in debugging this. > > > -- > Best regards, > Claudiu Vasadi > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 22 16:56:52 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0BF36580; Tue, 22 Oct 2013 16:56:52 +0000 (UTC) (envelope-from claudiu.vasadi@gmail.com) Received: from mail-ie0-x22c.google.com (mail-ie0-x22c.google.com [IPv6:2607:f8b0:4001:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BB5322803; Tue, 22 Oct 2013 16:56:51 +0000 (UTC) Received: by mail-ie0-f172.google.com with SMTP id tp5so2232905ieb.31 for ; Tue, 22 Oct 2013 09:56:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=W3EpCpG3w5RS61hCsJSAmmDzbAnmhMWAQ13xkpyqgYM=; b=ZZKxTqC+sTSLxVrW5hFDjqtFfYGrD+m7BOxhHIr/b3hQnm4YFtx4iUp1EjGpotul35 gl2x3ElbGw29C0TLaWmDRnHcHlOtncYDRlH6e/MoFPcMAZz6hdj5jFPIQsDA7Hv+hVbz 4Ci9xEaT5taKHaJECosjKi9qAh+7uKlzqnT3DJKorWj593V6MebyKyrYHHG0DItcJM6M T0E/Ii6pKhd9m3UwEydJrE7tF2ztmnQ9eJy/7xdvwm9442Vn4kz3ARhgAcmXZHdzrVq3 aCKtrF36vXe1MXtMpNE3Upk2wW8d+TA/26BBjLsU1bE8XPuIeL4RSE2IDn/qoySthXFS VloA== MIME-Version: 1.0 X-Received: by 10.50.122.40 with SMTP id lp8mr14429326igb.24.1382461011070; Tue, 22 Oct 2013 09:56:51 -0700 (PDT) Received: by 10.64.134.169 with HTTP; Tue, 22 Oct 2013 09:56:51 -0700 (PDT) In-Reply-To: References: Date: Tue, 22 Oct 2013 18:56:51 +0200 Message-ID: Subject: Re: 9.2-STABLE r255918 with GENERIC and iwn - core dump From: claudiu vasadi To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-hackers@freebsd.org" , "freebsd-wireless@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Oct 2013 16:56:52 -0000 For when the time comes, I'm all in for any tests (if needed). On Tue, Oct 22, 2013 at 6:07 PM, Adrian Chadd wrote: > I know what's causing this! > > It's because when the management frame completes, there's a callback mbuf > tag (M_TXCB) that causes the driver to call the net80211 TX completion > callback. > > Now, because some drivers call the net80211 tx completion callback from > within their driver locks, it causes locking issues. So, someone (I don't > know or really care who) made it so whenever a TX completion occurs, the > net80211 code will schedule a callout to occur. This means the callout > occurs outside of the driver locks, solving that issue. > > This has a bunch of problems. > > * Firstly, if you have multiple management frames coming in, only the most > recent will be acknowledged. Tsk. There's only one callout, and it's per > vap. > * Secondly, no node reference is taken before scheduling the callout, so > if the node is destroyed (eg because the BSS is freed during a channel scan > or reset) and the callout still occurs, it'll dereference a bad node. This > is the crash cause. > * Thirdly, the cancellation occurs in the VAP state change path. It > doesn't know about the node(s) that just received TX completions. Since the > callback is per vap, there's no way to figure out which node needs > dereferencing.. so things blow up. > > The solution is just to undo this brain damaged solution and require that > drivers call the TX completion callback with no driver locks held. That's > on my TODO list but it'll take a little more time. Now that 10 has branched > I'll be happy to just flip that switch in -HEAD and deal with the locking > fallout. > > Thanks, > > > > -adrian > > > > On 22 October 2013 07:28, claudiu vasadi wrote: > >> Hi everyone, >> >> I have a Lenovo Thinkpad T420s with Intel core i7 @ 2.70GHz, 8GB RAM, >> Intel >> SSD 160GB and iwn0: mem >> 0xf4200000-0xf4201fff irq 17 at device 0.0 on pci3 >> >> Today, while connecting to different AP's, I noticed at one point that I >> was not getting an IP although the wifi card was associated. Within >> "wifimgr", I did a "Save and Reconnect" and then got a core dump. >> >> Bellow, the bt: >> >> >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and you >> are >> welcome to change it and/or distribute copies of it under certain >> conditions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for >> details. >> This GDB was configured as "amd64-marcel-freebsd"... >> >> Unread portion of the kernel message buffer: >> >> >> Fatal trap 12: page fault while in kernel mode >> cpuid = 0; apic id = 00 >> fault virtual address = 0xffffff801e5f7000 >> fault code = supervisor read data, page not present >> instruction pointer = 0x20:0xffffffff80a10431 >> stack pointer = 0x28:0xffffff8000276980 >> frame pointer = 0x28:0xffffff8000276a20 >> code segment = base 0x0, limit 0xfffff, type 0x1b >> = DPL 0, pres 1, long 1, def32 0, gran 1 >> processor eflags = interrupt enabled, resume, IOPL = 0 >> current process = 12 (swi4: clock) >> trap number = 12 >> panic: page fault >> cpuid = 0 >> KDB: stack backtrace: >> #0 0xffffffff80948a06 at kdb_backtrace+0x66 >> #1 0xffffffff8090e50e at panic+0x1ce >> #2 0xffffffff80cf3440 at trap_fatal+0x290 >> #3 0xffffffff80cf37a1 at trap_pfault+0x211 >> #4 0xffffffff80cf3d54 at trap+0x344 >> #5 0xffffffff80cdd093 at calltrap+0x8 >> #6 0xffffffff808dfddd at intr_event_execute_handlers+0xfd >> #7 0xffffffff808e15cd at ithread_loop+0x9d >> #8 0xffffffff808dc82f at fork_exit+0x11f >> #9 0xffffffff80cdd5be at fork_trampoline+0xe >> Uptime: 8h20m28s >> Dumping 952 out of 8106 MB:..2% (CTRL-C to abort) (CTRL-C to abort) >> (CTRL-C to abort) (CTRL-C to abort) ..11% (CTRL-C to abort) (CTRL-C to >> abort) ..21%..31% (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) >> (CTRL-C to abort) (CTRL-C to abort) ..41% (CTRL-C to abort) (CTRL-C to >> abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C >> to >> abort) (CTRL-C to abort) (CTRL-C to abort) ..51% (CTRL-C to abort) >> (CTRL-C to abort) ..61% (CTRL-C to abort) ..71% (CTRL-C to abort) >> ..81%..91% >> >> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from >> /boot-mount/boot/kernel/zfs.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/zfs.ko >> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from >> /boot-mount/boot/kernel/opensolaris.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/opensolaris.ko >> Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from >> /boot-mount/boot/kernel/geom_eli.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/geom_eli.ko >> Reading symbols from /boot/kernel/crypto.ko...Reading symbols from >> /boot-mount/boot/kernel/crypto.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/crypto.ko >> Reading symbols from /boot/kernel/linux.ko...Reading symbols from >> /boot-mount/boot/kernel/linux.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/linux.ko >> Reading symbols from /boot/kernel/drm.ko...Reading symbols from >> /boot-mount/boot/kernel/drm.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/drm.ko >> Reading symbols from /boot/modules/nvidia.ko...done. >> Loaded symbols for /boot/modules/nvidia.ko >> Reading symbols from /boot/kernel/mmc.ko...Reading symbols from >> /boot-mount/boot/kernel/mmc.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/mmc.ko >> Reading symbols from /boot/kernel/mmcsd.ko...Reading symbols from >> /boot-mount/boot/kernel/mmcsd.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/mmcsd.ko >> Reading symbols from /boot/kernel/acpi_call.ko...done. >> Loaded symbols for /boot/kernel/acpi_call.ko >> Reading symbols from /boot/kernel/umodem.ko...Reading symbols from >> /boot-mount/boot/kernel/umodem.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/umodem.ko >> Reading symbols from /boot/modules/vboxnetflt.ko...done. >> Loaded symbols for /boot/modules/vboxnetflt.ko >> Reading symbols from /boot/modules/vboxdrv.ko...done. >> Loaded symbols for /boot/modules/vboxdrv.ko >> Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from >> /boot-mount/boot/kernel/netgraph.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/netgraph.ko >> Reading symbols from /boot/kernel/ng_ether.ko...Reading symbols from >> /boot-mount/boot/kernel/ng_ether.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/ng_ether.ko >> Reading symbols from /boot/modules/vboxnetadp.ko...done. >> Loaded symbols for /boot/modules/vboxnetadp.ko >> #0 doadump (textdump=) at pcpu.h:234 >> 234 pcpu.h: No such file or directory. >> in pcpu.h >> (kgdb) bt >> #0 doadump (textdump=) at pcpu.h:234 >> #1 0xffffffff8090dfe6 in kern_reboot (howto=260) at >> /usr/src/sys/kern/kern_shutdown.c:449 >> #2 0xffffffff8090e4e7 in panic (fmt=0x1
) at >> /usr/src/sys/kern/kern_shutdown.c:637 >> #3 0xffffffff80cf3440 in trap_fatal (frame=0xc, eva=> out>) >> at /usr/src/sys/amd64/amd64/trap.c:879 >> #4 0xffffffff80cf37a1 in trap_pfault (frame=0xffffff80002768d0, >> usermode=0) at /usr/src/sys/amd64/amd64/trap.c:795 >> #5 0xffffffff80cf3d54 in trap (frame=0xffffff80002768d0) at >> /usr/src/sys/amd64/amd64/trap.c:463 >> #6 0xffffffff80cdd093 in calltrap () at >> /usr/src/sys/amd64/amd64/exception.S:232 >> #7 0xffffffff80a10431 in ieee80211_tx_mgt_timeout >> (arg=0xffffff801e5f7000) >> at /usr/src/sys/net80211/ieee80211_output.c:2487 >> #8 0xffffffff809246e8 in softclock (arg=) at >> /usr/src/sys/kern/kern_timeout.c:518 >> #9 0xffffffff808dfddd in intr_event_execute_handlers (p=> out>, ie=0xfffffe0007221b00) >> at /usr/src/sys/kern/kern_intr.c:1272 >> #10 0xffffffff808e15cd in ithread_loop (arg=0xfffffe0007209460) at >> /usr/src/sys/kern/kern_intr.c:1285 >> #11 0xffffffff808dc82f in fork_exit (callout=0xffffffff808e1530 >> , arg=0xfffffe0007209460, >> frame=0xffffff8000276b00) at /usr/src/sys/kern/kern_fork.c:990 >> #12 0xffffffff80cdd5be in fork_trampoline () at >> /usr/src/sys/amd64/amd64/exception.S:606 >> #13 0x0000000000000000 in ?? () >> >> >> One thing to keep in mind is that since I started using geli+ZFS >> (installed >> with PC-BSD 9.1 cd), I always got "Cannot reset interface wlan0 - exit >> status 1" with "wifimgr" whichever action i did (ex: reconect, rescan, >> up/down,etc). >> >> >> I would appreciate some help in debugging this. >> >> >> -- >> Best regards, >> Claudiu Vasadi >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org >> " >> > > -- Best regards, Claudiu Vasadi From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 22 20:28:13 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3BEAB31A; Tue, 22 Oct 2013 20:28:13 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 14E2E25D1; Tue, 22 Oct 2013 20:28:13 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 13DAAB98D; Tue, 22 Oct 2013 16:28:12 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Subject: Re: Is freebsd-update braindamaged, or I'm using it wrong? Date: Tue, 22 Oct 2013 15:50:10 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201310221550.10291.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 22 Oct 2013 16:28:12 -0400 (EDT) Cc: Alexander Yerenkow , Ivan Voras X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Oct 2013 20:28:13 -0000 On Tuesday, October 01, 2013 5:38:15 am Alexander Yerenkow wrote: > To make better tool (than current behaviour of mergemaster regarding > configs/source files) which could make merge an easy task it *must* have > such things: > > a) way to get Original configs/files from revision from you are upgrading > ($Rev1) > b) way to get Original configs/files from revision to you are upgrading > ($Rev100) > c) have ability to ignore differencies in comments > d) have ability to treat special cases (as $FreeBSD$ - just took newer line) > > Then, your each new file will be $Rev100 + diff_changes(CURRENT, $Rev1) + > diff_changes($Rev100, $Rev1). > Note, that in case that your diffs are none diff_changes(CURRENT, $Rev1) = > 0, then you can simply get new file. > Same thing in case that only $FreeBSD$ changed. > > I have some PoC-es for this, but not in shell, maybe I'll come up someday > with full tool. Try etcupdate. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 23 00:15:18 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 136E2E84 for ; Wed, 23 Oct 2013 00:15:18 +0000 (UTC) (envelope-from schmiedgen@gmx.net) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A4196231D for ; Wed, 23 Oct 2013 00:15:17 +0000 (UTC) Received: from pepe.smoke.local ([88.74.235.131]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0MPlY2-1Vcbqr2jcw-0053pc for ; Wed, 23 Oct 2013 02:15:15 +0200 Message-ID: <52671512.7000406@gmx.net> Date: Wed, 23 Oct 2013 02:15:14 +0200 From: Michael Schmiedgen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: Is freebsd-update braindamaged, or I'm using it wrong? References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:I8bQjNUYqEPOxkLE/WZ5FA8GwE+kgUJImgrJWR8x7frzpmmL62o GFiQFn2OiSw9WmZLkJytbOHaOERUjFakkFodElIzPtZ8BkEtvcjHoA+o4QZI5rjB7iZcGZa KutumjCRtNOTAGkiuO8JqyyW9WuGt8UaPkCVqkXGwzIwFT+l4e0ofHF2oLEK/1i8rLCkfRi exJSD3DFhbR7p3ES6hVyA== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Oct 2013 00:15:18 -0000 On 10/01/13 11:16, Ivan Voras wrote: > This is the first time I've used freebsd-update in years, and I'm > immediately flagging it as something I won't use in the future. For > the last half hour it has been forcing me to manually resolve, one by > one, in an editor, hundreds of "merge conflicts" such as these: > > 1 <<<<<<< current version > 2 # $FreeBSD: release/9.0.0/etc/gettytab 209954 2010-07-12 19:09:18Z bcr $ > 3 ======= > 4 # $FreeBSD: release/9.2.0/etc/gettytab 243623 2012-11-27 19:23:54Z peterj $ > 5 >>>>>>> 9.2-RELEASE > 6 # from: @(#)gettytab 5.14 (Berkeley) 3/27/91 > 7 # Another thing was that I had to invoke pwd_mkdb manually after password file was merged. I wondered, because behaviour in mergemaster is to recognize this. Michael From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 23 01:17:11 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id ABCA14E0 for ; Wed, 23 Oct 2013 01:17:11 +0000 (UTC) (envelope-from erdgeist@erdgeist.org) Received: from elektropost.org (elektropost.org [217.115.13.199]) by mx1.freebsd.org (Postfix) with ESMTP id F1F99267D for ; Wed, 23 Oct 2013 01:17:10 +0000 (UTC) Received: (qmail 21523 invoked from network); 23 Oct 2013 01:17:07 -0000 Received: from elektropost.org (HELO elektropost.org) (erdgeist@erdgeist.org) by elektropost.org with CAMELLIA256-SHA encrypted SMTP; 23 Oct 2013 01:17:07 -0000 Message-ID: <52672393.4080109@erdgeist.org> Date: Wed, 23 Oct 2013 03:17:07 +0200 From: Dirk Engling User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: Is freebsd-update braindamaged, or I'm using it wrong? References: <52671512.7000406@gmx.net> In-Reply-To: <52671512.7000406@gmx.net> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Oct 2013 01:17:11 -0000 On 23.10.13 02:15, Michael Schmiedgen wrote: > Another thing was that I had to invoke pwd_mkdb manually after password > file was merged. I wondered, because behaviour in mergemaster is to > recognize this. And while we're at it: man services says to call services_mkdb after touching /etc/services. However, running it gives me hundreds of errors about "services_mkdb: duplicate service `1'" So is the man page wrong? When is the last time the mkdb command has been run on the shipped /etc/services file? Regards, erdgeist From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 23 01:18:27 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3417D5CD; Wed, 23 Oct 2013 01:18:27 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-qa0-x22d.google.com (mail-qa0-x22d.google.com [IPv6:2607:f8b0:400d:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D5591268D; Wed, 23 Oct 2013 01:18:26 +0000 (UTC) Received: by mail-qa0-f45.google.com with SMTP id ii20so3849549qab.18 for ; Tue, 22 Oct 2013 18:18:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=EAaaBS6C0UFYdUKrtFZ0lb+L9Jk1MzO8AMEY3FHr+x8=; b=CCuGf7cWcWyjkmtWWXjca/aX1/fdHgr5HSm/46SW8cYMwJ5Xbk/6CnuXhCSVbvyD11 9dL7T9MyRrpnWGcXNvsGyOu4F/Jf7c/0httrEDPaN2Dk+yOFdpNY/pjBGljF4LSDYyaU l9yr6axv5rZ0ScerJPzzmgdlxGQ9Rt/xGcmPwqpSplM4PXlIcA4hcObwIcxG0Suvoy9t eb5Jq+hvFiZ5MddRNgMFJwh+uBgN94fykhHki7fMwgp/2+OXaNBb2/QD2OtCbFQNGZd1 qJhG9YLmFsFf9vJy4RT8e3dMCERktGYKkH1FcpUplPwjV+aIpzAFlWQKS6rRkArIy+M9 eXFg== MIME-Version: 1.0 X-Received: by 10.224.36.201 with SMTP id u9mr757200qad.76.1382491106050; Tue, 22 Oct 2013 18:18:26 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.207.66 with HTTP; Tue, 22 Oct 2013 18:18:25 -0700 (PDT) In-Reply-To: References: Date: Tue, 22 Oct 2013 18:18:25 -0700 X-Google-Sender-Auth: DvGJb_WlJ7MkU3exnbX1O1DWZxM Message-ID: Subject: Re: 9.2-STABLE r255918 with GENERIC and iwn - core dump From: Adrian Chadd To: claudiu vasadi Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-hackers@freebsd.org" , "freebsd-wireless@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Oct 2013 01:18:27 -0000 Grr, it's slightly more complicated than that. That whole timer mess is actually used for two things: * if the management transmit succeeds - it acts as a short-interval (a few seconds) timer to ensure that the probe request ends up providing some response that transitions to auth; otherwise it aborts and transitions back to the SCAN state; * if the management transmit fails - it immediately transitions back to SCAN. So, the "correct" fix involves correctly locking things and turning that timer into a "if I fail to transition into a run state, then just go back to scanning" timeout. It sucks and it's going to take an evening of deep thought to figure out the correct solution to. Thanks for reminding me about this mess! -adrian From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 23 09:08:42 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C40B1BD1; Wed, 23 Oct 2013 09:08:42 +0000 (UTC) (envelope-from rank1seeker@gmail.com) Received: from mail-ea0-x230.google.com (mail-ea0-x230.google.com [IPv6:2a00:1450:4013:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 39C3D2E6B; Wed, 23 Oct 2013 09:08:42 +0000 (UTC) Received: by mail-ea0-f176.google.com with SMTP id q16so253590ead.7 for ; Wed, 23 Oct 2013 02:08:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:to:cc:subject:date; bh=lpWy4XxVUQR+7Y5whbztLr/FMbQ5jnpbsd+VnelIV5g=; b=RNSQROSTE5lFwvsewewVxP4qdCzImsxM7XUITHrDvGnlF/MdXG5aQtNvVrjzSRrlyx AQGhmkqxMWPbJXVATLUo2mRBesAiZ7k7AmpWN9bzwMBQZvxC2xrsfElNVN2YNrgO+ulM WEHZBI1cLF/lZg4Mzv+RsUhNIgGSvX1BeEi7xQb8vxf4uH+CdjJyMXVvVIelBE4SEA+s bC31H8a7NIzLEBe9RTdBqElobtEKgNz4QfiC0LuJrVioMsFhAi6gYqdfDKJiYbdfkZFO JEBosi3oz6sSuF+62kRgzZECjqkIW4jo+1JtM10ClcOqW/Yrfov2/QdO+Nit+ld9zrrf Y5yg== X-Received: by 10.14.225.199 with SMTP id z47mr711469eep.24.1382519320673; Wed, 23 Oct 2013 02:08:40 -0700 (PDT) Received: from DOMYPC ([82.193.208.225]) by mx.google.com with ESMTPSA id a1sm67619500eem.1.2013.10.23.02.08.38 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 23 Oct 2013 02:08:39 -0700 (PDT) Message-ID: <20131023.090839.469.1@DOMY-PC> From: rank1seeker@gmail.com To: "John Baldwin" Subject: Re: UFS related panic (daily <-> find) Date: Wed, 23 Oct 2013 11:08:39 +0200 X-Mailer: POP Peeper (3.8.1.0) Cc: Adam Vande More , hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Oct 2013 09:08:42 -0000 > > > > Same drill as before, see what instruction this is. Actually, this > > looks > > > > to > > > > be in the same location as your last panic, so a NULL pointer is 0x1 > > > > instead > > > > of 0x0 again. In my experience, this would still indicate failing RAM > > to > > > > me, > > > > memtest86+ notwithstanding (memtest86+ is single threaded AFAIK, so it > > may > > > > not stress the hardware quite the same, e.g. if the error is heat > > related, > > > > etc.). > > > > > > > > > memtest* cannot conclusively diagnose a dimm as good. Usually the only > > > practical solution is to swap modules with known good ones. > > > > > > > > > 0xc082c552 : cmp %ecx,0x24(%eax) > > PREVIOUS we talked about > > 0xc083bd42 : cmp %ecx,0x24(%eax) > > CURRENT ONE > > Different instruction pointer doesn't matter. The error is in the memory > that %eax is loaded from in a prior instruction. > > > Now, after all this I recompiled kernel and world and there was no crash. > > How can it be, when it is far more stresing dan daily's 'find'?! > > Because it might have shuffled where the bad memory cell now lives by having > the kernel text + data laid out differently in RAM? > > > I see addresses 0xc08* and 0xc06* appearing each time, so as I have four > > DDR1 (400) modules, each of 256 MB = 1GB, can those addresses aid me in > > targeting failing module? > > The virtual addresses (0xc*) do not matter. They are not physical addresses > which are what you would need. > > > If I can't use memtest86+-4.20, to determine failing module, then what is a > > use of it at all? > > Test RAM speed perhaps? > > Swap out your dimms. That's really the only test, esp. if you have a > reproducible crash. That is exactly what I did. I've halfed dimms. Depending on a result, I'll half them again in one of directions. Unfortunately, crash isn't reproducible, so I'll just hang with it for a month. Domagoj From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 23 19:51:25 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0FC5024D; Wed, 23 Oct 2013 19:51:25 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-qc0-x233.google.com (mail-qc0-x233.google.com [IPv6:2607:f8b0:400d:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B14AA2BCC; Wed, 23 Oct 2013 19:51:24 +0000 (UTC) Received: by mail-qc0-f179.google.com with SMTP id k18so780774qcv.10 for ; Wed, 23 Oct 2013 12:51:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=UxpzX6vbVBf75B0zRLq+T6Vv58N2D+Fn2+pMrBno7Dk=; b=vmG7RlKJrWhc08VgiXzBsorZxngi/mHR+TVLwoc2Z8BQSVA3XeUulHxIU/cM3zr+I5 HV97boW/+HuEfvh5NwIxQsHFkpHhBfzL/yrRjK30CZ50dudtC9JctyLWvtM5ROMjlaMl zw6bOYI/+q5jbHx97x0VPNRm7/ihTqi3ae+CgtMY8y6eO8fJWk8Pnh8OH2NPC8LZjSjJ odFf9OVNgoaSYBhjfLgwchxUtW4H8YjdzTDvXypalULav3qoEL2uppgRfuJPAd4POJ9t x3jwgh7c5bagc3ZyMh50ZZufN7CLRTqtgf6s/neZNVaVVJpagihhK7o9qo/nm+zya5Hs 4Nng== MIME-Version: 1.0 X-Received: by 10.49.62.3 with SMTP id u3mr4619583qer.6.1382557883752; Wed, 23 Oct 2013 12:51:23 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.207.66 with HTTP; Wed, 23 Oct 2013 12:51:23 -0700 (PDT) In-Reply-To: References: Date: Wed, 23 Oct 2013 12:51:23 -0700 X-Google-Sender-Auth: OhMTojlssnnIqRfvD4U-PPlbNrs Message-ID: Subject: Re: 9.2-STABLE r255918 with GENERIC and iwn - core dump From: Adrian Chadd To: claudiu vasadi , "freebsd-wireless@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Oct 2013 19:51:25 -0000 Hi, Please try this: http://people.freebsd.org/~adrian/ath/20131023-net80211-txmgt-locking-2.diff It implements what I think is the mostly-right fix: * remove the node reference in the callout, it's actually not freaking needed! * grab the ic lock when doing the timeout callout, ensuring consistency * expose ieee80211_new_state_locked(), as a few places will already hold the ic lock when wanting to update state Please try it out and let me know if it fixes your panics. Thanks! -adrian On 22 October 2013 07:28, claudiu vasadi wrote: > Hi everyone, > > I have a Lenovo Thinkpad T420s with Intel core i7 @ 2.70GHz, 8GB RAM, Intel > SSD 160GB and iwn0: mem > 0xf4200000-0xf4201fff irq 17 at device 0.0 on pci3 > > Today, while connecting to different AP's, I noticed at one point that I > was not getting an IP although the wifi card was associated. Within > "wifimgr", I did a "Save and Reconnect" and then got a core dump. > > Bellow, the bt: > > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you > are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address = 0xffffff801e5f7000 > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff80a10431 > stack pointer = 0x28:0xffffff8000276980 > frame pointer = 0x28:0xffffff8000276a20 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 12 (swi4: clock) > trap number = 12 > panic: page fault > cpuid = 0 > KDB: stack backtrace: > #0 0xffffffff80948a06 at kdb_backtrace+0x66 > #1 0xffffffff8090e50e at panic+0x1ce > #2 0xffffffff80cf3440 at trap_fatal+0x290 > #3 0xffffffff80cf37a1 at trap_pfault+0x211 > #4 0xffffffff80cf3d54 at trap+0x344 > #5 0xffffffff80cdd093 at calltrap+0x8 > #6 0xffffffff808dfddd at intr_event_execute_handlers+0xfd > #7 0xffffffff808e15cd at ithread_loop+0x9d > #8 0xffffffff808dc82f at fork_exit+0x11f > #9 0xffffffff80cdd5be at fork_trampoline+0xe > Uptime: 8h20m28s > Dumping 952 out of 8106 MB:..2% (CTRL-C to abort) (CTRL-C to abort) > (CTRL-C to abort) (CTRL-C to abort) ..11% (CTRL-C to abort) (CTRL-C to > abort) ..21%..31% (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) > (CTRL-C to abort) (CTRL-C to abort) ..41% (CTRL-C to abort) (CTRL-C to > abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to > abort) (CTRL-C to abort) (CTRL-C to abort) ..51% (CTRL-C to abort) > (CTRL-C to abort) ..61% (CTRL-C to abort) ..71% (CTRL-C to abort) > ..81%..91% > > Reading symbols from /boot/kernel/zfs.ko...Reading symbols from > /boot-mount/boot/kernel/zfs.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/zfs.ko > Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from > /boot-mount/boot/kernel/opensolaris.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/opensolaris.ko > Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from > /boot-mount/boot/kernel/geom_eli.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/geom_eli.ko > Reading symbols from /boot/kernel/crypto.ko...Reading symbols from > /boot-mount/boot/kernel/crypto.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/crypto.ko > Reading symbols from /boot/kernel/linux.ko...Reading symbols from > /boot-mount/boot/kernel/linux.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/linux.ko > Reading symbols from /boot/kernel/drm.ko...Reading symbols from > /boot-mount/boot/kernel/drm.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/drm.ko > Reading symbols from /boot/modules/nvidia.ko...done. > Loaded symbols for /boot/modules/nvidia.ko > Reading symbols from /boot/kernel/mmc.ko...Reading symbols from > /boot-mount/boot/kernel/mmc.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/mmc.ko > Reading symbols from /boot/kernel/mmcsd.ko...Reading symbols from > /boot-mount/boot/kernel/mmcsd.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/mmcsd.ko > Reading symbols from /boot/kernel/acpi_call.ko...done. > Loaded symbols for /boot/kernel/acpi_call.ko > Reading symbols from /boot/kernel/umodem.ko...Reading symbols from > /boot-mount/boot/kernel/umodem.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/umodem.ko > Reading symbols from /boot/modules/vboxnetflt.ko...done. > Loaded symbols for /boot/modules/vboxnetflt.ko > Reading symbols from /boot/modules/vboxdrv.ko...done. > Loaded symbols for /boot/modules/vboxdrv.ko > Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from > /boot-mount/boot/kernel/netgraph.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/netgraph.ko > Reading symbols from /boot/kernel/ng_ether.ko...Reading symbols from > /boot-mount/boot/kernel/ng_ether.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/ng_ether.ko > Reading symbols from /boot/modules/vboxnetadp.ko...done. > Loaded symbols for /boot/modules/vboxnetadp.ko > #0 doadump (textdump=) at pcpu.h:234 > 234 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) bt > #0 doadump (textdump=) at pcpu.h:234 > #1 0xffffffff8090dfe6 in kern_reboot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:449 > #2 0xffffffff8090e4e7 in panic (fmt=0x1
) at > /usr/src/sys/kern/kern_shutdown.c:637 > #3 0xffffffff80cf3440 in trap_fatal (frame=0xc, eva=) > at /usr/src/sys/amd64/amd64/trap.c:879 > #4 0xffffffff80cf37a1 in trap_pfault (frame=0xffffff80002768d0, > usermode=0) at /usr/src/sys/amd64/amd64/trap.c:795 > #5 0xffffffff80cf3d54 in trap (frame=0xffffff80002768d0) at > /usr/src/sys/amd64/amd64/trap.c:463 > #6 0xffffffff80cdd093 in calltrap () at > /usr/src/sys/amd64/amd64/exception.S:232 > #7 0xffffffff80a10431 in ieee80211_tx_mgt_timeout (arg=0xffffff801e5f7000) > at /usr/src/sys/net80211/ieee80211_output.c:2487 > #8 0xffffffff809246e8 in softclock (arg=) at > /usr/src/sys/kern/kern_timeout.c:518 > #9 0xffffffff808dfddd in intr_event_execute_handlers (p= out>, ie=0xfffffe0007221b00) > at /usr/src/sys/kern/kern_intr.c:1272 > #10 0xffffffff808e15cd in ithread_loop (arg=0xfffffe0007209460) at > /usr/src/sys/kern/kern_intr.c:1285 > #11 0xffffffff808dc82f in fork_exit (callout=0xffffffff808e1530 > , arg=0xfffffe0007209460, > frame=0xffffff8000276b00) at /usr/src/sys/kern/kern_fork.c:990 > #12 0xffffffff80cdd5be in fork_trampoline () at > /usr/src/sys/amd64/amd64/exception.S:606 > #13 0x0000000000000000 in ?? () > > > One thing to keep in mind is that since I started using geli+ZFS (installed > with PC-BSD 9.1 cd), I always got "Cannot reset interface wlan0 - exit > status 1" with "wifimgr" whichever action i did (ex: reconect, rescan, > up/down,etc). > > > I would appreciate some help in debugging this. > > > -- > Best regards, > Claudiu Vasadi > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 23 20:10:48 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0AC119DA; Wed, 23 Oct 2013 20:10:48 +0000 (UTC) (envelope-from claudiu.vasadi@gmail.com) Received: from mail-ie0-x236.google.com (mail-ie0-x236.google.com [IPv6:2607:f8b0:4001:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B9ECE2CF0; Wed, 23 Oct 2013 20:10:47 +0000 (UTC) Received: by mail-ie0-f182.google.com with SMTP id as1so2275731iec.13 for ; Wed, 23 Oct 2013 13:10:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=H64GEOoLUEbLFohtulDjwlTdi+vUPUmL2/2qADFQRjs=; b=jNiSLOuEQNUSvVnz9TfUBcGulSLHag/jG2hQlCt6eT7jPi6fwkyDnBDI6JQ59SYlzO riHAoxQrxDErVNOQtSbeU6flR5/VhkUHYuSn0GZsztHj9JUMfFcwxaVmHgFjHcFYzLta luM//lb8AEikp886Yzd4Xzq/NNRCoDoWIfSNvwbGLpt45XDjc5nuH/P9QLV0R4aIo7i6 ft32GiLSv7S6qWGQnO7oMYdDwkGxUQeE8EIzDCzoaHgxOmsUfG1aG0DkZR1Gr0uka0F5 68ScQWv+yRFA3rpvh813uJoVkuyQ/v5dMjzhAzf4GCcivyayAFHV7cMLV6DneetXSoWB M8MQ== MIME-Version: 1.0 X-Received: by 10.50.122.40 with SMTP id lp8mr1837693igb.24.1382559047120; Wed, 23 Oct 2013 13:10:47 -0700 (PDT) Received: by 10.64.134.169 with HTTP; Wed, 23 Oct 2013 13:10:46 -0700 (PDT) In-Reply-To: References: Date: Wed, 23 Oct 2013 22:10:46 +0200 Message-ID: Subject: Re: 9.2-STABLE r255918 with GENERIC and iwn - core dump From: claudiu vasadi To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-hackers@freebsd.org" , "freebsd-wireless@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Oct 2013 20:10:48 -0000 Hi, Still getting the "Cannot reset interface wlan0 - exit status 1" in wifimgr but no crash yet. Will keep trying :D On Wed, Oct 23, 2013 at 9:51 PM, Adrian Chadd wrote: > Hi, > > Please try this: > > > http://people.freebsd.org/~adrian/ath/20131023-net80211-txmgt-locking-2.diff > > It implements what I think is the mostly-right fix: > > * remove the node reference in the callout, it's actually not freaking > needed! > * grab the ic lock when doing the timeout callout, ensuring consistency > * expose ieee80211_new_state_locked(), as a few places will already hold > the ic lock when wanting to update state > > Please try it out and let me know if it fixes your panics. > > Thanks! > > > > -adrian > > > > On 22 October 2013 07:28, claudiu vasadi wrote: > >> Hi everyone, >> >> I have a Lenovo Thinkpad T420s with Intel core i7 @ 2.70GHz, 8GB RAM, >> Intel >> SSD 160GB and iwn0: mem >> 0xf4200000-0xf4201fff irq 17 at device 0.0 on pci3 >> >> Today, while connecting to different AP's, I noticed at one point that I >> was not getting an IP although the wifi card was associated. Within >> "wifimgr", I did a "Save and Reconnect" and then got a core dump. >> >> Bellow, the bt: >> >> >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and you >> are >> welcome to change it and/or distribute copies of it under certain >> conditions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for >> details. >> This GDB was configured as "amd64-marcel-freebsd"... >> >> Unread portion of the kernel message buffer: >> >> >> Fatal trap 12: page fault while in kernel mode >> cpuid = 0; apic id = 00 >> fault virtual address = 0xffffff801e5f7000 >> fault code = supervisor read data, page not present >> instruction pointer = 0x20:0xffffffff80a10431 >> stack pointer = 0x28:0xffffff8000276980 >> frame pointer = 0x28:0xffffff8000276a20 >> code segment = base 0x0, limit 0xfffff, type 0x1b >> = DPL 0, pres 1, long 1, def32 0, gran 1 >> processor eflags = interrupt enabled, resume, IOPL = 0 >> current process = 12 (swi4: clock) >> trap number = 12 >> panic: page fault >> cpuid = 0 >> KDB: stack backtrace: >> #0 0xffffffff80948a06 at kdb_backtrace+0x66 >> #1 0xffffffff8090e50e at panic+0x1ce >> #2 0xffffffff80cf3440 at trap_fatal+0x290 >> #3 0xffffffff80cf37a1 at trap_pfault+0x211 >> #4 0xffffffff80cf3d54 at trap+0x344 >> #5 0xffffffff80cdd093 at calltrap+0x8 >> #6 0xffffffff808dfddd at intr_event_execute_handlers+0xfd >> #7 0xffffffff808e15cd at ithread_loop+0x9d >> #8 0xffffffff808dc82f at fork_exit+0x11f >> #9 0xffffffff80cdd5be at fork_trampoline+0xe >> Uptime: 8h20m28s >> Dumping 952 out of 8106 MB:..2% (CTRL-C to abort) (CTRL-C to abort) >> (CTRL-C to abort) (CTRL-C to abort) ..11% (CTRL-C to abort) (CTRL-C to >> abort) ..21%..31% (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) >> (CTRL-C to abort) (CTRL-C to abort) ..41% (CTRL-C to abort) (CTRL-C to >> abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C >> to >> abort) (CTRL-C to abort) (CTRL-C to abort) ..51% (CTRL-C to abort) >> (CTRL-C to abort) ..61% (CTRL-C to abort) ..71% (CTRL-C to abort) >> ..81%..91% >> >> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from >> /boot-mount/boot/kernel/zfs.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/zfs.ko >> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from >> /boot-mount/boot/kernel/opensolaris.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/opensolaris.ko >> Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from >> /boot-mount/boot/kernel/geom_eli.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/geom_eli.ko >> Reading symbols from /boot/kernel/crypto.ko...Reading symbols from >> /boot-mount/boot/kernel/crypto.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/crypto.ko >> Reading symbols from /boot/kernel/linux.ko...Reading symbols from >> /boot-mount/boot/kernel/linux.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/linux.ko >> Reading symbols from /boot/kernel/drm.ko...Reading symbols from >> /boot-mount/boot/kernel/drm.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/drm.ko >> Reading symbols from /boot/modules/nvidia.ko...done. >> Loaded symbols for /boot/modules/nvidia.ko >> Reading symbols from /boot/kernel/mmc.ko...Reading symbols from >> /boot-mount/boot/kernel/mmc.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/mmc.ko >> Reading symbols from /boot/kernel/mmcsd.ko...Reading symbols from >> /boot-mount/boot/kernel/mmcsd.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/mmcsd.ko >> Reading symbols from /boot/kernel/acpi_call.ko...done. >> Loaded symbols for /boot/kernel/acpi_call.ko >> Reading symbols from /boot/kernel/umodem.ko...Reading symbols from >> /boot-mount/boot/kernel/umodem.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/umodem.ko >> Reading symbols from /boot/modules/vboxnetflt.ko...done. >> Loaded symbols for /boot/modules/vboxnetflt.ko >> Reading symbols from /boot/modules/vboxdrv.ko...done. >> Loaded symbols for /boot/modules/vboxdrv.ko >> Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from >> /boot-mount/boot/kernel/netgraph.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/netgraph.ko >> Reading symbols from /boot/kernel/ng_ether.ko...Reading symbols from >> /boot-mount/boot/kernel/ng_ether.ko.symbols...done. >> done. >> Loaded symbols for /boot/kernel/ng_ether.ko >> Reading symbols from /boot/modules/vboxnetadp.ko...done. >> Loaded symbols for /boot/modules/vboxnetadp.ko >> #0 doadump (textdump=) at pcpu.h:234 >> 234 pcpu.h: No such file or directory. >> in pcpu.h >> (kgdb) bt >> #0 doadump (textdump=) at pcpu.h:234 >> #1 0xffffffff8090dfe6 in kern_reboot (howto=260) at >> /usr/src/sys/kern/kern_shutdown.c:449 >> #2 0xffffffff8090e4e7 in panic (fmt=0x1
) at >> /usr/src/sys/kern/kern_shutdown.c:637 >> #3 0xffffffff80cf3440 in trap_fatal (frame=0xc, eva=> out>) >> at /usr/src/sys/amd64/amd64/trap.c:879 >> #4 0xffffffff80cf37a1 in trap_pfault (frame=0xffffff80002768d0, >> usermode=0) at /usr/src/sys/amd64/amd64/trap.c:795 >> #5 0xffffffff80cf3d54 in trap (frame=0xffffff80002768d0) at >> /usr/src/sys/amd64/amd64/trap.c:463 >> #6 0xffffffff80cdd093 in calltrap () at >> /usr/src/sys/amd64/amd64/exception.S:232 >> #7 0xffffffff80a10431 in ieee80211_tx_mgt_timeout >> (arg=0xffffff801e5f7000) >> at /usr/src/sys/net80211/ieee80211_output.c:2487 >> #8 0xffffffff809246e8 in softclock (arg=) at >> /usr/src/sys/kern/kern_timeout.c:518 >> #9 0xffffffff808dfddd in intr_event_execute_handlers (p=> out>, ie=0xfffffe0007221b00) >> at /usr/src/sys/kern/kern_intr.c:1272 >> #10 0xffffffff808e15cd in ithread_loop (arg=0xfffffe0007209460) at >> /usr/src/sys/kern/kern_intr.c:1285 >> #11 0xffffffff808dc82f in fork_exit (callout=0xffffffff808e1530 >> , arg=0xfffffe0007209460, >> frame=0xffffff8000276b00) at /usr/src/sys/kern/kern_fork.c:990 >> #12 0xffffffff80cdd5be in fork_trampoline () at >> /usr/src/sys/amd64/amd64/exception.S:606 >> #13 0x0000000000000000 in ?? () >> >> >> One thing to keep in mind is that since I started using geli+ZFS >> (installed >> with PC-BSD 9.1 cd), I always got "Cannot reset interface wlan0 - exit >> status 1" with "wifimgr" whichever action i did (ex: reconect, rescan, >> up/down,etc). >> >> >> I would appreciate some help in debugging this. >> >> >> -- >> Best regards, >> Claudiu Vasadi >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org >> " >> > > -- Best regards, Claudiu Vasadi From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 23 20:12:06 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B397FB09; Wed, 23 Oct 2013 20:12:06 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-qa0-x22a.google.com (mail-qa0-x22a.google.com [IPv6:2607:f8b0:400d:c00::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6176F2D09; Wed, 23 Oct 2013 20:12:06 +0000 (UTC) Received: by mail-qa0-f42.google.com with SMTP id w8so4549751qac.1 for ; Wed, 23 Oct 2013 13:12:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ptGaPrkKH2ITZuOvN+pDH0mPSk/FVwrhwrS3zW/t/Rk=; b=FXDaZu3o6ewgdKd3nao3CvR7M6GxKnFJBN3l1srVmdGtc7Rqp/+qvqy5vAR6zYwVWg DyglWY43mt+hIf++xUz5fRky8mv7fQJFOUW4pywbRAtBgzWVPxwTVtrLcNIUmBwpUiEn g5xWK5l1yFi5q+bVWMKkQ8/w+KZXa7Bc+pzl2UdISfV4k68rkFT2WkTWAMb8Q9iy/Xt8 5Nlptnj0BPWVoHc8MVSmiYE4tgpDO6voQpmZx8vInYPjgIt9WqeoYO1/VPT+lxloDY9b HUS+o0Ir/97IXSlDugnXhNAC0LjXyvU3FMHXnv5JzmSbT8jL6xTMTOmmtrQ4VhaABTgz SjCg== MIME-Version: 1.0 X-Received: by 10.224.63.199 with SMTP id c7mr6500239qai.74.1382559125416; Wed, 23 Oct 2013 13:12:05 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.207.66 with HTTP; Wed, 23 Oct 2013 13:12:05 -0700 (PDT) In-Reply-To: References: Date: Wed, 23 Oct 2013 13:12:05 -0700 X-Google-Sender-Auth: 6ZbBgbIeEKK4TEPHC5R0axUaAOc Message-ID: Subject: Re: 9.2-STABLE r255918 with GENERIC and iwn - core dump From: Adrian Chadd To: claudiu vasadi Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-hackers@freebsd.org" , "freebsd-wireless@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Oct 2013 20:12:06 -0000 On 23 October 2013 13:10, claudiu vasadi wrote: > Hi, > > Still getting the "Cannot reset interface wlan0 - exit status 1" in > wifimgr but no crash yet. Will keep trying :D > I have no idea about that. It's likely there's some net80211/iwn bug(s) but I don't use wifimgr so I don't know what it's doing. For that I'd bug the wifimgr people in PCBSD. they can always file bug reports with me :) Thanks, -adrian From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 23 20:14:19 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DC949D27; Wed, 23 Oct 2013 20:14:18 +0000 (UTC) (envelope-from claudiu.vasadi@gmail.com) Received: from mail-ie0-x22b.google.com (mail-ie0-x22b.google.com [IPv6:2607:f8b0:4001:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 982312D3A; Wed, 23 Oct 2013 20:14:18 +0000 (UTC) Received: by mail-ie0-f171.google.com with SMTP id tp5so2250648ieb.16 for ; Wed, 23 Oct 2013 13:14:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=4SHMoPUlD4awTCckXGIQmib87gYFYIdoXi5b0NzFynQ=; b=o/0zqZYZErvbRaP8ACA9sF86apSA9il4BXxUIbCk2/AQGjipf4+I++yn3X5WUiNz5f w1JulIg7iZNUIee/PhFTf7PfhyN9sTLUUk6thfVe7Jmu70XHH6gXLOvDbiOaYrBH/po/ m2mZGYmtUYNm5Y/OyaOT+yTpcin6rrCk2uOj0MUJhJkxae1k7jmC2Hbj7X8Xq4yZsQPy it2d+jZl93TlOxPQi9PnKXfB86b7NAwzkIVIU2hQhg4wWpGqu3KfzdrT2EBIBnnFFFS6 S/JyI8sucexNGDKSdQgBqPGfds2y2kNfr5+zpLQCVlOuCzRzBvwdanNfsO4dS9QO+QU/ ToTg== MIME-Version: 1.0 X-Received: by 10.43.57.79 with SMTP id wf15mr1595191icb.14.1382559257886; Wed, 23 Oct 2013 13:14:17 -0700 (PDT) Received: by 10.64.134.169 with HTTP; Wed, 23 Oct 2013 13:14:17 -0700 (PDT) In-Reply-To: References: Date: Wed, 23 Oct 2013 22:14:17 +0200 Message-ID: Subject: Re: 9.2-STABLE r255918 with GENERIC and iwn - core dump From: claudiu vasadi To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-hackers@freebsd.org" , "freebsd-wireless@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Oct 2013 20:14:19 -0000 Understood. Will do so. On Wed, Oct 23, 2013 at 10:12 PM, Adrian Chadd wrote: > > > > On 23 October 2013 13:10, claudiu vasadi wrote: > >> Hi, >> >> Still getting the "Cannot reset interface wlan0 - exit status 1" in >> wifimgr but no crash yet. Will keep trying :D >> > > I have no idea about that. It's likely there's some net80211/iwn bug(s) > but I don't use wifimgr so I don't know what it's doing. > > For that I'd bug the wifimgr people in PCBSD. they can always file bug > reports with me :) > > Thanks, > > > > -adrian > > -- Best regards, Claudiu Vasadi From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 01:59:04 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C07B1418 for ; Thu, 24 Oct 2013 01:59:04 +0000 (UTC) (envelope-from pmaster@mindslayer.net) Received: from smtp85.ord1c.emailsrvr.com (smtp85.ord1c.emailsrvr.com [108.166.43.85]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A02982051 for ; Thu, 24 Oct 2013 01:59:03 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp3.relay.ord1c.emailsrvr.com (SMTP Server) with ESMTP id 9C44E5009E for ; Wed, 23 Oct 2013 21:59:02 -0400 (EDT) X-Virus-Scanned: OK Received: by smtp3.relay.ord1c.emailsrvr.com (Authenticated sender: peter-AT-farm-market.net) with ESMTPSA id 70F6350087 for ; Wed, 23 Oct 2013 21:59:02 -0400 (EDT) Message-ID: <52687ED8.6080309@mindslayer.net> Date: Wed, 23 Oct 2013 20:58:48 -0500 From: Puppet Master User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: FoxPro on FreeBSD Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 01:59:04 -0000 Hello, A couple of years ago I posted a message on the forums, regarding getting FoxPro (for SCO Unix) working on FreeBSD. The direct link to the forum is here: http://forums.freebsd.org/showthread.php?t=19764 From version 2.2.8 to 7.4 of FreeBSD this has worked flawlessly, but ceased to run with version 8.1. I would really love for this to continue working with future versions but have not found anyone that can help. A few days ago I received a private message on the forum requesting that I please post to hackers@ to see if anyone could help. The idea being that "Its unlikely the SCO binaries get tested very regularly." So here I am. In case anyone wants to test the FoxPro for FreeBSD, you can download my Perl script that installs it from here: http://www.at-vantage.com/foxfiles/foxp_install.txt (rename the foxp_install.txt to foxp_install.pl). This checks for i386, FreeBSD version before installing so if you plan on testing it on 8.1 or above, you may need to modify the script and comment out the sections that check for that. I do hope that someone can help, it would be great if the SCO binary for FoxPro would function again. Thanks. From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 03:32:15 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E5C44AFB for ; Thu, 24 Oct 2013 03:32:14 +0000 (UTC) (envelope-from toasty@dragondata.com) Received: from mail-yh0-x235.google.com (mail-yh0-x235.google.com [IPv6:2607:f8b0:4002:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A45F72559 for ; Thu, 24 Oct 2013 03:32:14 +0000 (UTC) Received: by mail-yh0-f53.google.com with SMTP id z20so700843yhz.40 for ; Wed, 23 Oct 2013 20:32:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dragondata.com; s=google; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=7den9KoU7io+u6qRLV1bOIQBzqfPvO0SO8yU7V4Z20c=; b=GbUM0xygspVgXEiyaym+NaGkzEEliNBYQmHrnqfXY8iMK+G2VVk+YwdCKgz7Ac0RG+ xBaExaLo71TfPiEoTvPPSWr0DlZMlZA31WTw0TK+3tlfqvKm51wL1fmgfMOYyoz1PDm2 iTKpNZQzQTvYOoPbEA5I5Ezo93gmSjkeHNbCY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=7den9KoU7io+u6qRLV1bOIQBzqfPvO0SO8yU7V4Z20c=; b=nANP4cZndUe0gaIejb3Ozv6dK3i1Xfcx12Qr+2h+uDIusqQltZtdtBn1znBC4rLoXg RkFx3bQ2CDMZ8AfhBg34BWnaBAUx4BVvAp0Za9dYClgY94fFq6yrfBnb6jNVUau6vzL1 0WlTkySv6KOnwIp9v8WV0DpWeu1DmihKr7qNGcTUrDO7axMguMt9iiRfOgbfHYXqsV4i 76PdjhgYEhqsBeJ2r0RIaS77PN1MkxdtcgEgKnYcQ7rfXWkw0TZQiptYzwOfsYsOIGIO 69yMCUFpEoCeGgs/deRc5lqZWC6wxuh43xPfp+3i1qzo5WOoE1AkR3ANTWhvtA/0z/ng h25g== X-Gm-Message-State: ALoCoQnF6JkIW6XRCnwsQqWkFbIM/xdxnUIMuNC7OjfC/cQjVJOclDXkv4oSZvkCbkfeiQr57wsv X-Received: by 10.236.139.198 with SMTP id c46mr362451yhj.78.1382585533592; Wed, 23 Oct 2013 20:32:13 -0700 (PDT) Received: from unassigned.v6.your.org ([2001:4978:1:45:b5b6:f351:31fb:459f]) by mx.google.com with ESMTPSA id w8sm50130246yhg.8.2013.10.23.20.32.12 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 23 Oct 2013 20:32:13 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1812\)) Subject: Re: FoxPro on FreeBSD From: Kevin Day In-Reply-To: <52687ED8.6080309@mindslayer.net> Date: Wed, 23 Oct 2013 22:32:11 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <9B89077C-6BE7-49F1-9F22-19FAD9F6C3ED@dragondata.com> References: <52687ED8.6080309@mindslayer.net> To: Puppet Master X-Mailer: Apple Mail (2.1812) Cc: "freebsd-hackers@freebsd.org Hackers" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 03:32:15 -0000 I did some debugging, and watched how the process was getting launched, = and I've managed to get it to load! The problem was that COFF files expect to be mapped into memory at = address 0, something that processes are no longer allowed to do. Run "sysctl security.bsd.map_at_zero=3D1=94 or add = =93security.bsd.map_at_zero=3D1=93 to /etc/sysctl.conf and you should = have it working. We probably should either make an exception for COFF = files to bypass this the sysctl restriction, or at least print a more = helpful error than just letting the process segfault because it didn=92t = get mapped where it was supposed to go. =97 Kevin On Oct 23, 2013, at 8:58 PM, Puppet Master = wrote: > Hello, >=20 > A couple of years ago I posted a message on the forums, regarding = getting FoxPro (for SCO Unix) working on FreeBSD. The direct link to the = forum is here: > http://forums.freebsd.org/showthread.php?t=3D19764 >=20 > =46rom version 2.2.8 to 7.4 of FreeBSD this has worked flawlessly, but = ceased to run with version 8.1. >=20 > I would really love for this to continue working with future versions = but have not found anyone that can help. A few days ago I received a = private message on the forum requesting that I please post to hackers@ = to see if anyone could help. The idea being that "Its unlikely the SCO = binaries get tested very regularly." >=20 > So here I am. In case anyone wants to test the FoxPro for FreeBSD, = you can download my Perl script that installs it from here: = http://www.at-vantage.com/foxfiles/foxp_install.txt >=20 > (rename the foxp_install.txt to foxp_install.pl). > This checks for i386, FreeBSD version before installing so if you plan = on testing it on 8.1 or above, you may need to modify the script and = comment out the sections that check for that. >=20 > I do hope that someone can help, it would be great if the SCO binary = for FoxPro would function again. >=20 > Thanks. >=20 >=20 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to = "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 05:54:52 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6F6ACEE2 for ; Thu, 24 Oct 2013 05:54:52 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3FD3E2B06 for ; Thu, 24 Oct 2013 05:54:52 +0000 (UTC) Received: from delphij-macbook.local (unknown [IPv6:2001:470:83bf:0:9080:b94f:95b7:ed12]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id CEF732A3D8; Wed, 23 Oct 2013 22:54:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1382594091; bh=2NudMPIzkXk2wME9emKD/pcBGYGxdmEK4O5IGzU7yZ8=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=4WoYNEsRT/lCEcfx7ANQ/rR5fZo6vmiIc/yNX+bLA6rN8lBBDvRmkqCOOeW2paQ/t o+jxNtGV4nGCR18Q4ytePFCTQWgvOUiqSS/pAQZIC2f+MxQ1XMMRDNY/F6LczNGtmA Q0iklSaqf8eZ3uDXd16BWQaeFLBOgY4Zl5EMyHMU= Message-ID: <5268B62B.3000104@delphij.net> Date: Wed, 23 Oct 2013 22:54:51 -0700 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Kevin Day , Puppet Master Subject: Re: FoxPro on FreeBSD References: <52687ED8.6080309@mindslayer.net> <9B89077C-6BE7-49F1-9F22-19FAD9F6C3ED@dragondata.com> In-Reply-To: <9B89077C-6BE7-49F1-9F22-19FAD9F6C3ED@dragondata.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Cc: "freebsd-hackers@freebsd.org Hackers" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 05:54:52 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 10/23/13, 8:32 PM, Kevin Day wrote: > I did some debugging, and watched how the process was getting > launched, and I've managed to get it to load! > > The problem was that COFF files expect to be mapped into memory at > address 0, something that processes are no longer allowed to do. > > Run "sysctl security.bsd.map_at_zero=1” or add > “security.bsd.map_at_zero=1“ to /etc/sysctl.conf and you should > have it working. We probably should either make an exception for > COFF files to bypass this the sysctl restriction, or at least print > a more helpful error than just letting the process segfault because > it didn’t get mapped where it was supposed to go. Wow, this is impressive find, indeed! Do they need to do the map at startup only, or do they want to explicitly map something at address 0 during runtime? > — Kevin > > > On Oct 23, 2013, at 8:58 PM, Puppet Master > wrote: > >> Hello, >> >> A couple of years ago I posted a message on the forums, regarding >> getting FoxPro (for SCO Unix) working on FreeBSD. The direct link >> to the forum is here: >> http://forums.freebsd.org/showthread.php?t=19764 >> >> From version 2.2.8 to 7.4 of FreeBSD this has worked flawlessly, >> but ceased to run with version 8.1. >> >> I would really love for this to continue working with future >> versions but have not found anyone that can help. A few days ago >> I received a private message on the forum requesting that I >> please post to hackers@ to see if anyone could help. The idea >> being that "Its unlikely the SCO binaries get tested very >> regularly." >> >> So here I am. In case anyone wants to test the FoxPro for >> FreeBSD, you can download my Perl script that installs it from >> here: http://www.at-vantage.com/foxfiles/foxp_install.txt >> >> (rename the foxp_install.txt to foxp_install.pl). This checks for >> i386, FreeBSD version before installing so if you plan on testing >> it on 8.1 or above, you may need to modify the script and comment >> out the sections that check for that. >> >> I do hope that someone can help, it would be great if the SCO >> binary for FoxPro would function again. >> >> Thanks. >> >> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To >> unsubscribe, send any mail to >> "freebsd-hackers-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To > unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" > -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJSaLYrAAoJEJW2GBstM+nsLJUQAKrSrqgBcMmzlURu9XFWedCc pcr9HyqNqssb2h0P1EiSZAOt39LccXtbyWRLghwxfNUc3UPsb9ShFBVKia4LJiqe H8yqgbg2cw8SKLpYthTVfKKxqR1wKiaV0NN8Hh6ZMuo5Hp0X/Dkd83VHoZNy8aY0 WjAOTpUs094hC8PtA9SfnH+BWzFxse6IXqAQtL/XUwfRkrZWCxoReUvgDgKdDDju uoQtmykVM2tMF+HwdAKOIcK8WiBNzTZDoNt0R0F+VbXOLDG+ziNJ4x70j005EOYt uClwFTVpkpMhf44xaM8PS7ta4pqO3DuhqMFSWMK6PyA/cjj4pMvwLJQDad6NViH5 pAK3XILry6Q4YxcXKM1sglPk+Vpb+/3RyW44LZAzFi0SdTAUytbaRvbRgVx5HVvl 8oi0r6DOjYQTAHyU0r+I4uZy7F+btniq7ul/5lqiIx4q+o5xWlQXnDzY+tvh7gJg LkkDWUuJVfQPpAFRbsSU7c0RsoucR9yzviaEvVKn3pxGasOTpJCgnZQB46K+F0yy v5vYCNpYvUxsz7dm16BgneqlHONnuj61uGf93/bhoB1wMOV/cjdwZDjwBTiry0X8 OLTwQwB9wXgWNA+m9/pe1AzZPj6admr7KDeQsSw+TsPHquwiJzqp5mDfM73g4m+m LK18ePNLK2SWq8HSUbYM =3qLK -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 06:37:55 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 1EEA4C46 for ; Thu, 24 Oct 2013 06:37:55 +0000 (UTC) (envelope-from toasty@dragondata.com) Received: from mail-ie0-x22e.google.com (mail-ie0-x22e.google.com [IPv6:2607:f8b0:4001:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D94972D52 for ; Thu, 24 Oct 2013 06:37:54 +0000 (UTC) Received: by mail-ie0-f174.google.com with SMTP id qd12so3130713ieb.5 for ; Wed, 23 Oct 2013 23:37:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dragondata.com; s=google; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=akjQPXhiZWKEKdy2OWc0GsXuEP7WodfxkXMjlioBvxE=; b=YI35iPZI9ga7YXH2b9PwFJsqdFLbxPTF5B3AKFPpULtZahxtezl55slpGMpg70QB4m y7+SATMZf0LeBMi5H8t9icJyolx0NgR5QmO4Oe6btvaq3Dn8yZB9QtILK9DJDH9bvcaB hKAR0lWXzP00EzNg+BMstBJW/MOrxa9OU/n8Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=akjQPXhiZWKEKdy2OWc0GsXuEP7WodfxkXMjlioBvxE=; b=PupO/RoJt5uh2yEqn1zk0/98zIpnhVQMT0HqZXwU6sdHMxAySdXvuBDhpuqK5FmNhc ynVqHGSZVgfw9PwdP6w/bBZTwyvbqj9d6H1qA6j2s7Aj2fwg8tYPXZbiznMVyZeTDpNc PObGqUlk1D+4rqBqOeAnOLEywzT054tXGdKwve3OeOlH0Kk1+DPjfhK/lVmHB/euZY7q p9iQePzZXUdz+iSJQBb/5WK6xO7lgr63GpALDbifYFvAuLeCmUEMlNH39DNdKq6fyK6X MvUu2qUizyPBoO+FYWzpz8H7sE5sWLmJimwYXnASzsGyAOgkeSMnP9G484ZfOUWPNvX+ nlWg== X-Gm-Message-State: ALoCoQn9/zRCt+AiPpwlUVZV0OHDtiwobU+ZIeDBjICMEYOSjWOperG5LGL0jPhO45CZeo8a5NI6 X-Received: by 10.43.98.202 with SMTP id cp10mr674767icc.28.1382596674040; Wed, 23 Oct 2013 23:37:54 -0700 (PDT) Received: from vpn132.rw1.your.org (vpn132.rw1.your.org. [204.9.51.132]) by mx.google.com with ESMTPSA id ka5sm10675687igb.2.2013.10.23.23.37.52 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 23 Oct 2013 23:37:53 -0700 (PDT) Content-Type: multipart/signed; boundary="Apple-Mail=_540E5AEC-C04D-4A1D-8577-276AD9832813"; protocol="application/pkcs7-signature"; micalg=sha1 Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1812\)) Subject: Re: FoxPro on FreeBSD From: Kevin Day In-Reply-To: <5268B62B.3000104@delphij.net> Date: Thu, 24 Oct 2013 01:37:51 -0500 Message-Id: <8A799DDB-3D5C-4418-B064-A2B7821EE0F2@dragondata.com> References: <52687ED8.6080309@mindslayer.net> <9B89077C-6BE7-49F1-9F22-19FAD9F6C3ED@dragondata.com> <5268B62B.3000104@delphij.net> To: d@delphij.net, Xin Li X-Mailer: Apple Mail (2.1812) Cc: Puppet Master , "freebsd-hackers@freebsd.org Hackers" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 06:37:55 -0000 --Apple-Mail=_540E5AEC-C04D-4A1D-8577-276AD9832813 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Oct 24, 2013, at 12:54 AM, Xin Li wrote: > Signed PGP part > On 10/23/13, 8:32 PM, Kevin Day wrote: > > I did some debugging, and watched how the process was getting > > launched, and I've managed to get it to load! > >=20 > > The problem was that COFF files expect to be mapped into memory at > > address 0, something that processes are no longer allowed to do. > >=20 > > Run "sysctl security.bsd.map_at_zero=3D1=94 or add > > =93security.bsd.map_at_zero=3D1=93 to /etc/sysctl.conf and you = should > > have it working. We probably should either make an exception for > > COFF files to bypass this the sysctl restriction, or at least print > > a more helpful error than just letting the process segfault because > > it didn=92t get mapped where it was supposed to go. >=20 > Wow, this is impressive find, indeed! Do they need to do the map at > startup only, or do they want to explicitly map something at address 0 > during runtime? It=92s the COFF loader in sys/i386/ibcs2 that=92s attempting to do this, = with some debug printing enabled on the ibcs2 module, you can see the = layout of the binary: i =3D 0, s_name =3D .text, s_vaddr =3D 000000d0, s_scnptr =3D 208 s_size = =3D 1f9260 i =3D 1, s_name =3D .data, s_vaddr =3D 00400330, s_scnptr =3D 2069296 = s_size =3D 10598 i =3D 2, s_name =3D .bss, s_vaddr =3D 004108c8, s_scnptr =3D 0 s_size =3D = 1ebb0 i =3D 3, s_name =3D .comment, s_vaddr =3D 00000000, s_scnptr =3D 2136264 = s_size =3D feb4 which maps to these calls: vm_mmap(&vmspace->vm_map, &0x00000000, 0x1fa000, 0x5, VM_PROT_ALL, = MAP_PRIVATE | MAP_FIXED, OBJT_VNODE, vp, 0x0) vm_mmap(&vmspace->vm_map, &0x00400000, 0x10000, 0x7, VM_PROT_ALL, = MAP_PRIVATE | MAP_FIXED, OBJT_VNODE, vp, 0x1f9000) vm_map_find(&vmspace->vm_map, NULL, 0, &0x00410000,0x20000, = VMFS_NO_SPACE, VM_PROT_ALL, VM_PROT_ALL, 0) vm_map_find(&vmspace->vm_map, NULL, 0, &0x430000, PAGE_SIZE, FALSE, = VM_PROT_ALL, VM_PROT_ALL, 0) Nothing is returning any errors, but the .text session isn=92t getting = mapped to the desired location (0x0). If map_at_zero is set to 0, the = process=92s vm_map has min_offset set to PAGE_SIZE instead of 0.=20 What=92s actually happening is pretty subtle. if MAP_FIXED is set, = vm_mmap() uses vm_map_fixed() to create the mapping. Inside = vm_map_fixed(), it uses vm_map_insert() which would properly error out = that this mapping is impossible (we want 0x0, but the process=92s = vm_map.min_offset is 0x1000), but vm_map_fixed() calls = VM_MAP_RANGE_CHECK first: VM_MAP_RANGE_CHECK(map, start, end); (void) vm_map_delete(map, start, end); result =3D vm_map_insert(map, object, offset, start, end, prot, VM_MAP_RANGE_CHECK does: if (start < vm_map_min(map)) \ start =3D vm_map_min(map); \ which looks like the wrong thing to do here. vm_mmap() thinks it=92s = requesting 0x0 through 0x1fa000, but now the request just silently got = changed to 0x1000 through 0x1fa000. So what the ibcs2 module thinks .text is being loaded at 0, ends up = being loaded at 0x1000, and is 0x1000 bytes too small. It then jumps to = the wrong starting address, and the process crashes.=20 Also to clarify my original posting, COFF itself isn=92t the issue here, = just that this specific binary wants its .text section to begin at a = virtual address below 0x1000.=20 =97 Kevin --Apple-Mail=_540E5AEC-C04D-4A1D-8577-276AD9832813 Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIPLzCCBN0w ggPFoAMCAQICEHGS++YZX6xNEoV0cTSiGKcwDQYJKoZIhvcNAQEFBQAwezELMAkGA1UEBhMCR0Ix GzAZBgNVBAgMEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBwwHU2FsZm9yZDEaMBgGA1UECgwR Q29tb2RvIENBIExpbWl0ZWQxITAfBgNVBAMMGEFBQSBDZXJ0aWZpY2F0ZSBTZXJ2aWNlczAeFw0w NDAxMDEwMDAwMDBaFw0yODEyMzEyMzU5NTlaMIGuMQswCQYDVQQGEwJVUzELMAkGA1UECBMCVVQx FzAVBgNVBAcTDlNhbHQgTGFrZSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRSVVNUIE5ldHdvcmsx ITAfBgNVBAsTGGh0dHA6Ly93d3cudXNlcnRydXN0LmNvbTE2MDQGA1UEAxMtVVROLVVTRVJGaXJz dC1DbGllbnQgQXV0aGVudGljYXRpb24gYW5kIEVtYWlsMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A MIIBCgKCAQEAsjmFpPJ9q0E7YkY3rs3BYHW8OWX5ShpHornMSMxqmNVNNRm5pELlzkniii8efNIx B8dOtINknS4p1aJkxIW9hVE1eaROaJB7HHqkkqgX8pgV8pPMyaQylbsMTzC9mKALi+VuG6JG+ni8 om+rWV6lL8/K2m2qL+usobNqqrcuZzWLeeEeaYji5kbNoKXqvgvOdjp6Dpvq/NonWz1zHyLmSGHG TPNpsaguG7bUMSAsvIKKjqQOpdeJQ/wWWq8dcdcRWdq6hw2v+vPhwvCkxWeM1tZUOt4KpLoDd7Nl yP0e03RiqhjKaJMeoYV+9Udly/hNVyh00jT/MLbu9mIwFIws6wIDAQABo4IBJzCCASMwHwYDVR0j BBgwFoAUoBEKIz6W8Qfs4q8p74Klf9AwpLQwHQYDVR0OBBYEFImCZ33EnSZwAEu0UEh83j2uBG59 MA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8EBTADAQH/MB0GA1UdJQQWMBQGCCsGAQUFBwMCBggr BgEFBQcDBDARBgNVHSAECjAIMAYGBFUdIAAwewYDVR0fBHQwcjA4oDagNIYyaHR0cDovL2NybC5j b21vZG9jYS5jb20vQUFBQ2VydGlmaWNhdGVTZXJ2aWNlcy5jcmwwNqA0oDKGMGh0dHA6Ly9jcmwu Y29tb2RvLm5ldC9BQUFDZXJ0aWZpY2F0ZVNlcnZpY2VzLmNybDARBglghkgBhvhCAQEEBAMCAQYw DQYJKoZIhvcNAQEFBQADggEBAJ2Vyzy4fqUJxB6/C8LHdo45PJTGEKpPDMngq4RdiVTgZTvzbRx8 NywlVF+WIfw3hJGdFdwUT4HPVB1rbEVgxy35l1FM+WbKPKCCjKbI8OLp1Er57D9Wyd12jMOCAU9s APMeGmF0BEcDqcZAV5G8ZSLFJ2dPV9tkWtmNH7qGL/QGrpxp7en0zykX2OBKnxogL5dMUbtGB8SK N04g4wkxaMeexIud6H4RvDJoEJYRmETYKlFgTYjrdDrfQwYyyDlWjDoRUtNBpEMD9O3vMyfbOeAU TibJ2PU54om4k123KSZB6rObroP8d3XK6Mq1/uJlSmM+RMTQw16Hc6mYHK9/FX8wggUaMIIEAqAD AgECAhBtGeqnGU9qMyLmIjJ6qnHeMA0GCSqGSIb3DQEBBQUAMIGuMQswCQYDVQQGEwJVUzELMAkG A1UECBMCVVQxFzAVBgNVBAcTDlNhbHQgTGFrZSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRSVVNU IE5ldHdvcmsxITAfBgNVBAsTGGh0dHA6Ly93d3cudXNlcnRydXN0LmNvbTE2MDQGA1UEAxMtVVRO LVVTRVJGaXJzdC1DbGllbnQgQXV0aGVudGljYXRpb24gYW5kIEVtYWlsMB4XDTExMDQyODAwMDAw MFoXDTIwMDUzMDEwNDgzOFowgZMxCzAJBgNVBAYTAkdCMRswGQYDVQQIExJHcmVhdGVyIE1hbmNo ZXN0ZXIxEDAOBgNVBAcTB1NhbGZvcmQxGjAYBgNVBAoTEUNPTU9ETyBDQSBMaW1pdGVkMTkwNwYD VQQDEzBDT01PRE8gQ2xpZW50IEF1dGhlbnRpY2F0aW9uIGFuZCBTZWN1cmUgRW1haWwgQ0EwggEi MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCShIRbS1eY1F4vi6ThQMijU1hfZmXxMk73nzJ9 VdB4TFW3QpTg+SdxB8XGaaS5MsTxQBqQzCdWYn8XtXFpruUgG+TLY15gyqJB9mrho/+43x9IbWVD jCouK2M4d9+xF6zC2oIC1tQyatRnbyATj1w1+uVUgK/YcQodNwoCUFNslR2pEBS0mZVZEjH/CaLS TNxS297iQAFbSGjdxUq04O0kHzqvcV8H46y/FDuwJXFoPfQP1hdYRhWBPGiLi4MPbXohV+Y0sNsy fuNK4aVScmQmkU6lkg//4LFg/RpvaFGZY40ai6XMQpubfSJj06mg/M6ekN9EGfRcWzW6FvOnm//B AgMBAAGjggFLMIIBRzAfBgNVHSMEGDAWgBSJgmd9xJ0mcABLtFBIfN49rgRufTAdBgNVHQ4EFgQU ehNOAHRbxnhjZCfBL+KgW7x5xXswDgYDVR0PAQH/BAQDAgEGMBIGA1UdEwEB/wQIMAYBAf8CAQAw EQYDVR0gBAowCDAGBgRVHSAAMFgGA1UdHwRRME8wTaBLoEmGR2h0dHA6Ly9jcmwudXNlcnRydXN0 LmNvbS9VVE4tVVNFUkZpcnN0LUNsaWVudEF1dGhlbnRpY2F0aW9uYW5kRW1haWwuY3JsMHQGCCsG AQUFBwEBBGgwZjA9BggrBgEFBQcwAoYxaHR0cDovL2NydC51c2VydHJ1c3QuY29tL1VUTkFkZFRy dXN0Q2xpZW50X0NBLmNydDAlBggrBgEFBQcwAYYZaHR0cDovL29jc3AudXNlcnRydXN0LmNvbTAN BgkqhkiG9w0BAQUFAAOCAQEAhda+eFdVbTN/RFL+QtUGqAEDgIr7DbL9Sr/2r0FJ9RtaxdKtG3Nu PukmfOZMmMEwKN/L+0I8oSU+CnXW0D05hmbRoZu1TZtvryhsHa/l6nRaqNqxwPF1ei+eupN5yv7i kR5WdLL4jdPgQ3Ib7Y/9YDkgR/uLrzplSDyYPaUlv73vYOBJ5RbI6z9Dg/Dg7g3B080zX5vQvWBq szv++tTJOjwf7Zv/m0kzvkIpOYPuM2kugp1FTahp2oAbHj3SGl18R5mlmwhtEpmG1l1XBxunML5L SUS4kH7K0Xk467Qz+qA6XSZYnmFVGLQh1ZnV4ENAQjC+6qXnlNKw/vN1+X9u5zCCBSwwggQUoAMC AQICEQDbETdDYf7wYKjx8ymk38yAMA0GCSqGSIb3DQEBBQUAMIGTMQswCQYDVQQGEwJHQjEbMBkG A1UECBMSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRowGAYDVQQKExFDT01P RE8gQ0EgTGltaXRlZDE5MDcGA1UEAxMwQ09NT0RPIENsaWVudCBBdXRoZW50aWNhdGlvbiBhbmQg U2VjdXJlIEVtYWlsIENBMB4XDTEzMDYxNjAwMDAwMFoXDTE0MDYxNjIzNTk1OVowJjEkMCIGCSqG SIb3DQEJARYVdG9hc3R5QGRyYWdvbmRhdGEuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB CgKCAQEAvoIO+cLWLe7YYAGV/WdoWC85K8uIgstlYMg/bC8eGbC7AY/nuQXpRV5+xlTXgN7qry/m 6XErlaw1U3rmwlNyjMhJdYaPZclywBKKpYnc3sp0q2A6naeVmOF/t4QDImtfc3sV7SaEkIr7zssK MFTnkOX57g1r3MuiYoHBx1cMaWXYCJ5LDzsynwHGAExYuziRzXcu4sRZ1HBJlQ8hM3yhTTGGOQv1 H1ky13a1RxXC+uoTtYFyrxdBgPUd4eGF1tILHtK9NXnU6lhey90wDa2jmQOJQErgYuYPZriSuBXz QobK7tGcjMBgBQ1U+gxaTyThbXgxfb1MTjDx46hSl8Z35wIDAQABo4IB5TCCAeEwHwYDVR0jBBgw FoAUehNOAHRbxnhjZCfBL+KgW7x5xXswHQYDVR0OBBYEFO9wHp89I1B980w64KR38bmtuHFYMA4G A1UdDwEB/wQEAwIFoDAMBgNVHRMBAf8EAjAAMCAGA1UdJQQZMBcGCCsGAQUFBwMEBgsrBgEEAbIx AQMFAjARBglghkgBhvhCAQEEBAMCBSAwRgYDVR0gBD8wPTA7BgwrBgEEAbIxAQIBAQEwKzApBggr BgEFBQcCARYdaHR0cHM6Ly9zZWN1cmUuY29tb2RvLm5ldC9DUFMwVwYDVR0fBFAwTjBMoEqgSIZG aHR0cDovL2NybC5jb21vZG9jYS5jb20vQ09NT0RPQ2xpZW50QXV0aGVudGljYXRpb25hbmRTZWN1 cmVFbWFpbENBLmNybDCBiAYIKwYBBQUHAQEEfDB6MFIGCCsGAQUFBzAChkZodHRwOi8vY3J0LmNv bW9kb2NhLmNvbS9DT01PRE9DbGllbnRBdXRoZW50aWNhdGlvbmFuZFNlY3VyZUVtYWlsQ0EuY3J0 MCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5jb21vZG9jYS5jb20wIAYDVR0RBBkwF4EVdG9hc3R5 QGRyYWdvbmRhdGEuY29tMA0GCSqGSIb3DQEBBQUAA4IBAQCBaQ8dcaprzzREiMtsc2UtOPSHFiCy dcd5OjE6BN+pkcQozhx3nol9dFKJ+YfGvIxIjHmDGFTOgJgJvjRZ0D1Hw2WJCEtyD+U6yi/cnDFu Ksl039qafzbah6ft2r+GM0QufuFmrBi/bTWU3lGuhL8TKOvsWeLFkyGqtv9AJz2vg7j7dpxutLQY NWnrt7nS2x6p4f1LXu3iwczefyNNFUYwE9zXAT0Uwn48g2iijuf9vekfpqtHBmfSu0tSfd3FS3JC hmFp1fMxnWOnuZ529HFtGeYzr1K8Tp+JEVPjzPCxymVFsZ945Vzj0kc0DT3f9N5Gdw6uybrUwupM NHJJCB9VMYIDrjCCA6oCAQEwgakwgZMxCzAJBgNVBAYTAkdCMRswGQYDVQQIExJHcmVhdGVyIE1h bmNoZXN0ZXIxEDAOBgNVBAcTB1NhbGZvcmQxGjAYBgNVBAoTEUNPTU9ETyBDQSBMaW1pdGVkMTkw NwYDVQQDEzBDT01PRE8gQ2xpZW50IEF1dGhlbnRpY2F0aW9uIGFuZCBTZWN1cmUgRW1haWwgQ0EC EQDbETdDYf7wYKjx8ymk38yAMAkGBSsOAwIaBQCgggHZMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0B BwEwHAYJKoZIhvcNAQkFMQ8XDTEzMTAyNDA2Mzc1MlowIwYJKoZIhvcNAQkEMRYEFEqIfdDFZTC/ k0is9fRIRSdaZOBVMIG6BgkrBgEEAYI3EAQxgawwgakwgZMxCzAJBgNVBAYTAkdCMRswGQYDVQQI ExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNVBAcTB1NhbGZvcmQxGjAYBgNVBAoTEUNPTU9ETyBD QSBMaW1pdGVkMTkwNwYDVQQDEzBDT01PRE8gQ2xpZW50IEF1dGhlbnRpY2F0aW9uIGFuZCBTZWN1 cmUgRW1haWwgQ0ECEQDbETdDYf7wYKjx8ymk38yAMIG8BgsqhkiG9w0BCRACCzGBrKCBqTCBkzEL MAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9y ZDEaMBgGA1UEChMRQ09NT0RPIENBIExpbWl0ZWQxOTA3BgNVBAMTMENPTU9ETyBDbGllbnQgQXV0 aGVudGljYXRpb24gYW5kIFNlY3VyZSBFbWFpbCBDQQIRANsRN0Nh/vBgqPHzKaTfzIAwDQYJKoZI hvcNAQEBBQAEggEAS7PsuujTPlmJjn37R60KW68DGdg80gNgmhB004+d9F2+fSUET5LqqNLcBPWU /qns76CpkQ9spbJMaLwsIb9QgWZfx5bWFgdjV5fZlWhdYY/uKACHjMirjBlqBaSEhfMIosLcrpJY 12eJo71nIz4q//0rhjeqTxwvhlfqdRWzqyjnpD4pBbwJO1eg5+B+IYZqE3FeR9cTavU2Q21r2hFd uLPfb98924P3ySXGwM8I9mSLzEW65VJRy77FBqhuKFkJIEFABeKa6u3o3VeHXje0qrpNkR8fzJR9 9c7bckaIQHg2SjC5WR0ZnvVHyNaZG2klH9egZJ1hoqwonADX5bokLQAAAAAAAA== --Apple-Mail=_540E5AEC-C04D-4A1D-8577-276AD9832813-- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 07:05:01 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E41C7293; Thu, 24 Oct 2013 07:05:01 +0000 (UTC) (envelope-from shrikanth07@gmail.com) Received: from mail-ve0-x22d.google.com (mail-ve0-x22d.google.com [IPv6:2607:f8b0:400c:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 971B82EF1; Thu, 24 Oct 2013 07:05:01 +0000 (UTC) Received: by mail-ve0-f173.google.com with SMTP id jw12so1103170veb.18 for ; Thu, 24 Oct 2013 00:05:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=bMqm/z9FR+HfH2VkKSR9IxJL7CawKYPiRbKKEbFYvTg=; b=XGg/t0zC1ZlrJKGMiNCIimJ/055rIUWa6b5wAMSTz0bWXlTCRxNLNCXzXWq8/xfUsD de09ICf9++L1zqJImA2FaR6f3f594KHL3HxR8KHazK/h80WcpU8LdjWLGsaeo1B69MFU sn8esmdLvc/NHRxC7EtI1aqRGs88mSpdTCxlWO7cKziolIiYVGv7H5iu3tMjsBILirol eQTpxYTJe/x0zVUriOaGidbHsVV4ZAz/Ud85FFjvTFsMCBUVoaG026/V0m3iu505kFly Q/oTUvluhnC2AwQ7VdRJ4RHK1jT9NmHqPXV1qJKFQ4ivkxh3LqBfNO6TA3vzxZJg6ori v6yw== MIME-Version: 1.0 X-Received: by 10.52.230.102 with SMTP id sx6mr543021vdc.15.1382598300695; Thu, 24 Oct 2013 00:05:00 -0700 (PDT) Received: by 10.58.133.131 with HTTP; Thu, 24 Oct 2013 00:05:00 -0700 (PDT) Date: Thu, 24 Oct 2013 00:05:00 -0700 Message-ID: Subject: FreeBSD Machine check architecture and memory controller hub chipset From: Shrikanth Kamath To: freebsd-hackers@freebsd.org, freebsd-i386@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 07:05:02 -0000 The mca_check_status function checks the status of the Intel MCA registers to generate the mca records. When reporting compound error codes which indicate memory errors, does the memory controller hub chipset (say e.g Intel E7320 MCH is the participating memory controller) registers need to be set to assist the processor to generate the compound error codes corresponding to errors from memory controller? I want to be able to extend the mca_scan function to scan the memory controller hub chipset registers if this compound error code were to be generated. Reference, section 15.9.2 in Intel 64 and IA-32 architectures software developers manual (Vol 3). -- Shrikanth R K From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 07:48:35 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id EE7AF179 for ; Thu, 24 Oct 2013 07:48:35 +0000 (UTC) (envelope-from satan@ukr.net) Received: from hell.ukr.net (hell.ukr.net [212.42.67.68]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AB47021AE for ; Thu, 24 Oct 2013 07:48:35 +0000 (UTC) Received: from satan by hell.ukr.net with local ID 1VZFec-000Dd9-9r ; Thu, 24 Oct 2013 10:48:26 +0300 Date: Thu, 24 Oct 2013 10:48:26 +0300 From: Vitalij Satanivskij To: freebsd-hackers@freebsd.org Subject: FreeBSD 10.0-BETA1 #8 r256765M spend too much time in locks Message-ID: <20131024074826.GA50853@hell.ukr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.22 (2013-10-16) Cc: satan@ukr.net X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 07:48:36 -0000 Hello. After upgrading system from old current (r245701) to fresh current r255173 (than switch to stable/10 r256765M) found some strange system behavior: Diff from r256765M anf r256765 is Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 256765) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy) @@ -5147,7 +5147,7 @@ len = l2hdr->b_asize; cdata = zio_data_buf_alloc(len); csize = zio_compress_data(ZIO_COMPRESS_LZ4, l2hdr->b_tmp_cdata, - cdata, l2hdr->b_asize, (size_t)SPA_MINBLOCKSIZE); + cdata, l2hdr->b_asize, (size_t)(1ULL << l2hdr->b_dev->l2ad_vdev->vdev_ashift)); if (csize == 0) { /* zero block, indicate that there's nothing to write */ But same situation was befor patch. System load to high CPU: 6.8% user, 0.0% nice, 57.3% system, 0.8% interrupt, 35.1% idle hotkernel (dtrace script) says kernel`__mtx_unlock_flags 292 0.3% kernel`__mtx_lock_flags 315 0.3% zfs.ko`lzjb_compress 349 0.3% kernel`__rw_wlock_hard 686 0.7% kernel`spinlock_exit 1050 1.0% kernel`vmem_xalloc 7516 7.3% kernel`_sx_xlock_hard 8664 8.5% kernel`acpi_cpu_c1 9737 9.5% kernel`cpu_idle 20189 19.7% kernel`__mtx_lock_sleep 45952 44.9% Trying to understand where is a problem I'm build kernel with lock profiling. and get some data (for one minute ) (file attached) using agregation find most lock's is in 14,159818 /usr/src/sys/kern/subr_vmem.c:1128(sleep mutex:kmem arena) 9,553523 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1597(sx:buf_hash_table.ht_locks[i].ht_lock) 8,386943 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3541(sx:l2arc_buflist_mtx) 8,110206 /usr/src/sys/kern/subr_vmem.c:1230(sleep mutex:kmem arena) 5,909429 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mru->arcs_locks[i].arcs_lock) 5,452206 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mfu->arcs_locks[i].arcs_lock) 5,050224 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c:303(sx:tx->tx_cpu[c].tc_open_lock) 4,232819 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:891(sx:buf_hash_table.ht_locks[i].ht_lock) 4,211348 /usr/src/sys/kern/vfs_subr.c:2101(lockmgr:zfs) 4,011656 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:862(sx:buf_hash_table.ht_locks[i].ht_lock) 3,823698 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2009(sx:arc_eviction_mtx) 2,697344 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:126(sx:h->hash_mutexes[i]) 2,343242 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mfu->arcs_locks[i].arcs_lock) 1,752713 /usr/src/sys/kern/vfs_lookup.c:707(lockmgr:zfs) 1,580856 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1136(sx:zfsvfs->z_hold_mtx[i]) 1,534242 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1291(sx:arc_mfu->arcs_locks[i].arcs_lock) 1,331583 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:129(sx:db->db_mtx) 1,105058 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:427(sx:vq->vq_lock) 1,080855 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:396(sx:vq->vq_lock) 0,858568 /usr/src/sys/kern/vfs_cache.c:488(rw:Name Cache) 0,831652 /usr/src/sys/vm/vm_kern.c:344(rw:kmem vm object) 0,815439 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1845(sx:buf_hash_table.ht_locks[i].ht_lock) 0,812613 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mru->arcs_locks[i].arcs_lock) 0,794274 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1529(lockmgr:zfs) 0,669845 /usr/src/sys/vm/uma_core.c:2097(sleep mutex:zio_cache) Short system description CPU E5-1650 MEM 128Gb ddr3-1600 Storage subsystem 36x1Tb WD RE4 drives on LSI SAS2308 Controler 3x180Gb Intel ssd 530 series as l2 cache POOL is 18 mirrors, two drives in ich mirror and 3 cache devices eg. .... mirror-14 ONLINE 0 0 0 gpt/disk28 ONLINE 0 0 0 gpt/disk29 ONLINE 0 0 0 mirror-15 ONLINE 0 0 0 gpt/disk30 ONLINE 0 0 0 gpt/disk31 ONLINE 0 0 0 mirror-16 ONLINE 0 0 0 gpt/disk32 ONLINE 0 0 0 gpt/disk33 ONLINE 0 0 0 mirror-17 ONLINE 0 0 0 gpt/disk34 ONLINE 0 0 0 gpt/disk35 ONLINE 0 0 0 cache ada1 ONLINE 0 0 0 ada2 ONLINE 0 0 0 ada3 ONLINE 0 0 0 POOL have two ZFS main with options (diffs from default) - compression lz4 secondarycache all sync disabled Data size for it around 6Tb (compresed) eg disk1 refcompressratio 1.56x - disk1 written 5,99T - disk1 logicalused 10,8T - disk1 logicalreferenced 9,32T - and another with options recordsize 4K, before it was 32k, but internal software use data blocks mostly 4k so we try to change (without real data realocate ) compresion -s off sync disabled secondarycache all DATA size on it around 1.4Tb ARC configured to use max 80Gb top most time looks like this Mem: 14G Active, 13G Inact, 94G Wired, 497M Cache, 3297M Buf, 2214M Free ARC: 80G Total, 2010M MFU, 70G MRU, 49M Anon, 3822M Header, 4288M Other LA's - around 10-20 depend on day time. zpool iostat disk1 1 capacity operations bandwidth pool alloc free read write read write ---------- ----- ----- ----- ----- ----- ----- disk1 7,45T 8,86T 546 1,49K 16,4M 13,4M disk1 7,45T 8,86T 272 3,91K 11,7M 27,4M disk1 7,45T 8,86T 344 2,94K 7,26M 25,2M disk1 7,45T 8,86T 224 2,02K 9,91M 21,8M disk1 7,45T 8,86T 244 2,35K 8,23M 18,4M disk1 7,45T 8,86T 245 2,54K 6,44M 23,4M disk1 7,45T 8,86T 114 2,94K 3,28M 13,3M disk1 7,45T 8,86T 288 4,43K 6,09M 33,5M disk1 7,45T 8,86T 157 1,26K 2,98M 15,7M disk1 7,45T 8,86T 94 842 3,07M 13,6M disk1 7,45T 8,86T 327 1,71K 9,63M 8,21M disk1 7,45T 8,86T 237 1,81K 5,73M 29,3M disk1 7,45T 8,86T 247 3,47K 5,17M 29,6M disk1 7,45T 8,86T 165 2,37K 3,22M 16,7M disk1 7,45T 8,86T 155 3,23K 3,27M 23,9M Strange as timeout seted up to 10sec's. What intresting - after reboot system work fine for some time, at last while ARC not become 80G size. Low limit for arc is 40gb, strange but old system can take memory from arc eg like this Mem: 32G Active, 8797M Inact, 78G Wired, 2370M Cache, 209M Buf, 3977M Free ARC: 43G Total, 2204M MFU, 28G MRU, 135M Anon, 7898M Header, 5301M Other On new ARC getting to it max allowed size. So for now question is, what it can be, what we can try to improve system perfomance and so on? From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 07:50:25 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E6D55309 for ; Thu, 24 Oct 2013 07:50:25 +0000 (UTC) (envelope-from satan@ukr.net) Received: from hell.ukr.net (hell.ukr.net [212.42.67.68]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A1D2921FA for ; Thu, 24 Oct 2013 07:50:25 +0000 (UTC) Received: from satan by hell.ukr.net with local ID 1VZFgV-000Df7-Sg ; Thu, 24 Oct 2013 10:50:23 +0300 Date: Thu, 24 Oct 2013 10:50:23 +0300 From: Vitalij Satanivskij To: Vitalij Satanivskij Subject: Re: FreeBSD 10.0-BETA1 #8 r256765M spend too much time in locks Message-ID: <20131024075023.GA52443@hell.ukr.net> References: <20131024074826.GA50853@hell.ukr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131024074826.GA50853@hell.ukr.net> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 07:50:26 -0000 EEE forget to notice - old system have none compression on main zfs Vitalij Satanivskij wrote: VS> Hello. VS> VS> After upgrading system from old current (r245701) to fresh current r255173 (than switch to stable/10 r256765M) VS> found some strange system behavior: VS> VS> Diff from r256765M anf r256765 is VS> Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c VS> =================================================================== VS> --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 256765) VS> +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy) VS> @@ -5147,7 +5147,7 @@ VS> len = l2hdr->b_asize; VS> cdata = zio_data_buf_alloc(len); VS> csize = zio_compress_data(ZIO_COMPRESS_LZ4, l2hdr->b_tmp_cdata, VS> - cdata, l2hdr->b_asize, (size_t)SPA_MINBLOCKSIZE); VS> + cdata, l2hdr->b_asize, (size_t)(1ULL << l2hdr->b_dev->l2ad_vdev->vdev_ashift)); VS> VS> if (csize == 0) { VS> /* zero block, indicate that there's nothing to write */ VS> VS> VS> But same situation was befor patch. VS> VS> VS> System load to high VS> CPU: 6.8% user, 0.0% nice, 57.3% system, 0.8% interrupt, 35.1% idle VS> VS> hotkernel (dtrace script) says VS> VS> kernel`__mtx_unlock_flags 292 0.3% VS> kernel`__mtx_lock_flags 315 0.3% VS> zfs.ko`lzjb_compress 349 0.3% VS> kernel`__rw_wlock_hard 686 0.7% VS> kernel`spinlock_exit 1050 1.0% VS> kernel`vmem_xalloc 7516 7.3% VS> kernel`_sx_xlock_hard 8664 8.5% VS> kernel`acpi_cpu_c1 9737 9.5% VS> kernel`cpu_idle 20189 19.7% VS> kernel`__mtx_lock_sleep 45952 44.9% VS> VS> VS> VS> Trying to understand where is a problem I'm build kernel with lock profiling. VS> VS> and get some data (for one minute ) VS> VS> (file attached) VS> VS> using agregation find most lock's is in VS> VS> 14,159818 /usr/src/sys/kern/subr_vmem.c:1128(sleep mutex:kmem arena) VS> 9,553523 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1597(sx:buf_hash_table.ht_locks[i].ht_lock) VS> 8,386943 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3541(sx:l2arc_buflist_mtx) VS> 8,110206 /usr/src/sys/kern/subr_vmem.c:1230(sleep mutex:kmem arena) VS> 5,909429 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mru->arcs_locks[i].arcs_lock) VS> 5,452206 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mfu->arcs_locks[i].arcs_lock) VS> 5,050224 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c:303(sx:tx->tx_cpu[c].tc_open_lock) VS> 4,232819 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:891(sx:buf_hash_table.ht_locks[i].ht_lock) VS> 4,211348 /usr/src/sys/kern/vfs_subr.c:2101(lockmgr:zfs) VS> 4,011656 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:862(sx:buf_hash_table.ht_locks[i].ht_lock) VS> 3,823698 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2009(sx:arc_eviction_mtx) VS> 2,697344 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:126(sx:h->hash_mutexes[i]) VS> 2,343242 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mfu->arcs_locks[i].arcs_lock) VS> 1,752713 /usr/src/sys/kern/vfs_lookup.c:707(lockmgr:zfs) VS> 1,580856 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1136(sx:zfsvfs->z_hold_mtx[i]) VS> 1,534242 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1291(sx:arc_mfu->arcs_locks[i].arcs_lock) VS> 1,331583 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:129(sx:db->db_mtx) VS> 1,105058 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:427(sx:vq->vq_lock) VS> 1,080855 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:396(sx:vq->vq_lock) VS> 0,858568 /usr/src/sys/kern/vfs_cache.c:488(rw:Name Cache) VS> 0,831652 /usr/src/sys/vm/vm_kern.c:344(rw:kmem vm object) VS> 0,815439 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1845(sx:buf_hash_table.ht_locks[i].ht_lock) VS> 0,812613 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mru->arcs_locks[i].arcs_lock) VS> 0,794274 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1529(lockmgr:zfs) VS> 0,669845 /usr/src/sys/vm/uma_core.c:2097(sleep mutex:zio_cache) VS> VS> VS> VS> VS> Short system description VS> CPU E5-1650 VS> MEM 128Gb ddr3-1600 VS> VS> Storage subsystem VS> VS> 36x1Tb WD RE4 drives on LSI SAS2308 Controler VS> 3x180Gb Intel ssd 530 series as l2 cache VS> VS> VS> POOL is 18 mirrors, two drives in ich mirror and 3 cache devices VS> VS> eg. VS> .... VS> mirror-14 ONLINE 0 0 0 VS> gpt/disk28 ONLINE 0 0 0 VS> gpt/disk29 ONLINE 0 0 0 VS> mirror-15 ONLINE 0 0 0 VS> gpt/disk30 ONLINE 0 0 0 VS> gpt/disk31 ONLINE 0 0 0 VS> mirror-16 ONLINE 0 0 0 VS> gpt/disk32 ONLINE 0 0 0 VS> gpt/disk33 ONLINE 0 0 0 VS> mirror-17 ONLINE 0 0 0 VS> gpt/disk34 ONLINE 0 0 0 VS> gpt/disk35 ONLINE 0 0 0 VS> cache VS> ada1 ONLINE 0 0 0 VS> ada2 ONLINE 0 0 0 VS> ada3 ONLINE 0 0 0 VS> VS> VS> POOL have two ZFS VS> VS> main with options (diffs from default) - VS> compression lz4 VS> secondarycache all VS> sync disabled VS> VS> Data size for it around 6Tb (compresed) eg VS> disk1 refcompressratio 1.56x - VS> disk1 written 5,99T - VS> disk1 logicalused 10,8T - VS> disk1 logicalreferenced 9,32T - VS> VS> VS> and another with options VS> recordsize 4K, before it was 32k, but internal software use data blocks mostly 4k so we try to change (without real data realocate ) VS> compresion -s off VS> sync disabled VS> secondarycache all VS> VS> DATA size on it around 1.4Tb VS> VS> ARC configured to use max 80Gb VS> VS> top most time looks like this VS> VS> Mem: 14G Active, 13G Inact, 94G Wired, 497M Cache, 3297M Buf, 2214M Free VS> ARC: 80G Total, 2010M MFU, 70G MRU, 49M Anon, 3822M Header, 4288M Other VS> VS> VS> LA's - around 10-20 depend on day time. VS> VS> VS> zpool iostat disk1 1 VS> capacity operations bandwidth VS> pool alloc free read write read write VS> ---------- ----- ----- ----- ----- ----- ----- VS> disk1 7,45T 8,86T 546 1,49K 16,4M 13,4M VS> disk1 7,45T 8,86T 272 3,91K 11,7M 27,4M VS> disk1 7,45T 8,86T 344 2,94K 7,26M 25,2M VS> disk1 7,45T 8,86T 224 2,02K 9,91M 21,8M VS> disk1 7,45T 8,86T 244 2,35K 8,23M 18,4M VS> disk1 7,45T 8,86T 245 2,54K 6,44M 23,4M VS> disk1 7,45T 8,86T 114 2,94K 3,28M 13,3M VS> disk1 7,45T 8,86T 288 4,43K 6,09M 33,5M VS> disk1 7,45T 8,86T 157 1,26K 2,98M 15,7M VS> disk1 7,45T 8,86T 94 842 3,07M 13,6M VS> disk1 7,45T 8,86T 327 1,71K 9,63M 8,21M VS> disk1 7,45T 8,86T 237 1,81K 5,73M 29,3M VS> disk1 7,45T 8,86T 247 3,47K 5,17M 29,6M VS> disk1 7,45T 8,86T 165 2,37K 3,22M 16,7M VS> disk1 7,45T 8,86T 155 3,23K 3,27M 23,9M VS> VS> Strange as timeout seted up to 10sec's. VS> VS> What intresting - after reboot system work fine for some time, at last while ARC not become 80G size. VS> Low limit for arc is 40gb, strange but old system can take memory from arc eg like this VS> VS> VS> Mem: 32G Active, 8797M Inact, 78G Wired, 2370M Cache, 209M Buf, 3977M Free VS> ARC: 43G Total, 2204M MFU, 28G MRU, 135M Anon, 7898M Header, 5301M Other VS> VS> On new ARC getting to it max allowed size. VS> VS> So for now question is, what it can be, what we can try to improve system perfomance and so on? VS> VS> VS> VS> _______________________________________________ VS> freebsd-hackers@freebsd.org mailing list VS> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers VS> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 07:53:50 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DAF804EF for ; Thu, 24 Oct 2013 07:53:49 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 79F4F223A for ; Thu, 24 Oct 2013 07:53:49 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id r9O6rNNK052461; Thu, 24 Oct 2013 09:53:23 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua r9O6rNNK052461 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id r9O6rN8l052460; Thu, 24 Oct 2013 09:53:23 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 24 Oct 2013 09:53:23 +0300 From: Konstantin Belousov To: d@delphij.net Subject: Re: FoxPro on FreeBSD Message-ID: <20131024065323.GA10625@kib.kiev.ua> References: <52687ED8.6080309@mindslayer.net> <9B89077C-6BE7-49F1-9F22-19FAD9F6C3ED@dragondata.com> <5268B62B.3000104@delphij.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="5mCyUwZo2JvN/JJP" Content-Disposition: inline In-Reply-To: <5268B62B.3000104@delphij.net> User-Agent: Mutt/1.5.22 (2013-10-16) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: Puppet Master , "freebsd-hackers@freebsd.org Hackers" , Kevin Day X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 07:53:50 -0000 --5mCyUwZo2JvN/JJP Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Oct 23, 2013 at 10:54:51PM -0700, Xin Li wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 >=20 > On 10/23/13, 8:32 PM, Kevin Day wrote: > > I did some debugging, and watched how the process was getting > > launched, and I've managed to get it to load! > >=20 > > The problem was that COFF files expect to be mapped into memory at > > address 0, something that processes are no longer allowed to do. > >=20 > > Run "sysctl security.bsd.map_at_zero=3D1? or add > > ?security.bsd.map_at_zero=3D1? to /etc/sysctl.conf and you should > > have it working. We probably should either make an exception for > > COFF files to bypass this the sysctl restriction, or at least print > > a more helpful error than just letting the process segfault because > > it didn?t get mapped where it was supposed to go. >=20 > Wow, this is impressive find, indeed! Do they need to do the map at > startup only, or do they want to explicitly map something at address 0 > during runtime? How is this impressive ? Several variations of a.out and COFF binaries are specified to start mapping of the text at 0. This was one of the reasons why I argued for sysctl to enable/disable mmap at zero by sysctl instead of abruptly disallowing it, when corresponding 'security' feature was discussed. I documented the tuning needed to run a.out binaries in aout(4), there are much more for it than just map at zero. If somebody wants to provide similar hints for the COFF images, I cannot object. --5mCyUwZo2JvN/JJP Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJSaMPiAAoJEJDCuSvBvK1BclYP/iSVQHMQEkwlBQs2jopoMKhj YZliOL2nz+lO/D6wfePkTOUbv1dmEhl/2tWl1bCpk9yN6KJuDOz2B6r6DVHA1c/j /QjHB/OtkTOxQjw2lGG1j4eciG+TlDMUvtgRVmSr2g+nc7OLZmcjRZJdaR8LT/zv 2QMC/npZe3iffjKCK2j3LkT0Tn61/EiN3YpKOADyRQXQXHZ1x/fMvLFS9elOWHhm Q5Xj9NV7OIeKieaHcnoVx6Im8mLa/UM7KdfhYcyVEyhURN5cAqUNqptlaTEgVchx G32P8vCNwKK6IXGLEvtRGLVAIPkkVMtjkXHq3aiGBnHRgY59GNMyt6HhiRfH3Eqm qGT4rlNGMjrwGL9TT4a93BATXR7P/odq79uId6b4nE8dxNMMxvN6NR4ZYE5afcP6 BFOQn9siL1h0s/SY1zN4+/O5Qlpuw4mQsgc4APLbUFkvFGKSW8ra95gJVtGComcX LkAg1XrOwUDLBhzUiHGw83wcgKElAfgCt3fdaLdLxCnb/JU86zhKgZS360kBvJkS e1LGOTHu+eWugRSDlKcdwxR+7tBFCEY5+rHruvDdULnZ90oulc2LZh9amwiw4dEP 6lGjdiARPlyWaycBWByGq+lsKo4nm2nm50WNDMKCecSck5H0qui3oR6pCn6Uvjme c89AfTwpLylOjeVE326x =WRUj -----END PGP SIGNATURE----- --5mCyUwZo2JvN/JJP-- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 07:59:36 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 72F4787C for ; Thu, 24 Oct 2013 07:59:36 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from ibiza.webweaving.org (ibiza.webweaving.org [204.109.56.32]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 197C622D2 for ; Thu, 24 Oct 2013 07:59:35 +0000 (UTC) Received: from pikmeer.webweaving.org (pikmeer.webweaving.org [178.18.23.51]) by ibiza.webweaving.org (8.14.7/8.14.7) with ESMTP id r9O7xJwu011696; Thu, 24 Oct 2013 07:59:19 GMT (envelope-from dirkx@webweaving.org) Received: from [192.168.189.36] (nat.net.ipw-berlin.de [217.89.118.250]) (authenticated bits=0) by pikmeer.webweaving.org (8.14.7/8.14.7) with ESMTP id r9O7xHhQ037759 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Thu, 24 Oct 2013 07:59:19 GMT (envelope-from dirkx@webweaving.org) X-Authentication-Warning: pikmeer.webweaving.org: Host nat.net.ipw-berlin.de [217.89.118.250] claimed to be [192.168.189.36] Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Subject: Re: FoxPro on FreeBSD From: Dirk-Willem van Gulik In-Reply-To: <8A799DDB-3D5C-4418-B064-A2B7821EE0F2@dragondata.com> Date: Thu, 24 Oct 2013 09:59:26 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <36B00CF4-E8C5-4339-8A35-148F5E707353@webweaving.org> References: <52687ED8.6080309@mindslayer.net> <9B89077C-6BE7-49F1-9F22-19FAD9F6C3ED@dragondata.com> <5268B62B.3000104@delphij.net> <8A799DDB-3D5C-4418-B064-A2B7821EE0F2@dragondata.com> To: Kevin Day X-Mailer: Apple Mail (2.1510) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (ibiza.webweaving.org [204.109.56.32]); Thu, 24 Oct 2013 07:59:20 +0000 (UTC) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (pikmeer.webweaving.org [178.18.23.51]); Thu, 24 Oct 2013 07:59:19 +0000 (UTC) Cc: Xin Li , d@delphij.net, "freebsd-hackers@freebsd.org Hackers" , Puppet Master X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 07:59:36 -0000 On 24 Oct 2013, at 08:37, Kevin Day wrote: > On Oct 24, 2013, at 12:54 AM, Xin Li wrote: >=20 >> On 10/23/13, 8:32 PM, Kevin Day wrote: >>> I did some debugging, and watched how the process was getting >>> launched, and I've managed to get it to load! >>>=20 >>> The problem was that COFF files expect to be mapped into memory at >>> address 0, something that processes are no longer allowed to do. ... > Nothing is returning any errors, but the .text session isn=92t getting = mapped to the desired location (0x0). If map_at_zero is set to 0, the = process=92s vm_map has min_offset set to PAGE_SIZE instead of 0.=20 ... > Also to clarify my original posting, COFF itself isn=92t the issue = here, just that this specific binary wants its .text section to begin at = a virtual address below 0x1000.=20 Thanks for getting to the bottom of this - may be good to indeed make = the error a lot more human readable - I had the same issue with an old = modified RSA BSafe library - and spend ages looking in the wrong place = for that sigfault. Dw.= From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 10:43:09 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A23E1818 for ; Thu, 24 Oct 2013 10:43:09 +0000 (UTC) (envelope-from satan@ukr.net) Received: from hell.ukr.net (hell.ukr.net [212.42.67.68]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5DB3921F1 for ; Thu, 24 Oct 2013 10:43:08 +0000 (UTC) Received: from satan by hell.ukr.net with local ID 1VZINe-0001DS-JG ; Thu, 24 Oct 2013 13:43:06 +0300 Date: Thu, 24 Oct 2013 13:43:06 +0300 From: Vitalij Satanivskij To: Vitalij Satanivskij Subject: Re: FreeBSD 10.0-BETA1 #8 r256765M spend too much time in locks Message-ID: <20131024104306.GA4354@hell.ukr.net> References: <20131024074826.GA50853@hell.ukr.net> <20131024075023.GA52443@hell.ukr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131024075023.GA52443@hell.ukr.net> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 10:43:09 -0000 Another strange's disabling zfs prefetch gives perfomance boost, at last LA now 4-10 and not 10-20 ^) Look's like system and software need more memory to work quickly, but I dont understand why on newer version of freebsd memory size used by ARC laways == vfs.zfs.arc_max and never go lower? For example vfs.zfs.arc_min=34359738368 and vfs.zfs.arc_max=85899345920 on early revision's actual size of arc was 40-76gb depend's on day time day of week (and load in that's times) but now arc grows up to high limit and stay here. Is changes in vm (I know about vmem) also change priority of memory alocation for diferent subsytems? Vitalij Satanivskij wrote: VS> VS> VS> EEE forget to notice - old system have none compression on main zfs VS> VS> VS> VS> Vitalij Satanivskij wrote: VS> VS> Hello. VS> VS> VS> VS> After upgrading system from old current (r245701) to fresh current r255173 (than switch to stable/10 r256765M) VS> VS> found some strange system behavior: VS> VS> VS> VS> Diff from r256765M anf r256765 is VS> VS> Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c VS> VS> =================================================================== VS> VS> --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 256765) VS> VS> +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy) VS> VS> @@ -5147,7 +5147,7 @@ VS> VS> len = l2hdr->b_asize; VS> VS> cdata = zio_data_buf_alloc(len); VS> VS> csize = zio_compress_data(ZIO_COMPRESS_LZ4, l2hdr->b_tmp_cdata, VS> VS> - cdata, l2hdr->b_asize, (size_t)SPA_MINBLOCKSIZE); VS> VS> + cdata, l2hdr->b_asize, (size_t)(1ULL << l2hdr->b_dev->l2ad_vdev->vdev_ashift)); VS> VS> VS> VS> if (csize == 0) { VS> VS> /* zero block, indicate that there's nothing to write */ VS> VS> VS> VS> VS> VS> But same situation was befor patch. VS> VS> VS> VS> VS> VS> System load to high VS> VS> CPU: 6.8% user, 0.0% nice, 57.3% system, 0.8% interrupt, 35.1% idle VS> VS> VS> VS> hotkernel (dtrace script) says VS> VS> VS> VS> kernel`__mtx_unlock_flags 292 0.3% VS> VS> kernel`__mtx_lock_flags 315 0.3% VS> VS> zfs.ko`lzjb_compress 349 0.3% VS> VS> kernel`__rw_wlock_hard 686 0.7% VS> VS> kernel`spinlock_exit 1050 1.0% VS> VS> kernel`vmem_xalloc 7516 7.3% VS> VS> kernel`_sx_xlock_hard 8664 8.5% VS> VS> kernel`acpi_cpu_c1 9737 9.5% VS> VS> kernel`cpu_idle 20189 19.7% VS> VS> kernel`__mtx_lock_sleep 45952 44.9% VS> VS> VS> VS> VS> VS> VS> VS> Trying to understand where is a problem I'm build kernel with lock profiling. VS> VS> VS> VS> and get some data (for one minute ) VS> VS> VS> VS> (file attached) VS> VS> VS> VS> using agregation find most lock's is in VS> VS> VS> VS> 14,159818 /usr/src/sys/kern/subr_vmem.c:1128(sleep mutex:kmem arena) VS> VS> 9,553523 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1597(sx:buf_hash_table.ht_locks[i].ht_lock) VS> VS> 8,386943 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3541(sx:l2arc_buflist_mtx) VS> VS> 8,110206 /usr/src/sys/kern/subr_vmem.c:1230(sleep mutex:kmem arena) VS> VS> 5,909429 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mru->arcs_locks[i].arcs_lock) VS> VS> 5,452206 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mfu->arcs_locks[i].arcs_lock) VS> VS> 5,050224 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c:303(sx:tx->tx_cpu[c].tc_open_lock) VS> VS> 4,232819 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:891(sx:buf_hash_table.ht_locks[i].ht_lock) VS> VS> 4,211348 /usr/src/sys/kern/vfs_subr.c:2101(lockmgr:zfs) VS> VS> 4,011656 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:862(sx:buf_hash_table.ht_locks[i].ht_lock) VS> VS> 3,823698 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2009(sx:arc_eviction_mtx) VS> VS> 2,697344 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:126(sx:h->hash_mutexes[i]) VS> VS> 2,343242 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mfu->arcs_locks[i].arcs_lock) VS> VS> 1,752713 /usr/src/sys/kern/vfs_lookup.c:707(lockmgr:zfs) VS> VS> 1,580856 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1136(sx:zfsvfs->z_hold_mtx[i]) VS> VS> 1,534242 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1291(sx:arc_mfu->arcs_locks[i].arcs_lock) VS> VS> 1,331583 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:129(sx:db->db_mtx) VS> VS> 1,105058 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:427(sx:vq->vq_lock) VS> VS> 1,080855 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:396(sx:vq->vq_lock) VS> VS> 0,858568 /usr/src/sys/kern/vfs_cache.c:488(rw:Name Cache) VS> VS> 0,831652 /usr/src/sys/vm/vm_kern.c:344(rw:kmem vm object) VS> VS> 0,815439 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1845(sx:buf_hash_table.ht_locks[i].ht_lock) VS> VS> 0,812613 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mru->arcs_locks[i].arcs_lock) VS> VS> 0,794274 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1529(lockmgr:zfs) VS> VS> 0,669845 /usr/src/sys/vm/uma_core.c:2097(sleep mutex:zio_cache) VS> VS> VS> VS> VS> VS> VS> VS> VS> VS> Short system description VS> VS> CPU E5-1650 VS> VS> MEM 128Gb ddr3-1600 VS> VS> VS> VS> Storage subsystem VS> VS> VS> VS> 36x1Tb WD RE4 drives on LSI SAS2308 Controler VS> VS> 3x180Gb Intel ssd 530 series as l2 cache VS> VS> VS> VS> VS> VS> POOL is 18 mirrors, two drives in ich mirror and 3 cache devices VS> VS> VS> VS> eg. VS> VS> .... VS> VS> mirror-14 ONLINE 0 0 0 VS> VS> gpt/disk28 ONLINE 0 0 0 VS> VS> gpt/disk29 ONLINE 0 0 0 VS> VS> mirror-15 ONLINE 0 0 0 VS> VS> gpt/disk30 ONLINE 0 0 0 VS> VS> gpt/disk31 ONLINE 0 0 0 VS> VS> mirror-16 ONLINE 0 0 0 VS> VS> gpt/disk32 ONLINE 0 0 0 VS> VS> gpt/disk33 ONLINE 0 0 0 VS> VS> mirror-17 ONLINE 0 0 0 VS> VS> gpt/disk34 ONLINE 0 0 0 VS> VS> gpt/disk35 ONLINE 0 0 0 VS> VS> cache VS> VS> ada1 ONLINE 0 0 0 VS> VS> ada2 ONLINE 0 0 0 VS> VS> ada3 ONLINE 0 0 0 VS> VS> VS> VS> VS> VS> POOL have two ZFS VS> VS> VS> VS> main with options (diffs from default) - VS> VS> compression lz4 VS> VS> secondarycache all VS> VS> sync disabled VS> VS> VS> VS> Data size for it around 6Tb (compresed) eg VS> VS> disk1 refcompressratio 1.56x - VS> VS> disk1 written 5,99T - VS> VS> disk1 logicalused 10,8T - VS> VS> disk1 logicalreferenced 9,32T - VS> VS> VS> VS> VS> VS> and another with options VS> VS> recordsize 4K, before it was 32k, but internal software use data blocks mostly 4k so we try to change (without real data realocate ) VS> VS> compresion -s off VS> VS> sync disabled VS> VS> secondarycache all VS> VS> VS> VS> DATA size on it around 1.4Tb VS> VS> VS> VS> ARC configured to use max 80Gb VS> VS> VS> VS> top most time looks like this VS> VS> VS> VS> Mem: 14G Active, 13G Inact, 94G Wired, 497M Cache, 3297M Buf, 2214M Free VS> VS> ARC: 80G Total, 2010M MFU, 70G MRU, 49M Anon, 3822M Header, 4288M Other VS> VS> VS> VS> VS> VS> LA's - around 10-20 depend on day time. VS> VS> VS> VS> VS> VS> zpool iostat disk1 1 VS> VS> capacity operations bandwidth VS> VS> pool alloc free read write read write VS> VS> ---------- ----- ----- ----- ----- ----- ----- VS> VS> disk1 7,45T 8,86T 546 1,49K 16,4M 13,4M VS> VS> disk1 7,45T 8,86T 272 3,91K 11,7M 27,4M VS> VS> disk1 7,45T 8,86T 344 2,94K 7,26M 25,2M VS> VS> disk1 7,45T 8,86T 224 2,02K 9,91M 21,8M VS> VS> disk1 7,45T 8,86T 244 2,35K 8,23M 18,4M VS> VS> disk1 7,45T 8,86T 245 2,54K 6,44M 23,4M VS> VS> disk1 7,45T 8,86T 114 2,94K 3,28M 13,3M VS> VS> disk1 7,45T 8,86T 288 4,43K 6,09M 33,5M VS> VS> disk1 7,45T 8,86T 157 1,26K 2,98M 15,7M VS> VS> disk1 7,45T 8,86T 94 842 3,07M 13,6M VS> VS> disk1 7,45T 8,86T 327 1,71K 9,63M 8,21M VS> VS> disk1 7,45T 8,86T 237 1,81K 5,73M 29,3M VS> VS> disk1 7,45T 8,86T 247 3,47K 5,17M 29,6M VS> VS> disk1 7,45T 8,86T 165 2,37K 3,22M 16,7M VS> VS> disk1 7,45T 8,86T 155 3,23K 3,27M 23,9M VS> VS> VS> VS> Strange as timeout seted up to 10sec's. VS> VS> VS> VS> What intresting - after reboot system work fine for some time, at last while ARC not become 80G size. VS> VS> Low limit for arc is 40gb, strange but old system can take memory from arc eg like this VS> VS> VS> VS> VS> VS> Mem: 32G Active, 8797M Inact, 78G Wired, 2370M Cache, 209M Buf, 3977M Free VS> VS> ARC: 43G Total, 2204M MFU, 28G MRU, 135M Anon, 7898M Header, 5301M Other VS> VS> VS> VS> On new ARC getting to it max allowed size. VS> VS> VS> VS> So for now question is, what it can be, what we can try to improve system perfomance and so on? VS> VS> VS> VS> VS> VS> VS> VS> _______________________________________________ VS> VS> freebsd-hackers@freebsd.org mailing list VS> VS> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers VS> VS> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" VS> _______________________________________________ VS> freebsd-hackers@freebsd.org mailing list VS> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers VS> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 10:49:40 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 133DA56E; Thu, 24 Oct 2013 10:49:40 +0000 (UTC) (envelope-from fbsd@opal.com) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D2B9D2274; Thu, 24 Oct 2013 10:49:39 +0000 (UTC) Received: from pool-71-174-185-176.bstnma.east.verizon.net ([71.174.185.176] helo=homobox.opal.com) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1VZITt-000HOM-5w; Thu, 24 Oct 2013 10:49:33 +0000 Received: from shibato (ANice-652-1-319-97.w86-193.abo.wanadoo.fr [86.193.178.97]) (authenticated bits=0) by homobox.opal.com (8.14.4/8.14.4) with ESMTP id r9OAnRvS002537 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 24 Oct 2013 06:49:28 -0400 (EDT) (envelope-from fbsd@opal.com) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 71.174.185.176 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX18Tmh02OKDIdc3TM82D//r9 Date: Thu, 24 Oct 2013 12:49:15 +0200 From: "J.R. Oldroyd" To: Adrian Chadd Subject: Re: 9.2-STABLE r255918 with GENERIC and iwn - core dump Message-ID: <20131024124915.315d25c4@shibato> In-Reply-To: References: X-Mailer: Claws Mail 3.9.2 (GTK+ 2.24.19; amd64-portbld-freebsd9.2) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/ni96b7Sh5/dGHs+xDk3ABl="; protocol="application/pgp-signature" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (homobox.opal.com [71.174.185.176]); Thu, 24 Oct 2013 06:49:30 -0400 (EDT) X-Spam-Status: No, score=1.0 required=5.0 tests=AWL,BAYES_50, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RCVD_IN_SORBS_DUL,RDNS_DYNAMIC shortcircuit=no autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on homobox.opal.com Cc: "freebsd-hackers@freebsd.org" , "freebsd-wireless@freebsd.org" , claudiu vasadi X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 10:49:40 -0000 --Sig_/ni96b7Sh5/dGHs+xDk3ABl= Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Wed, 23 Oct 2013 13:12:05 -0700 Adrian Chadd wrote: > > On 23 October 2013 13:10, claudiu vasadi wrote: >=20 > > Hi, > > > > Still getting the "Cannot reset interface wlan0 - exit status 1" in > > wifimgr but no crash yet. Will keep trying :D > > >=20 > I have no idea about that. It's likely there's some net80211/iwn bug(s) b= ut > I don't use wifimgr so I don't know what it's doing. >=20 > For that I'd bug the wifimgr people in PCBSD. they can always file bug > reports with me :) >=20 > Thanks, >=20 >=20 >=20 > -adrian wifimgr isn't a PCBSD tool. wifimgr just a GUI front-end invoking standard FreeBSD command line tools to do what needs to be done. In this case, the command being executed is: /etc/rc.d/netif restart wlan0 Run this command from a shell to make sure there are no errors. Also check that wifimgr's su module is installed properly. It is %%PREFIX%%/libexec/wifimgrsu and should be mode 4511. -jr --Sig_/ni96b7Sh5/dGHs+xDk3ABl= Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iEYEARECAAYFAlJo+zAACgkQls33urr0k4lPsgCgkkK832hIZHr+cHkXbcGG66qY IuEAnjiRPaUUQcxy4KDoRq11BEKb+vmK =g9Np -----END PGP SIGNATURE----- --Sig_/ni96b7Sh5/dGHs+xDk3ABl=-- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 10:58:12 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 09815DB0; Thu, 24 Oct 2013 10:58:12 +0000 (UTC) (envelope-from claudiu.vasadi@gmail.com) Received: from mail-ie0-x231.google.com (mail-ie0-x231.google.com [IPv6:2607:f8b0:4001:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B98882334; Thu, 24 Oct 2013 10:58:11 +0000 (UTC) Received: by mail-ie0-f177.google.com with SMTP id e14so3452008iej.22 for ; Thu, 24 Oct 2013 03:58:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=bxjHRdtrJh99pKv9NtlYEnvfuQmeEsOqaxaRZT7MrT4=; b=KxxJYxOPhscCXgDUGPtEVBZ6Sdpa16tbuy5tbLfJxo8RhL7Bkk8I4beMFHRmxlmmq/ 65hDvxtiaL6rj4Pc9ExIVz7LtEIpErUX30dGsA4V/CmBjw5m7a52BESKirvPq0TQSogG 8bX0WXaZySPUDIYbEv8SXN3dv1P1RN7De+GdH0KG2eE1aJDrm46SXY7leS2Frh/vyis2 Aq1HFsbkpkrrfWXE6WQo3wTBnjBbiF7v+FEtOYTnFvOZat1rrnA777xUTbZPdHsYSOny GauikiC8BU5t4KTz1GigfcWPIL24hPVZVTMjR8cRdSQsVU8YlaHXKaFSYJ1fcmGgYcG7 mD3w== MIME-Version: 1.0 X-Received: by 10.50.67.107 with SMTP id m11mr1215787igt.11.1382612291197; Thu, 24 Oct 2013 03:58:11 -0700 (PDT) Received: by 10.64.134.169 with HTTP; Thu, 24 Oct 2013 03:58:11 -0700 (PDT) In-Reply-To: <20131024124915.315d25c4@shibato> References: <20131024124915.315d25c4@shibato> Date: Thu, 24 Oct 2013 12:58:11 +0200 Message-ID: Subject: Re: 9.2-STABLE r255918 with GENERIC and iwn - core dump From: claudiu vasadi To: "J.R. Oldroyd" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-hackers@freebsd.org" , Adrian Chadd , "freebsd-wireless@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 10:58:12 -0000 Hi, No errors and the file with mode 4511 exists. On Thu, Oct 24, 2013 at 12:49 PM, J.R. Oldroyd wrote: > On Wed, 23 Oct 2013 13:12:05 -0700 Adrian Chadd > wrote: > > > > On 23 October 2013 13:10, claudiu vasadi > wrote: > > > > > Hi, > > > > > > Still getting the "Cannot reset interface wlan0 - exit status 1" in > > > wifimgr but no crash yet. Will keep trying :D > > > > > > > I have no idea about that. It's likely there's some net80211/iwn bug(s) > but > > I don't use wifimgr so I don't know what it's doing. > > > > For that I'd bug the wifimgr people in PCBSD. they can always file bug > > reports with me :) > > > > Thanks, > > > > > > > > -adrian > > wifimgr isn't a PCBSD tool. > > wifimgr just a GUI front-end invoking standard FreeBSD command line > tools to do what needs to be done. In this case, the command being > executed is: > > /etc/rc.d/netif restart wlan0 > > Run this command from a shell to make sure there are no errors. > Also check that wifimgr's su module is installed properly. It is > %%PREFIX%%/libexec/wifimgrsu and should be mode 4511. > > -jr > -- Best regards, Claudiu Vasadi From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 11:55:21 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E2076CB8 for ; Thu, 24 Oct 2013 11:55:21 +0000 (UTC) (envelope-from satan@ukr.net) Received: from hell.ukr.net (hell.ukr.net [212.42.67.68]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4296E27E0 for ; Thu, 24 Oct 2013 11:55:21 +0000 (UTC) Received: from satan by hell.ukr.net with local ID 1VZJVX-000J0j-6q ; Thu, 24 Oct 2013 14:55:19 +0300 Date: Thu, 24 Oct 2013 14:55:19 +0300 From: Vitalij Satanivskij To: Vitalij Satanivskij Subject: Re: FreeBSD 10.0-BETA1 #8 r256765M spend too much time in locks Message-ID: <20131024115519.GA72359@hell.ukr.net> References: <20131024074826.GA50853@hell.ukr.net> <20131024075023.GA52443@hell.ukr.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="fdj2RfSjLxBAspz7" Content-Disposition: inline In-Reply-To: <20131024075023.GA52443@hell.ukr.net> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 11:55:22 -0000 --fdj2RfSjLxBAspz7 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Forget about profiling log Vitalij Satanivskij wrote: VS> VS> VS> EEE forget to notice - old system have none compression on main zfs VS> VS> VS> VS> Vitalij Satanivskij wrote: VS> VS> Hello. VS> VS> VS> VS> After upgrading system from old current (r245701) to fresh current r255173 (than switch to stable/10 r256765M) VS> VS> found some strange system behavior: VS> VS> VS> VS> Diff from r256765M anf r256765 is VS> VS> Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c VS> VS> =================================================================== VS> VS> --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 256765) VS> VS> +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy) VS> VS> @@ -5147,7 +5147,7 @@ VS> VS> len = l2hdr->b_asize; VS> VS> cdata = zio_data_buf_alloc(len); VS> VS> csize = zio_compress_data(ZIO_COMPRESS_LZ4, l2hdr->b_tmp_cdata, VS> VS> - cdata, l2hdr->b_asize, (size_t)SPA_MINBLOCKSIZE); VS> VS> + cdata, l2hdr->b_asize, (size_t)(1ULL << l2hdr->b_dev->l2ad_vdev->vdev_ashift)); VS> VS> VS> VS> if (csize == 0) { VS> VS> /* zero block, indicate that there's nothing to write */ VS> VS> VS> VS> VS> VS> But same situation was befor patch. VS> VS> VS> VS> VS> VS> System load to high VS> VS> CPU: 6.8% user, 0.0% nice, 57.3% system, 0.8% interrupt, 35.1% idle VS> VS> VS> VS> hotkernel (dtrace script) says VS> VS> VS> VS> kernel`__mtx_unlock_flags 292 0.3% VS> VS> kernel`__mtx_lock_flags 315 0.3% VS> VS> zfs.ko`lzjb_compress 349 0.3% VS> VS> kernel`__rw_wlock_hard 686 0.7% VS> VS> kernel`spinlock_exit 1050 1.0% VS> VS> kernel`vmem_xalloc 7516 7.3% VS> VS> kernel`_sx_xlock_hard 8664 8.5% VS> VS> kernel`acpi_cpu_c1 9737 9.5% VS> VS> kernel`cpu_idle 20189 19.7% VS> VS> kernel`__mtx_lock_sleep 45952 44.9% VS> VS> VS> VS> VS> VS> VS> VS> Trying to understand where is a problem I'm build kernel with lock profiling. VS> VS> VS> VS> and get some data (for one minute ) VS> VS> VS> VS> (file attached) VS> VS> VS> VS> using agregation find most lock's is in VS> VS> VS> VS> 14,159818 /usr/src/sys/kern/subr_vmem.c:1128(sleep mutex:kmem arena) VS> VS> 9,553523 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1597(sx:buf_hash_table.ht_locks[i].ht_lock) VS> VS> 8,386943 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3541(sx:l2arc_buflist_mtx) VS> VS> 8,110206 /usr/src/sys/kern/subr_vmem.c:1230(sleep mutex:kmem arena) VS> VS> 5,909429 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mru->arcs_locks[i].arcs_lock) VS> VS> 5,452206 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mfu->arcs_locks[i].arcs_lock) VS> VS> 5,050224 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c:303(sx:tx->tx_cpu[c].tc_open_lock) VS> VS> 4,232819 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:891(sx:buf_hash_table.ht_locks[i].ht_lock) VS> VS> 4,211348 /usr/src/sys/kern/vfs_subr.c:2101(lockmgr:zfs) VS> VS> 4,011656 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:862(sx:buf_hash_table.ht_locks[i].ht_lock) VS> VS> 3,823698 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2009(sx:arc_eviction_mtx) VS> VS> 2,697344 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:126(sx:h->hash_mutexes[i]) VS> VS> 2,343242 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mfu->arcs_locks[i].arcs_lock) VS> VS> 1,752713 /usr/src/sys/kern/vfs_lookup.c:707(lockmgr:zfs) VS> VS> 1,580856 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1136(sx:zfsvfs->z_hold_mtx[i]) VS> VS> 1,534242 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1291(sx:arc_mfu->arcs_locks[i].arcs_lock) VS> VS> 1,331583 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:129(sx:db->db_mtx) VS> VS> 1,105058 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:427(sx:vq->vq_lock) VS> VS> 1,080855 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:396(sx:vq->vq_lock) VS> VS> 0,858568 /usr/src/sys/kern/vfs_cache.c:488(rw:Name Cache) VS> VS> 0,831652 /usr/src/sys/vm/vm_kern.c:344(rw:kmem vm object) VS> VS> 0,815439 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1845(sx:buf_hash_table.ht_locks[i].ht_lock) VS> VS> 0,812613 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mru->arcs_locks[i].arcs_lock) VS> VS> 0,794274 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1529(lockmgr:zfs) VS> VS> 0,669845 /usr/src/sys/vm/uma_core.c:2097(sleep mutex:zio_cache) VS> VS> VS> VS> VS> VS> VS> VS> VS> VS> Short system description VS> VS> CPU E5-1650 VS> VS> MEM 128Gb ddr3-1600 VS> VS> VS> VS> Storage subsystem VS> VS> VS> VS> 36x1Tb WD RE4 drives on LSI SAS2308 Controler VS> VS> 3x180Gb Intel ssd 530 series as l2 cache VS> VS> VS> VS> VS> VS> POOL is 18 mirrors, two drives in ich mirror and 3 cache devices VS> VS> VS> VS> eg. VS> VS> .... VS> VS> mirror-14 ONLINE 0 0 0 VS> VS> gpt/disk28 ONLINE 0 0 0 VS> VS> gpt/disk29 ONLINE 0 0 0 VS> VS> mirror-15 ONLINE 0 0 0 VS> VS> gpt/disk30 ONLINE 0 0 0 VS> VS> gpt/disk31 ONLINE 0 0 0 VS> VS> mirror-16 ONLINE 0 0 0 VS> VS> gpt/disk32 ONLINE 0 0 0 VS> VS> gpt/disk33 ONLINE 0 0 0 VS> VS> mirror-17 ONLINE 0 0 0 VS> VS> gpt/disk34 ONLINE 0 0 0 VS> VS> gpt/disk35 ONLINE 0 0 0 VS> VS> cache VS> VS> ada1 ONLINE 0 0 0 VS> VS> ada2 ONLINE 0 0 0 VS> VS> ada3 ONLINE 0 0 0 VS> VS> VS> VS> VS> VS> POOL have two ZFS VS> VS> VS> VS> main with options (diffs from default) - VS> VS> compression lz4 VS> VS> secondarycache all VS> VS> sync disabled VS> VS> VS> VS> Data size for it around 6Tb (compresed) eg VS> VS> disk1 refcompressratio 1.56x - VS> VS> disk1 written 5,99T - VS> VS> disk1 logicalused 10,8T - VS> VS> disk1 logicalreferenced 9,32T - VS> VS> VS> VS> VS> VS> and another with options VS> VS> recordsize 4K, before it was 32k, but internal software use data blocks mostly 4k so we try to change (without real data realocate ) VS> VS> compresion -s off VS> VS> sync disabled VS> VS> secondarycache all VS> VS> VS> VS> DATA size on it around 1.4Tb VS> VS> VS> VS> ARC configured to use max 80Gb VS> VS> VS> VS> top most time looks like this VS> VS> VS> VS> Mem: 14G Active, 13G Inact, 94G Wired, 497M Cache, 3297M Buf, 2214M Free VS> VS> ARC: 80G Total, 2010M MFU, 70G MRU, 49M Anon, 3822M Header, 4288M Other VS> VS> VS> VS> VS> VS> LA's - around 10-20 depend on day time. VS> VS> VS> VS> VS> VS> zpool iostat disk1 1 VS> VS> capacity operations bandwidth VS> VS> pool alloc free read write read write VS> VS> ---------- ----- ----- ----- ----- ----- ----- VS> VS> disk1 7,45T 8,86T 546 1,49K 16,4M 13,4M VS> VS> disk1 7,45T 8,86T 272 3,91K 11,7M 27,4M VS> VS> disk1 7,45T 8,86T 344 2,94K 7,26M 25,2M VS> VS> disk1 7,45T 8,86T 224 2,02K 9,91M 21,8M VS> VS> disk1 7,45T 8,86T 244 2,35K 8,23M 18,4M VS> VS> disk1 7,45T 8,86T 245 2,54K 6,44M 23,4M VS> VS> disk1 7,45T 8,86T 114 2,94K 3,28M 13,3M VS> VS> disk1 7,45T 8,86T 288 4,43K 6,09M 33,5M VS> VS> disk1 7,45T 8,86T 157 1,26K 2,98M 15,7M VS> VS> disk1 7,45T 8,86T 94 842 3,07M 13,6M VS> VS> disk1 7,45T 8,86T 327 1,71K 9,63M 8,21M VS> VS> disk1 7,45T 8,86T 237 1,81K 5,73M 29,3M VS> VS> disk1 7,45T 8,86T 247 3,47K 5,17M 29,6M VS> VS> disk1 7,45T 8,86T 165 2,37K 3,22M 16,7M VS> VS> disk1 7,45T 8,86T 155 3,23K 3,27M 23,9M VS> VS> VS> VS> Strange as timeout seted up to 10sec's. VS> VS> VS> VS> What intresting - after reboot system work fine for some time, at last while ARC not become 80G size. VS> VS> Low limit for arc is 40gb, strange but old system can take memory from arc eg like this VS> VS> VS> VS> VS> VS> Mem: 32G Active, 8797M Inact, 78G Wired, 2370M Cache, 209M Buf, 3977M Free VS> VS> ARC: 43G Total, 2204M MFU, 28G MRU, 135M Anon, 7898M Header, 5301M Other VS> VS> VS> VS> On new ARC getting to it max allowed size. VS> VS> VS> VS> So for now question is, what it can be, what we can try to improve system perfomance and so on? VS> VS> VS> VS> VS> VS> VS> VS> _______________________________________________ VS> VS> freebsd-hackers@freebsd.org mailing list VS> VS> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers VS> VS> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" VS> _______________________________________________ VS> freebsd-hackers@freebsd.org mailing list VS> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers VS> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" --fdj2RfSjLxBAspz7 Content-Type: application/octet-stream Content-Disposition: attachment; filename="locks_stats_2013-10-24_10_19.log.gz" Content-Transfer-Encoding: base64 H4sICGDNaFIAA2xvY2tzX3N0YXRzXzIwMTMtMTAtMjRfMTBfMTkubG9nAMy9W5MjN5Il/L6/ Isy+l5mHUsFxRz+UmUat3un9Rt09LfXsro2N0VhMZhZHzGQ2ySyV9OsXQOAWCEdEMJNZ2WW6 sJglEUQAfjl+/PjN9uPT3Tf7w+bnbx6Ph9tvTuf1+fS77n907tf9+kvX/bLenVf+lf91PpzX +/BueO1/bQ5PD2f3Yv35rv+pe7F5OK8+HfY3/oX7lO5hfb/9H+G/IV03etVB8bp8n8vBW6R4 9f7pdHx/Om7en349vb8/3Dztt6f3v92e3n/zjftrc3Ozf785PJyPu4/vD4/bh9Nhvz7uTu+f zif7/v394eH9bf8fnB7X32x+Jwyl3T+dvvzO/vbdB/uP1fr068PGf4F/DounLKxL6viqA8bz gqU28W1QzOCLBzZc/Of790/369XmcNzadVBJhF3Hfrt97O6fztsvv7NrXP32cLjZrjbrzaet WwzXQihe7RhlxEgtsJ0EStxOggExs5M/b48P7592j5uV/d1mvd+f3JqYJN0/uZ24vzu69cQN 6fT4sxSRElmDUobAsqdZrMF+5PZsV8AYheGunA6r08NNWIi2zy4sSPQfwhVhOh4ryhiRJv4B IeJzHGwGFZLp4UIetued/fv97mH1uPlolwGa2ENy/OV3583j7uHxnycPdaewR/GVDzUTc4e6 k+NFclqeo3Q7GWWG44tndPYZCm6Gj3C92Wwfz3EdkC6XjA+zE5qmUw6U6vQ2KPxycXHNTbw5 7Vc36/P65NcPpjcQN6d3H25Owz3ED0DrNcP3sD4AQ8PAJatuwHrzs7Wz27SI2iD4RycBW4S9 AwsvY2WdiKmsk+Rvugk//vRj9x9/+LH7LprG/msjn5iPz3CHgJHBW82VJGvwuLJ3/8kdCUl7 c7C7Xa1vbo6DMwH54yB9loLSNiajZZ1FazvYAttoPRBuG4ExJY1fiFCSSXepQRD7H6j+klm/ YJRKBhsM4e59pY3/o0TLsBAK5ooXy7m0zw+HR+dcQFCDOhcR7SeQ5FU5iPjau7PwygiqGobJ Wtvrrdv+kXsbFz26kwjEW4Pz/bsP5/vyyUuOXUTG0YtoLRYfHIZpr/i52DimCqd8s/1cuGXs /GcDW71PGzs3YwmoqMw5UB13IF81kY4WFaJwiDIuhtnjm/3wYAGg58++PeGjs3/cfE47kT+y iNIgP6CBhUjxwsxO2M22v//o/l4dt39/2p56/8aHS/nbj//S2T+622z7N6YfDxo12bWY5Yby 8/3qcX23PXqjUMVLjx+fbotlaMLkyPFzLZlIzrdYBigSHG58srNW0kZIK/u+3R5vtpGwCUg+ kOlDAUwZwab3lWGqsQ+A7cPh439vN+6p2IcK/rM/33f9m+7jjRIyxO1cUeU/UlFOQMbvD4SS sEBrN6kOkTOYwccLbeg1Aw77kLyXZb1xufloQ42Pq/uzf2hUKc7znoWvLwwRrDhH+eww2l9t 0PkIvUKoeeNyEx8fKdav+sGu+mF1Oh+fNufV8ZdBpDRav9t5hmYtnIqFrvk663dXrVh/2PVq reVO4xY9RdQLAorH293+/eOt/Wv1cDje9+e1uri33e1xfXe/fTgHAw9UhU+wnjE5SShyPyhc Jm0GOBTzMKenj8fV9rP9tE/rh5u9NyZUVMak/Hm8zRr6FaliHVLFJ2gzZl6EoiL4SKD+H0Km NV3TXa+PG7+jvA/c7W9X98en1d2nw+n87oP9bR/Fn/5z91/fpN/572MDtRBQWDvAgs2z265o /G42VGLhKWjDhUTzEa0ENLy422dnGZUNUAZ7+9mn+rfH7Xa135369Mim9mScpslouar3gSUP l137K1ycj4/WoHrr3ieZ9vfvPth/DIIhglx4u7doLEJtLIpt45XXXUZxijaiOMXHoYP1WKaR 1EvSf7E5S+ufv/vHyj9tZ3ZkdeG398SH7opoGOIrQkjrihhB1mAjY2u/7Auboc899FEkqZyL HEfgDIpYO75gMRIYvj+BdyEOuowjVW3wELyrK6MFzmL4ppnJQByP4JbN6BQH7CCBDW/GEI8P WU7uMip/Goql9D8pPYFAYnubUjVC6+DtRM61l4TWPggZRpI/fNu9AG9YGOD75LaPP0wff+xu 7Xs2hj1topkXCoH8bASPukLQJPzpuQ3IZjGjjow78ApJDAFBrOy5L4/lG5oUQVomBQ9+AEdo gLJGWjSTl43wY+tdp8+NuGrmwe0zn8g8KJGMVg8QjGHUYJeIamLzRv9SLT5D+8Ph5yf/KGz8 Wx8gJQnPRtwnFZRbDyZEYWy7YMsYsYYkrEuPtuEVIn9p7dZcCJ2D5ZRmK0aLh2hMyGEMyQHL cPWUCDNhCD3MPW0Ie+jfhqKaZnek46cxRY1Kz5NyWV5CcNaiX0+NuGf3uP2y8wk+7WPyx+Nh c7Yx0cxJxiNyqujgrdcNh/ypw8Mhuy3jdTOmGL5uFWJQPhdV5Bt48LikJmSc84Igqr9L9gFA zHPtYZEyxt6GkIALU2N0cmvDbWOuynVdLHC92ftAnUq/c789vvvwm3tzWH3LBj6bdwbY1mW8 YvaRY6CSMzkToBLg8DLufZvoVh0WjYpNlNib5Z6i/e3u4fbwaX36FIOhkFZJm5ukjxIJG7XP l2R7QEW0t/0jd38qPHBxxef4+Wb7uY/Y3Np5H0B83rz78HlYb+LZzCZ0R8oykUloBjfaNLy3 XPIYpZnEBjPSVQbyODaYAr9FXshbsPOnPneuAEpnybanUxHPGcrSw4ogLbP+qNwpHhfLlIw4 6hCKchHDAihOOJCjhuKYRqI6u/sYFmh3VS2MTK6ChYGkCBbWLxt7UAqPZkKGdkEo+nF38M9P V8bArut2e+xu1lu72jIsZzoaBlkQGnguB1gnnU1Hql70m+wefPiPmrB3eoiyPlRPD48eLUg3 zdp3TvKd6v/PDuHSpoFauaVZCwxz+3MFRIYLZvwz3VOHydgN9WsvH22+hXm5ZT5a3F7GZT6I w6WPmRbWTd7tn7yJqpMs6yydB7VL4NZPUzbcPZtdavu+RoqIwLjygaRUhblamHtrm82jSU5x UuILlfIp925MbCbMJNdy3kzaIHfSTGKRFnDcTAq6MNKaKyuDjW3/5cktcTriAzTrc8B8Yx38 YjwmnIvOQXLd35+2TykIzb4i8mqojTsKz5sgTyo5lajVFM6DN812LDSDrskTw4dEMSIOUAwd fCkRB2R7ITb3HTtVm9xJ9ClZc9T/aZUB84XxrVTj+NYXj0L0yhIABEKRCBgwmxuH8pL1KqAl dliZoGzsSIvcSPOp3AiMFn04b82CDMdQas1ELGBR6nLQuAm0TwlorOHo7AeuCYUPozPTjM66 2mf4b0LwW0aXlo5fliDs9v217Bdtf3u4s/nB/iswcMbp6KjcbEO53eGhR+kNr5GNzmMIaE3V no/wp+lcvfk65S3O5WXlrRdu3gxz5966/dV/P91/PKzM2xKpHm87R4Hddj9vf53hUBj0SXZi aUQ8h30XWF2sOgHNxoLz7Fy0SYeN0RTAV6m6K1vP5wTS5h8IqxFnkTTCR1i4A5E74Q/vYb/f Hj2NIv/WM1sqCHH7abMjk+tiOAQ/ERvSqdhQq1ZsOLE1jdQRYCEUkRCAu/tHn/noquzqfjCX ABkoiSQpD5mgmPElFDM2jYhgqTSoxlGJqfTyWlmfSjOokeVRKs1VDPyc3Y0LYZLmpfCcm2Vc sqZFs2aYGBwB93zkpbhk8/XzouURxA+EBhqmKguu8ZUNtAaF4pg4UG5Iy4OrVgTW2y3zjPuB F33d43nWLoxyhr9++/s//p/uT3/+/fdv6k5cse7H/frj6U1X8dO//vX7b3+fLiiCnXZGo0vI IPWrhnUR36Gc4fiOCER5LiWkE2uNSrzWXKlo3G3ika51vWgzb940qTZvyEzHT3MDs0/o5RyO gjhC5+acndOipgSU/o8i2AilZQtNBg4V17jzcfj/JDFFswW8FFfiDf9/Kk1al4DMNbWuNDoH +y4epnTABbIa+2L1uHv0wavjsQ9tv/1BRat8i2v2h7/8bXVaf96u1sftepJ1SnGSm0N/Gyvh 4wNzf2P/suvom5XKaPqm+7g7DOAJ2ntlAMgEbylJOCg2ZLSPLC7JEagHy3DQgXsFQrPXSkOZ EM00FCnxM43uoE1m5ddIQ5O9kg17JSXy2I3gaMrAZIID6kW3LmaPRzNPWRpjHx0XcXdyvgme nxp/Q9OtdI08+E10uMl8Q5jNQ5YYTKpKqhJLKxm+LwZolMj4xwx1TtcEdNxCtcx3IxiBr1Eq TmdJobUNDiIXz0NRzy5MQYE+Q4QTGZMq9hZUETWAxKK4nqcCaJFYKCp6929f6cDkFDanYD0f DqSWjOVapxC9Penp2YzI+Nm0rka9bMPunzKlm/eh9+H07sPh5N4uTAePPDHrvyJlEjjE6qyN NZkJDtRaQEM0BpxzMKaRN+8eVsd7DxNDBQMez1v7TX6t0/fydFEoekUH2WustM21Z7olHM8u cOmfIA/tR/b91fGXHoWyvyIabRIpynCda0+O1hK2Rufgqj7xqpkDuaTHZ8l0Oh8znEt/bMA+ ldiZaYgjHPjd4UIqe67yTnEHhVkzZYjrPaIsWyWsJjbg+iikjgIkAtJCGohHgNjVhFvuvJuO ewTa4zHlc9DJHbY+/2Z7u37anyPvaVzIAYS4a4MvimAFIEnajJfmQqfH9Wa7Om3vBiRKngwI TbuuFc8uirLkxMC65/DZFTzPa5I2Cio5Usu40BwPoWC8cEyJMc5thJi7P5UZgkqqf2Vj2ukW aseKX7AbjYgRezR+oxZGjOv7G8nDP+/tx91sPdOY1Wva3a0352m8kTYgpcXUvAXlrfvTnSPo DZg2mO3SqMcErsVCguXckS0A4YQlMOeO/OcwKiLB2Ea0OgYT1h3x5I2k9ZADmgaQ1gW+ivt2 5KNZakK+/sMySn6fNmMOLAKafpo//OUn+uO3P5aPM6Vo5ePsJJ6OVFS9RUcrQGKU1yna6nG7 Pc7BINdsovE7wmuvdNsdPm+P+8P65v3t/un0aVjELZK1hJcpU7D/u1Trnuh/17xtEl3LfUh5 vJdCKF1UcDK6dNxaYDGuvLmrwGHh9ryw8uYyDkci6Atvh3cfdochJy/cMyMSRAN2k6IJl8Qk 0y5tvImfc8c5ufScn7b7fViG27sYC7uYNq6IZA+jZSgMT+wdI3U/1FX2TjX3rnEfGq0ES/OS OSv7s/U77uj15hUgmgGlon8DQ6WKuai1tfGpWaMrI5YD6V/ESWqMeCZ4RCB5FRFQIkPnlA2T UiAmrRXP2Y1Och3utyaAEqJ/rClAu2qx3JdM/bX1kM8sI5tjDQlUFyakfD8evwVNIbki5Zzc FOLe2yhrgnnRB6FjpwwTsnjfRsLlYaImdd911ySE9lwvxjUpu++m++6GW4UnS+h1uexajGC8 3ePfp4hXwAd9X/lxqwi61A8TMG95Pv+6sn/f7E7+YOlqEfZHKYRQYgzZc5tkk/E+WJuWL+lX 8AcCSMumJaxXpW4KSphICLDNNGJbZXvR7LrQQVw0XGiIGyIBnXheqX9kiP/y1z9/V+sosbKM zXKjnc2WWs0UHkBM+dF0hNbXOexV3O9dqj7z/Rs3TjyvWjeuWSa5iC6TxnIPjk0fiw9NuYf1 9bRRTpdYocd/fd/AtnNZmAE6ysJ2h5f01l3H/vzp2x++/2OiFWItsxQlY7vO4IXgd5+Q7taM +n+s7Dd/WDtmlTSzDPU32pU7h3nHamCCLoRJcboRPGcx1sXFT3cYGL6CWqfqGsw0GnQPMjNt d/5y55yb+3c2NdY7hyBfA0SUAYjDJPvX3MlMhC+hOOc4cQQIUW3E4TE0NruGMJ9p2N93j5+7 iIgDM7HZAljCgwxRJEXtIFlfknddJIy0GijNK7gWdqGVphINl7Ps1UutdF8jni4lKIQT7v6s XugpZhoXG9I+pXksKT86+y2q8kcPVsCanqLdlf5ps4MZj9Hi+1yJ5UIJ18+0RwtXUDCOdve+ 7ixqNGHnfnLMtlGDDheEChIJnYQKk06FETr1njAZ0wLg5VqAM6oWpFAaEFC1cSwpTtoFXKpi ajO83IdL4EJ36u1uv3U+teuzoadjTENYRLhU6nQBaUO+VIVgEL++oDYaQS+pkuqayVywLNw0 LYsYw52GCIL1ERUI0Bxpt4kALeZaMoMatwmIromjRPoyTEN0BUMrPjJhDMsWhcuwV9ZFGI4+ Pms5qmU44sLWei3yfne72t57Ogetqme96IV3gcFoMW4i+s2BZokjTSBl1kqJGP7kPMz/QV4D vqiPdDxTxEf6/1H0hjRn90zJfFRoggilZM07hfFayhqaNiNgKxPKXYCT884Y6dggoeynSIXG gl5Tn02JcQpKeg2FqjpR0WsoxVoENaW8NDLhPFhb1oL6RqSWl1aF3eJVUFyoiO2Si8BAVfba xnYQULHEBdTYYyjimm3KOoxaQ16lQbWKf4GI4I72PBPBcN6zHjg1qSgIhpPebUoXs5iIohqj zDD6k+Fkg6mjvyuANF6BKYE0twtAmn5jpRQFIOGBpPR+rilW9opmuAnrThskrNJgCWtEKllB kzCQBZG5SHdTUZB4SZVedR8/ProH7eU/QvP/3jX/D5pUKNI4buxzR4AGY01Nk7l+YQj7m3N3 u4efh7XPINdFpcwkSptLBmqEq6rFuJ/ZAKdK+n0hltkfSD6zFrVgLUGLAGwum62uMjw2xgrJ Yzrnr+nQtAS+kLIXZu6a1kauRRh6i1z33xAZW3QdFIeVYSkOMICVBZkhciPtYEJTLNDnhquF 0FC+5z9/2u49Gclxuawndr910bV9Gi34ofzs8uYU7zd7Udg4MJnrRZF1d2zJxS1Zt+kVd9WQ 8f445fGFSel1+r1kAN4X93vhvU1di1h91VUHKIUxswhKGT7wYrEMT3pSz8mio+k1ED/f96Gq 60QeVtDsH9nuu/Vx+7CeXgze9tox+BrnIHkkqZoeqctwcz7JNnMt7lJq4rGewrRAtbGWUEp9 XLrsRkO47INUlPIha1QgNAVra9BhAjR5+K+zh4o39zD0eztSZGK1KqJ6Dg0zxqa5gWvrHFAV ZLKgxQKOvXv9qM7AZVEdbhcahZAWb/DS+OQ/3rxj58d/+/77v/z7377/29wycAZ/16pItP1e yP74qKshQO93x8PTGBygpuD/FoE3K2pVtGreSVJbaLA9CI84WxQeUcTWAcHDEi5Rgu3k3px2 d/6cqBqKrQhzb3FSbs5HzyR07cJLuIStIuLS2P4KVkBEWeNebMU1YBShACABXscHGjrpcQvS 1GPGYI7cs2BEFUN9fLr9e7fZb9cD/RzIzjudMUoVSycLUpMKSFk1rMTFUDd+Zsr+jHTrW+RQ pJtDMCxZs6GvvEhSYxj6Ons4EfoWRyitg0qOXTfg6uuGmIyLMsR0vMjTINDEwiLQ6KUAs/RS 1KdLjgWakNPVMhgtvtXzCkjjjv5ctJBqzBOihOI4PeMwRDaXbgd13iAVj+xeuOhr6jC5eh++ BxH5fFX5eBvXe36G7lVTv7z7cP6yGk8pQjrMeSYdDrZUsGdyFFBK8pv6m6cbrwG0+RhX0YNc jNtzlvBgGVsrABjV6TgpRYbK7+7hh1fXFHCMbGhrBTE2NFiDHGvdPDGBlNZF7iFi07rWUrVE ZeqO1euYMKfz3ciSERZ8Frgavs9Ys+A81pmYLHKmeUqUIQcONJSfn/t/2zWhZgUiYKxQi12O 6B+SpUEM+QMUgBD44e+5DUKxDLwujIeBhQpjzVBi/SazAuTtRCg2glaG5plhTro/rwiISade wvVTLCYvTLEwLj4QHHqBFtu83siaBK9VFXA93q5+3v5aKIi2asatpvaLC+jhiYpaMAnPcC4K nl+wmHpbqsXY01L2sfd7BJoSVspqDJAyG7QwwpgZ+chrWqsgm6iNnpZNjPA9qDw8y7qH+IWk oakXREMcgDPaR37Nwkg2tKopP4UfRVzHE64li86JCVpLkHUkyuiiwbhzcYf/l5kLzK4RFkk+ HRahWyfwEsFlGrYDEJSyKqxdDoI2JLOoeV6APYrRgkxiDpEaxqSxJ0u7+rIxOZxuet2S4Trs 229fRXLaM3/97uG8RH+mAWa9wLrOBRK5RGzDqIRfMSljqKPc5Jp4SK3NrZKwXpPCxbuXq3ie 16efh31fuPxzYxbCRbtSMiuYmGZW+K89/kQJaH/eBUW2Af+oh/iU6KUjNpt8UoFoMj4fDklA d6ItmTV/PkY7gYYCaIaKy7zmtoy5zYim+CmZYvfqtv97ZaPNPvwUddz0hz/8WLV0WgNUdC4T nbMwUxAQeEUIipQHqEslIxQW6mGeKAorWanrEV4oWdYj09gqB09JvE2F8cbE4wHp0PXwLaSs tTTfWHM4UAvMaDTWD8iwWJ+ya/wrlpDGTypoTbylTdZc7hsAm+3MNg4gXWDQ0NiCRDxa5ItH A7kd+p/25CnsCPPTMP120yzEa+OYFAAyAalGZQ+sHMYDsVRAa0XiawQyAYRuBjK4026kI5fO 9Jm79lq3rz3KzMSHuT6neToZx1ltww63jZrhiATnC6/c9Sb5mli99/M7FmCvOHG+nHcyuXKE WKpqQsv23rHWwd4MUvs5ZQzlAOj2gXCRiiBl98OMxUrbAGasBEIV68cvO3oI6w0XcKa5iSgG N5QmSExJ3rOSo+QPpKEd/DUGjjqmJDJw1PrXHrFz0sY9q4Ua5oYl+XeNtSBEpRISNcq353Z9 0zdjPEAvWplr0j7DWEdCTIJe3JTd8+6whOVy9bC3Nh7jsEYJTkZX1zUpAX4BeC/go8QiAK05 E/opeco4kBlIklRxc3Fj9EQViepAzQZ5NxmiBgbmKmsI209a0zkqOVIiFK6RHWrrihRIWDMx GkjE26Qeb3bEwYbGpzfHCMydCjli2k9W9sssWRQFISiKn2AayEf9WGrFODc1YqwYZ01M1iOJ YSzjRBTPB2JZ026EilWiUeDfIhYkg0gbg/zwELLh9JOc14WAwagilxB23J8yhbO2zFJppoVB h2S0GXQUMmE5uFAENV1ULRXguaKj1xR39LEzAQzNF1qwRA6xni9P7ISk9RSYviqu3FyzPtUH oDJMpGoFoHmwZz5/nBYjmLp8FZVuKU7CxAimggnXayK09RPjFhWpJoVCPzELZnUGGqw8pyo0 nWrSelJjQ+K14FQm+TDQpZmAnGalwWtVYkMXKdE7sdLGzDO8HscHlrvZc1Bcirl6GK3pBFgk Xn75vJYBKSM3arap2ZjlLMEjzswseIQnTg0arL44yunRztHUlxHaifGIhEGTXzcMfNkylppS Idv5G74/DcbDM8UdRj4mFxe6cjJn/qQBx3ZRBtlUN4hqjKO5q/25Xd2cD8dpj9foFGVLG3ev J7bDBMyq7eBPVDRC6aVzIJcetSmEEE84Wq+vg/h7XHv12+FhhonI8GWwpnYyOjstx9dU1frg 993uYW2zsM9b30IyBNvRNfFG47whg7eaWzPSfhu1lSJ1b8xQdQ1DBUs5zgV79dfT5uxlNYIq RP/G1xnf7R2q/bjHvAwm+35j9+67JPzl8AZkEdAAk7S5lurNaV2xLC/q4rhUW9918xzdSfQk fl4LnSzQNHjt27t5PP/SEw+M7k2/kJx76N6JFhuQIeqnhguZZzdq3xBtA4Tek/kpsyEbbLbB FfrdhOFTB99qE+aqt60ehGcz790M43m8Fc1PBY4gSVh4TxcIIw1GH1JFkFKllvg6bFKlB1s2 vSdltClqcaSl0WYLPVlKX1qkDLv9YgOB7e3m4TwKZmSpsV8wxMr3edUBBsWrwWJuvfmI/0w5 bxSQGAz8cdNU+49gJMohuNQ3JUGuUhBRFgFO0zq8Xe2IqhJeRK2AidGRtT/oa7isDCfjA3Cz kBp2tAf71Hik/SvMeo200J605JPLw8Pql+PuvA1oLRDFy1TBvaLcbqPQCGvDtZqEqddpKvJr fgEQoYvKHsJ3Hz72UHMBHdg0WoZnCCktdGBd0n9hNGmJKFqDq/EcOt7DKODyobELpvrSMEMH P7DcFZSpa4oX0yhy+dzJQzaC+qsqJJ/WHmAIHain9bsP1vEPWj0QQyLSTIPh+xxYrlC+3uMe qn7SednPt3GW5193D1GskjEknzMGL2lpU6m3vs42OqDwt0D6E5JgOKFyumL1AhlRxVyJwRfS gTXE5hY+mG/Bw8BQe1+cvss/PZ22x3+Ozw6b1ZITj+r9mDRcQGjuT4o3H8i4FuF0WiuLbe2a tXgcsdg2WiNxZsWrcg49xnt7CiCvQ/iWx4h4RbyDZ07GHEVFo+FuPAHJidnh+sMKAgpPJpjq BEdXUZFN6Mepw9rmBf4f/hip6gZ++6/f/bHbfFo/OAJkmrdOmRhPQWTWd6CdWlryCDOPwrRr stRDfRco57PmDFEnccgumqMrSpdXYyo3xgzqxt7GnD7tHs6Sd4+bxye3DjcHS1WPUduHmCWT h+sINtWUA2Bf2aaqVuqG6BZ3TKG09efAjFFCrBasWpw4UTxNsJHSddFgVSNTZWEtLSHTKykt a88JenD/nwZjDK7JZuhDJRb638ahUivnalCjljIi5+ztMAftUOTYXgp0Jgi5hKRfWQeB1aq5 Z2b6X1oENhtQI3h+ioYkVSibcsW69agrw0wFDEy3AwaULgwKZ5WLpXCZ8zbH9cPN4f79fvd5 u3IjpA6Pv65Oh6ejvVV+Q0L3NvLjqECBCQZL2gDRdGRNLiI2DTFN6JO54pj8z936Yc6CtyZ8 tK49XGTBHckkQYpKS0HrVRinQoCP4QvzMYHKRWhJUT01FB33hJvh4Uyh8v3nKjly2iBTZ2FS oXvIw3XEhzyaMaqDhiGl1jdEBhDVHBQbskH7YrzNzXULUIySlk5F2N3ZP63vt0sUrF7Zpf/b Tz8u1dJ67eDipgQ2IwYgMhfMFPrFYCJ52xo7GHbuphXQGXGDkRlviBugzGkw6E2VptUoNuNT Rjhen2FHcyEDZuVggmiqqCCp0Q64iHiGPbemwX6iNd49cimUiEbAiY25YlgG5jxdS0t5zlDY pGQ24JXlxhevZWm0hvQjVbyafghq6iF0BvEdWhLUrwGJTPJFvmPEWmcSZSeWyk/pkTBSTFXJ 6lTAIxgwjsoujHCweWN+R5AvrvBS5QumZXE5MzzR5t88X9MY3FBQpYxkeny8GNdXr4VjMf0A 2zb1VIT+9fnL4+Gw7yFmqhGbwQxHI8DiuizqbxhcF4PPty2VOooTiU7UdJMxF0Zg1xhLevJT xAHiZOClgnuNwGjpqCesUUWQPoZtSnTjgSxrMJHZ0mFFFeeUE8bxp5hVH2jBbSuBtzxYnfHW ka7HOIxqWVLMzQtnudUvsoHAAZPZ8qn45ZniAueRWJ85Mflq9/AYuBJQaw4dVqeHm+SNETYZ VYMGwLSqjOJeaP/Gukd1KR7PMxu1PdXIjS+NCf7jh+7P//K/vv/up1BNkdHSsjTDoSu0xt3x iPGTt3mNAnDdJTGyebKmioxtnqNix+NPIovLRmyQG8OlCE+JC04bw1vtERl32Q15K7TuWkF4 K/1nSqnKJxKtr02EeYF8xjkj/vPtXUwjgtj1VQZp1Be7rA/jygy8kXcfjFd6i+AfS0Lwbv0G ct7MDZ8Hi+lLm9w63aBnLb36M40Aw7kmSD0GGD6jFJIYx2gFzbQ5QRg6SHmOaVkllTj3ZQyA A0AQ1BpiqnnOo8jPSYDOIKhSFE0I8YoLAqo0xuFPUBsCxXXV5fAaqMRoBUrUgG6kFQw2ojgA FA+GIbX0v/TulkN3GGA1Tg4DWYn0J3LEMBrIhwXBg4jBzEUMaATaGWRE68sw0GbCmK5DMQDC 7k8xPz11+to7ohvXFPFGCBHfA9Kjpu+QA7ipnGnPRSrmumg8VeWETd+HO5AoNwIbqOCviLsC ziPKMKnO/b7zaptLxpEp3KdQttCiI+bCEGiyOLHTwBTKVHtOoviz1xp1BkPVCtT+J5N7QXEN GDDNPsdWkhgCehd1jQ+l65+NA31MlD23b6XWQustdJ4qw6MUXIx4Wfx4hcVrA5kyCFM+BgQs 97/Cvn45Tbn8+tAKnpu+a7N/Op39tXQwO56YFqO84wdpWg7vScOQ2sie6wC8qMWggexxDNmj DHflyc3M8ahGrly5e4FAOW8SankkJzyoyTSXCvR2gmo9lCkcfusbFuZgnPamAC410+nWWl4P uXCwMopcACLKDiBQng5IuGDS29DlMdoCfd7iPP1weHo4Px52D1ltmiFNPJwOHmFG8lOn5uha Teskcy/+gmwDhOHNLvuL7XU29Qyv7efJSDfE5up6v8slGQ3mmovD8Mkxb/JI1k83O8f/tT9P sAlgDljjQQDl7Bm6SUU4JA0SDgXKt6a80ASCCCwpJqQpaAaDkQW+J5b4/4do+p5QYJPU/CMV 2HyDyXZGvLGR1SbUbZEDPp9/9SmKXGBjscMAyLhs/yvNuq9X0bT3bWGxYmp3YzMaGiMdXyp0 Es+kkzk5f+rn0vC+r8e+tb65OQ7ItRGr4gkrsvGY4Sk3ECL1MiudouOhsRKkHnu2Wd+7v1df HvtmGlF5vh/+8hP98dsfl3i+F6ri12az1s45Pj087B7unBq5C5IGK8rqUPmhgCkNeSJL23wW UETP425TK8IF7292x/OvwzZi0z8PLXNDN5UJjhYmp9ruDw2wToiLcazZyZKK9e5VatsXBbp7 5+e6Z0N1crm3n3Qz8wNA8sTHwfs2m7kkq5qTjB8/n4Z5bdVtFuJi4TbL9083j7KQWeN9xm3f zS4GDNKvSJnBYkfK6dJK/RVjRxZkeMZVL8A41omgVr1P6UK4BPfPnLCRf7ZZzwhFc6MTymQo XXxlFGlc9rqW0JPGojwd8OosDUeoUUT03wsrFbuQywmGN9YwU+UfFXeqKv+bxAlOjPRfYwnD tUr194MZFUr7YDhLSomUmDTVot0qxWSNFOCtUsPBrnf7w8d1X9Qpxg7nb+jkLgcpckQymEyk mBGgiFAJhzmNUhhiYr/++GFwKlCsG5iI1LG5IQzYPuiR5GaB7Wo0Q8ZnDrsiFr4NyPWserLr qV+Pt6vdTVHa4sVH5uIwH9AXCwSpWX1AeEDFLZXTgw5zSJLDAiZLXk7SsLGPqtXUUOPtSygo 4Xao/uzbo6tiQMa1U2PqX9pQxOSsW4hAiOn9osojVfVVR2cXtHPNeojAvmf9qGvn+XTY3zhz /5+7/woUESrGzsqpTI7NDBCh9FKewzWa+LRWU018XXnWi+XSRglsqasv0dxPx+3aK0fXSN44 nYFinkHeSTeTFVuOSvWfHCDOhEERxtNEjWE8al1aGGQPkAVuVZ62a/9EIiEJLlvKqvM4v+Iw jfOjhcmWLOhSDb85TsIfdvtErqaIt7AmXCHWWjKbWC20TkhG51QhY0Z3e7M+r6ePZqPhqp7/ NnurpsvGlNT5bi4b81iDM0nlFjRPDtP6dqXbqJRfmCO5KTH5dEbA9yWoFN5Q2qUxCC9NXcbi 8+iBbY5QfFnGAG7AzyhloHx8Ym1khTUwgd1dsdCx320P9/4fq5vdyXU20lqp+s7/ZHUTtWgo Ki1ONUexIWGNypXub5iOsfp0c5wNgS9tTmga+P3hbvew2a9PnupK5uj5+bOZ1CUBLNJeqc2s iveHRpbxWDkTNbtzjv+Lbw5a52+oiMUOnAs2p2iH7U/s4213dOZnmyB2WuS0cTVA9QB8SHOk dGomG4VgV850ewkDRBy32pQlZ2o5Aj0MoeVIHPq2e3pwNnB7029jCmPj4SkSTJuHpOeoIAXb ru2tNWnrmvKLp8f16n538sUyHiSRNvt3H+w/ygAstKjY489T1G3Tr8DGZ1RpmaiZzoSUBtzr LrhXTqN2BktXpomlY/qCDJ8bYtPTS5Rh3QJcxuM+v9ZcvD11CM9vuAphGqyCFj7Zmj404+xH WEZy9omRRLWK+Su3lzOlhVqQlMmDZhFLGYLKhtdYCspIYqKRvVPFTB9hWX8WCC6KSQPUl7+s AyE2MmMZwO3pDqwvWEqlQ82Fu7Lt1XMMI7Mg9ZJZYAWzLL5iRgyGhiccgnPeiBVqg4fsqCDA Bztaqnv22RkM3LKAxFjQTqM8rlKqilQU807XHt1oAEnRigsB+2jl4bEP4Ir+cxsdinEFBygz aARnD/6lzIWiEcVJ/F6gftVgQMOzUDJE7ykqRXYKA0w1PmWPiosakqo2AQ1zbQJYh5i9daUd WtJO3hzUE2g9Ss0GSpRgCpqS4KqVUsuFz+UqY824bAy9FKEgryFreAueZtwqaRJLgGsicaqc BvparljppiumMdRUaayxg5xSd6D10cXYm9okxP12IfC8SaCyaRIGj7W8eA2p+BSuXuqSYaSD PPTJqXAPItehrEvOYGV+bSPTll2AawoslXLcfLT+AbYJiFmRFB9MA8YU1LzJXbyGx/SkwozK OXGw37blUWRY9UQwXIwj07jm1j6icUnSmAw5D0apxmlcisOMTiOFmQgRRzpQL+HUJBvLgFZ6 bz/vG2Qs7N0q0/VZkpszqYrg5klkzWuSdWPyYxlppDTxwfBY/NQgxFHnMVcZoeeibEVMwM/U hbymjMfgQlI2WWxANWiTBMHw/Re0mOp5EMJEpQyenx6ADa5yc4GIT1gQKnH2DndNFa/ipjgR TTeVh1Pk7VLWo6ANqT618b++il0L4VXTrjUucUNORC1lLaOxvyN/oLG/Ta+RgNM1O2KrYDw6 v0W9oOU1dlkYRmukiZJUyCspUfj3VGnzQ6gv0SyqA41Ri/Nhddx8njTtHFdQecmNrG074mZM PsDJ+YEwhTAzjbpmNi3UDR0uh+3M1F6Y1Ln2cvjlYXtcwlpuvr4S9fbbv0wS+wVBwbUXPZMl rh/3uXitvO1z2YzPrZOh0ucW+FTOfIZU7HRGwPrcRjAMF3KAl3UJv/Kp8IzT9SlNcUosBa7i eZBay8SNZlJGUVQg9jfhQ3Olg7jjxWroBOvddpW+QCxKTcFYQ5pgqPvO/I2LtCu1Mg0pKkaK xvUUy9nnjQn829vCWsXZq+oM/uZpdd6heJLA7vDugyvTlVcowwVFUb1kWJTfgA7hnazW3OwT stf23DcqoW1Cb3Jub9c/bx/vYtAsEEiFE4k6GSZiAH0x4YbW0WdNuLl0L66k4u2UdZMlcSc8 fkGaptMyrbKfAxNBZBD2zDdCIDEmxhW0H6b5VCSuYNze4rJ5fNow0ekMTu9EQ0LcEz/w7md8 KF1rZPnzJP5GD8T62u77P/301/8bICsjx3kJNRo7oCCBLiWvX2/sGIRmzF5OeD1MBxqnujUJ 6Zna9LiKUK1NDxTpmxKSoFqwoEVkSI1yk5b0WpjsR0RxouxC3HixFD4hXkoIVOeF53HSc9uB s28JqQnmZXRd0uiSuBfTBac7y4ELqhr4La3nNM9RYPFCNwpr482ynZbPLPLpejaOQ4u73cN5 e5wHlFr1hm55P/kkfybUG6ikSAIts0B9+T7jjGL8Hezzr5JA6+kEmspYDrM2IWVJSefQ/SZ1 yvlZvjjJRdaTrsfZQe1AxtmBDspY9hZkuV3GWDzfUuW12BxWVjXboM3C3Lwg/FD5Ng0XXrA6 LG61cWBp/fC5ZoaFoi073iql98bHmq0p2NnE7n9KeMICtchjEazRibJ/TgaxoVsL5JqKNVEO myhUqYtC1PC2MVQsu1GlecJlBDPxiHHHdyXoohmSZKXYnsaMYoCGgNul8L9RNHKAOKcqY0L5 fc+o869ZJAun+T8LNFBcr36panC/vUfoB9k4qIF8aQoSMtVvlGTOHmW+6CgD742OkxbJWYOB ME/DRiFKJjKooyoOrpaTY3KvbHY012HpCjj/QB2W//+f/vxT6IjKmXz+v3dOc6l05LFhjIMS C/H+sTiPGY3/qU0dcBZoV5C7XO1/p1KBlvNEjabcyAZJy0mWIPejvw0u564laWr8/C2eyWd7 S7qP53VM58qKS3xCUptStTJVSQsSfL0Vaol+nGG6HV9Zxzz24zYVR9HLlzTpS7ShZbjbX+95 PN52vQx25xYZmOuG03EOwwihuO5DMKIdncvorsIVADcg1rudo3U7529udt/Y0HSeTvi6++i6 +VyYvLt/3Kc4mSokTqbSlEcqa2RJ1WI2XrsbJSmWiokKW6ZcAIggFQbaK/TFpcs4F5SCEwzH 3RiIefks7tTQ2vJZY0qrm0WCSVdNQLbIKM5pZZgM2Tph81IqK2RSnBCicWq8M2s2jE2ThaZz nj6u6Pvi+7Mdxnxttzer86dBWzoLHhx8S1jYDBtxxTMlIAG1wLgy+G64Cz5T2OCGPKOwAY1r BgvTrjhz0h/mw36/PbrfrvJvE+sGm0Cp7dPvQW1FtO6DOqmZ9aE9ksDdRLEk6+iysf5gQ4in TR//gfVAV8V1iw4we6vmWsCam4tqpHV5MNlC2EG+v9/fSF8pqdUH7vuVuGUoLsWox9KeKGpw Gq21dtzDkK8/xCVBgQxEAWU9ro/bh6ofDeMIKFpe2mRStBItoOLChsTcNgxMcEaqSoPPFoVu ABTWu1FFFDzTcji7cdxu9uvd/dB0uFBmdKoYo8PZRLEiRDOcNpeWXPFqqBKZTCu3Z65UZe3/ JZX1OVj7t5QszpAcPcem0QvzTIHNDYQw9rKFQJHyfuxFJ6gwhOSWAC6i8Lubqd7zk1kxAzlB FtcVEzh/6ZPSMif/x5m4t7ndnTan3cq15K1KhBWl6reGWl8mbjItObO7Xe3Xd3ed4zMfns6V SCUOKjZg8Bd2oGkhYwPa5K4ATmLuGrLa48+fq2UwGsHmwCkBD6iFT08Xy61EpGkThoqW2Jox radij7XvXK38z8b6/+lHgI/WvnyodRFgbPsZdUzXCfNwsnW24tl9lB2spbwKjbZnNKK2SSix C7r5vD5iWhjD8UDo6HWOkyi6pktD6+Z5ML0ZJ+5Tg+kxdY6YFlbvM0WWn9aM93kp/OZYKdSi 4Z/vOuwXf/604s5AdF0wHa4hUxCTEAEaoie3bl/EcFwLowRqNKSuJwVhdDTKDUpHA2tKWH1A rI8nEmXZSBEPLTeDVcw+jBaxovkwmq8vlcFOcm5Oz6xmtdotJuN7qoketDBHzE0SY/LnDj++ yZA47e7cp/PlranVN24JQS2tBDsLtrXZH/Ft7/c+jq+V77f3ZOZxNDxZE/hsVDCtyyr8mKTD Rmp35EuVpWADbeAm8cZ7Lfpboc2rJhGhgqHpIFhy754ykgQC0cwGLge6CUmnWSXe1Chefo3a CyXNKSnJQeq8fJlZQYOCpa40WOI8JF139mGAqhs5OSB1MVmwr5OyhYZygHrhv3nPuoPRQGBE y2DgnPq0b+icmq4Jc5fQcpfRHCyNIhw2cT6uH063/dxFh+GPQoh0FWsIye+HQFtIrPdcqnx+ nfNESes8YZKQ5ZiM4n34muVtxjXkTtHj0+ru0+F0nu0XjZEZJ6K4E5lVxklSjRGk2SAvsFSy H9XlZff9+sZ0jrv9+uM2zkUpWrAysiMHKtOpOgI2HW/Bw6iwWr4toEbgau2rmEoCayw3jvM8 Rsa9H5lmHRVpHnV9T8wYKTn9sn4spsjVFUv70xzzA8V6NwVr3NZQi2cjws/sCA0wGhfexjz2 cEpCcQmWTsdCZGpUGKc3lqlBCEddKQk1WBneIL5gByjheFMQ+vldYwoqyIWhfMEI+bx96E+l gKnmWar5uGsNlJG6PAxhWxThTeXDmvB0PfRRBTWsCq7qKC3Uk7NdGfbbIY1do4c4J5uueWN6 KgYecHzWceZjjp5h9fEtPmaYKODf9cyD5O4Q1wsKZ+pOgK/ThLVR3abSbIyiXqCygBrIbG+L oYTU1OFQXsQ1O7PCxCcWeJgXiqk1kLHFfbsLRSx6uRZ80FFhj5O9tu6hKAtkXTNKWmVNVGSk cqFSzrhQjJ8JYNAzlo/6XHjXpB5D46ij84oHcxIgXYb2nARUD3lw4RdoeNNCFDV+Pme6vP1p Hps2zWNTJwAYCuG6IhqCGF1qioPku6TrowivtYRUhSlYrGHBvSAGMF5PWsPSECBBESPmIcAg IDEACYkBIjNt1/VpRi6hq2c1WHN1pWPIipc106fm16Ba6vjsjJdnHrq2g4PMg2JroQ0JgDR2 ZvG4Snw0WRlX5c/PKkqsGM+VT4bTUm+AELQiQlRa6qzu4P/u2x+6H//4w7+n60GFCpVypW32 2X8Mc9W4cD2Zq57Tgv7qc2XaZ7CgZeqgeYURgH7qVnAIngK9Oq8/7rfffDrnbObT0EMAEq5y QJv7XCXykmAtGeI+QRy1oZSjN2nR6pHXgQ+VdQ1jC+fOXSex1Q1hbdELhzBGeUZMWRgmAM5c 8MxNM6yUWfXE6n7Ro1Hi1+BtUyFLCaSFiS0fh1sgNMNI+SyLei/NXZzFjb1HffI91nA8bVaf tuub6fil+fo6dUYBNLZGsTELl3NdTiJMdQvraqJDGE0ibNaw0kyt+VGErc1o0WXUQhh21Ljn pWBm14Ix2wmaZYMkeJ2isRb7YnW3fdged+4kKzaV47kJlDDK8aRz38hSqBK0mSm1HNP+cPj5 qS9ZqBmpJtRdU3wODoemQu3VKY/+1nkJAsyQMSE5qa2vsK6VYWUGcCJJ/fsj0PW6K498BBYm aTf5CEBMEAEUUiSgxZqnJHoqTD4lTOlq6ExzHGJZsGJO23ncLaAN1PkD5WA00eMgDTiXUbwJ ckYzbTQH1RGjYSQzSwFGVpu5FuyxXaDE8S0v5aWH4SVc0poM3mimwuAfhToR5ywvzDlf3rxo 6iIs/j0gYUcqmVjrtTPsD6BSWZabhgylqLEjtAskVGJOyGxLFE3EpY6grUI5a/NrvsBy//PK ztgrDuzdZKUe0xMiTYCDyJFyCWtx3zIELBhvadubcYFoCDOPwtUBzPw2m/Gn7/5tfkRPQ4/8 JTOuNEEV0fMN6FTqqFVEZtZEphJIzShr3BG4ZgD82T4kj+rJ/lJ9vnn3wb23cqr2JcQKMKLW 22vANbJ9lFBz+ezarPKYaq0TsQPOFmt0nr9YrzwMsivoYgw7TBRxY/7jlzbkVlCrrqXKXfOI nzYwc68aXD7TKmhdNRI57Vc3O+8DTegbsWfq5iYfJkqxFl6b8Gk0IPZqkv3Let1XZXQOYj+O J7GQ+iuBpFm9wFgO5RlJ8Ju9HA0/5zoBL2VP/jyssuMPvnUBFtbNxlHMqKVttiW8LJyh9Zf8 OoVSKRJbxLgeipaCzaN6s0m5EKlOauyliyOI3MtkZHlsfw3puzPK4eOvKVo64FmzQBJ+RgtC 1yrFth5pdSeW9HeMZCQHPExMbIQpiflNG8w3RxK/Xp4mocWkQHdU4ixF1qITXXflASVjFy66 wxfdLU2L56iVeXYMEB676KhO7p1rlovyDGJCxgVjLQaKmB42aRPpOnjtX5+/PB4Osd5UIJ0Z e9WkSOK63Jfmjl9YYbWWmhLWQyfWyDlI38MVsp5V8NVC+hHREmiYjzJo6wai7VPzT8aas74r jYKWkoUJgG68hA0g4y4pG+a7XaKmd/suqOpX4MYkXR1LFyxp4i7E0nufAUqJQigb4hMUWohi 4IooIAD3h5LRvqoaaoCDWfgul8HBb5LjRLWYXH5HVpH0IYbvQy7RvzqekF2hZiiHo8OmsNn0 SOCGj4aQms4hQwMaNaWyQaPmIIq2rLBbmtgLVU7dDOeRMRv98cEfzZamsnpY8y2ntbEJzbdK y9DKpAiLM1SFtvc6gojShTgppY8PMMhLOgGPsIy6P/+aDxBmYxlsDpz7IvhdSJ5lmdZgLAcr OjEbMTdTMxZ1aG00nmXBOBE8BILtZmprCWpqLN5MTVpzO4ff9OvZhZ/+9tc//fjTH/8tox+Y gpTEFckHbdQXrAIRz9vub2fKU5huk3u7yZJqBLh1fd6r9zbr83KcgAo2BIMSBE5ZbMCtDyjK HxkyarwW+ISCbziaXBWMgTxYy0mlZalwWulYR/K4PcyvksALLtEEHoxEpp7YM4iLiUbVOm7m ykgojG+TZV7h+PbC8T7ydbRKkTURIpjmqskyB/o2G/SDdgKSIESSHWLXzPn68IErYCVNei5w iPPjKFGZRqV4FqfxM+3jHtMGd8bVzi+K+HGdOYzdpXGdOTdZE13K9aOHXI3QdZdcA494C5v7 eNv5oLfbuu+1RWT/iw/VCqfMtRuk5jTfwHHnZ3MYhvgBwwC5uKCNJAv9wEjNtZfqaqq5goKx VAs1WmMQoDb8sqlBJYKl6mkCrRODta2g6sQu3Foqw1nlvLwGdB+225uTvYXWZ5XSfLJol0n5 nNIlYSDGouBGYDXOb605gsaislYKjEIwdrFjoX1FAJBkgoLh9KtAP4OBFVKwKhY9b9fHm8Mv D6vYO1uoGrBChyKRUoympgjxo7Q/NaKJ2l99AMcxTJnknBe5kY1l7pZMX7jKLNAh/qBFS1YO Y5hLnOjuZh+EzZxZAUZ15Vo2qK5OJ8V/rjYsFVwAZM9fcBPZaJJodONVBjOqaJo5zmp8qlJz o/UQ+kRlWKW1UMXDoXf9POlGqkjytWEIpJmPSN07ktpwEc2q7i2JvKju3eQ60YVuG/EwNQkN 8TAMKY1xQtBBPxP9XuhiKo4krevwBUmSaoUACkLgJEnXBxuuyxygcB2ajjVG87IhGEGKaTRu SOTa17W+A+Bd8sF3mMewWxMJF/qNVJ7dPcg+GFSpzn5zc0TAxeqTOP7sJ9QnF7VyhL9zlLpU RxkRcRiMDiuD7WdMCokaOxrvRlxtzofjtAV56ZCwF89T2h7tjt15+mtfQbNvvvsQf2LN8PCh I5WqPBZw+D4Q06qgLWvD4tBqw8KnUuPyU895rgVTQovq2d66H8YkvY/1QRkdP8TmCCag6iC0 66EPyzCEVKMKQorMSK0KfCVwQQJHwQXns0eGG+wq0CxUch196oKJExixWtTDjZYSqxk+cErB UnGgqxB3dKDbt4g7jcU3BwwsNYX5OJ4/eY1A6OO183Bmx0jQzoFmBB0YxqhuTfaeepTn3f3W yVt5h5T0ERLhs/H9GWk8vKWkwOs8PCOf9fCar68DZPjhEP/9dP/xsAKZb2VsYiOpfYMqksVG GEk9/UyZhsa4I041ztHTfR8Z1ZQT9/7fc/8hsiHAB+3ayfwy3lQvQ2P/u/2Tx8oYxwWDORpU 4A1mgqjWw5iejiHq5gYUIxDAkXlKTCqFrcYpbIdns0R2aCCfDHES32gMXtFYnLpPlTIFSJF6 /qTgDcIKJ6jofKGmQJFhOA1lLCbDBBNw8Xys9Mik0Oomt0Y2mc3RABo97E7rdio3tM6onnY6 aEU0MO6EASgplLkA0YZ0XYviRZ3YOKSLKTxQfB4dpJxpgQXJZH9KDHJCGGFF/BgDEDKUY47l fheWNPuWr9n6V+Itgunn4C2NIsdS55W5J9biO4Pv08HqjIcfxfNE8ufEM+TIwkWqEPW2BMmM 4nolEyiD19hyhJF5ADmfnDSpRA/6J9I8XWl4o0QAqCxfld5DrTo0LJ0h8vEOc8JrFZeF2f7p eOGMT+uHm54k55rL0TTqJqZRwFjwBVyKKD/KlE7eye5TklZhhPDIbxlErsAEypzq2UrrU88D 9W6q2p70w0lZCiB4gklgITsYPb+1Otvg/FKVuhB08pZasDxXirI8D0Pr+Kcr3NC1NqHtyI9b mw59+uaT61YZLuPbf/3uj93GPsSH7X4JlaslgnoJSOW35unBR6RUV4DZT3/8fbkOERwQc2ON 4gNiJjcKiOI1gyrvDmwPEByjtxVVEBty1z7U+3g3oGhVigjgYjgchwG4aEU4TS2/NMRV9ZbX y5x3d8f146fBMFmOQCPAGf54IhdoPKh7igvkivrXklS8TthtL+C/PDnawPQq8KEQ2exetopx YXg4X6/Y18HHGYImkIzD80pjlEJ1VawT+Hu32W/XD2/fV2UzyuN2fTqFwIbo8flkAlDf4/rM 0rWYXUQVe7ux4PjsEhSvw7px/PsXO7+Bo5F02tGwTMeN7g0UEwVHBOKFpkyKFtG6lnjAykIS WmWhYu506dYYyi8CyZrw37WrlSna1GQi2qQQQBEnkpoeoyAJZBq8byPr7APcr9zE8AqaDpSy BSLnWLBRjmov3n+Ldm6m5YT26GDVxc1pqV8vlUUfa8dyPiKm3cPvjl/+ifTeR2kRon0KIiZQ ggiWrpPmJkZLSlGhcYPL6jEwQ/akqZPHYQYryJgn4qZ2N6av9BEuVcvYuAFn0QEBXu/3Lpae YqhokOWDSNQow5tF+ipWxISMmKx9TqmyYjgfR0NcSLyIFKFOKeasfE51EsWN0poR/PTwuBrV Nwr1mSwDpkqjn5Iz16jWCFprVsgAF+/jRQeTVNDCZrN9nAlMGt6Ht4QCW3d9urqGh7J1dQ1U KguQzKyUedKlDaPS3bapXRN7wMRI+jz115MbZBAKfpPzmlu2Bb9Moqk7elXbeOMRcxaLMGdX hRl0/KqwJ66jMT5ZI1SqP2sdRv8MpBqqVSshJgv5215QjND2Pcxa6V2ZoJnSsRfaG2mA6ege Lusu3A/ImRTLRIQcFEajlaCkieTR6aGkjLA+rLFOycZ5o1IxTZGlyKUTSQuJeZkpoow2AixW 44lJQCY/CTmW0M0PIghdgWMGJuKNjQ5j+mXdQ0rptT01Q9MciDcu9JrLVkfXqZGtYgV0TtCO SRuEf+3B3wx4iS++gMm1sCbV6xB4YgwNzAj7lo3T3jR3Otq7/nGmJRMvRuqlqky5ZtAz/yeN MSZ5DJqi2ILiS2W76kOs69CiRQLpVyNdA1JaDUlDoKXgqcLmZqkNT0KksdPmpNh0pVBOCnKl 3uKE/Lbbr/a/fEzj//y2BE9tjWr80iBYYqTaHDzNPbOvWaWq2SNj1oWh5cZRhsumoVSqVWJr pyYmIJoXLWO5uQloChxHkk+ozn4KyfX0/FiGHFwtUHqLzRubrCqs/Dj0x7quRlQ9K6jOZkNI hV6gfZHBOa5bjXqdD2DHn8MkGk09h1/TF0PYKE+z768351PVL+iU/cJHa8JV5DM6pk2aSdTs F/Rzshf0C6qp/jIMswWcuZ75HUuwqJSpuQ69C0boNvDidksXjJNmpKWLgq6uaW7pAmsO+h5I YTK3g4nY4QaOQRRtuWc7DQ1G6I32kiuvwXZSpsV2wviy9hsYzB+9mC87Vncu7jVKb5A4UYgn zdql4dSNfZKb7fvb9enseu7DZi7cz+o/7jtMev5dfM/RHT+fvrk9h9Zz12p1e/5YQDxIy4lL /QiCp3uR0hZEhRAWhqgtpVg9vMvDwQY7bCT2qO15VQsfNWYyRtyAcr5vWWpKz1kXY4My1lQ0 P1bLwDnrdWrM69GxwyOnEcYmNMgJLzHloq4kF6bcPRZkGUZi/HkgjDSfS7PslZi4o8ZvpMAu sGvIWjLW8f2LkDdHxsEOKAtivBpMqnBQSC2oRus0hdCx+KpoNGDQjNVS5leyoToMnx3b0Abl 1lyNSp0ptzbHrUrdmXI7/PivFz971b9jVP0DzmE0ks51bVGcG9H5kQ3GBmoLtRS251icmm2Q cb9Q/pFCd6St1D9/xeXUFdcyNtlSk5qUlcwVZWotZmRptDvqNFkSrUnWitaYtSmJNJOcgdKy YO2xjH+n7eB1ZbvWAPTbsbt333i13fej1ig+qgSXyMRPhln6OOZqymnOaQdImcjmUcUQh4S6 T/ieWmUDS2LqYLFKYvCL2ij3XNRLV0qFirr9dGzxQfBxQkUVGzA5adwSKlsancjQ7DouEY24 JCQwnGfxOuerwmt7Z1LrKdhopSo4RRaK1K8TOmvZNvtYPKUo+hCpvrTmOF2H4PXkOQTg8SPd 6wvHuL3WjVrJW4x0DzLXPUj5sL4vqtJuIMjIXoA0Cs3wXAk7zoeaWT4astbwRwpZ3TJG1xWk BIFBvS9eRt0uVUTOqPVkaAA/FWdMDJz1GMjm0822x0EqiZsQh5wPx15qIEi26QT5F/vDjUTL tZQkFO15+1PLHqTH1FoGOh3oxcto9a23HhPnqJO7MOYYOtla72Zk3W1+aUZH13l8DPIGGzk3 qTqTsF2f34v6uAzLaG8Sm96fV78dHnIlFAOrBC4w8nzw3SyVLcWQzK6ValGcljqdanm18uXQ 2Ss/jaTkVWDtQDJpWqVAUEiTfRTw6FOp/T4NwomoiQVYpddNmmgfUZ4mhCale+sNTRGdigT+ amZaKlV1gXG/++jXsj5u+HH9cHPwM5xHFAf7U2QOazHHxyfDqRojE7Rtk2Fm8KjMwWSNCPG/ 17u955XXkwLcD/qX00E7wc9Kk1931RgiSCVK2pc499Q1n/hy0uFh9ctxd97GveSGFkzvsBbX OTZoFSr8JvET94DRfOYnrn3dNMD7aLE/V6uTXQK3D8oEjWCbOmgPsDCHqpvEZuCKq9QkyzT4 2Ne636j4Fx8nWdK3EMq+xRLiV0Mel2qMJxexv2vRULAad6Ok1o+uKSl5LUWv0sAr5fdZix4z MbW95FQ0G2ywDdE4LwYUWcgjqskVFPgUuQL1SUwOCsJpTArVreaauk260aM66ghIaSkwwpXP xFy1iHo4wnCmhe6b+oC4uDuqnTEgOoz2ZIHZwPpyMGjHupwuBzsmx3J0gOJEpXY9Z9kIPcrq 2UmlRF8py5OVf00hMZfnKQjNG+x368ZmOS+1jmQFFOBb8sIJxXOtfAVggsayFK3KLIc2mykH qwELPOUYLufrBTAuwB7grRgCwBrzo7vocHLcMH1gC9F0dzS9bVemfWY7RBbRoVtommpTgIUF noYpGT2q4cGVSJk8q14M3xdiqQjLdeRvIIo/jKVjkFKM5EXXd0YQJ9i/jLV3MLN/a9bHIoSw ocffXVxTCAihrPUnUOVxXOCpwcZe2q6WZtfYxWw++gbQnoZo39jd1po6NFPCs/IFlLLNefLs RFaNTiEeZNVuPCzqnDC6X2P0sGuPfeYmuEFu9dgwlAfqsiLU0PCEpy+KYEvHzAXOtIBC+y+d BVlObM/0oDYDdCQLjXjDUam0vBSgcyCfm1Ro8W5xPdvLQOmw1TIm68b43WQ4qNotbfWZUzP+ 07c/fP/HydiV4uYVSEsO49Il3Lmgtl8Co+ORTK5mgrNvFI208jnnN76QnE+wsXJ3YBYRZYNJ x/n9rC8wSmqag9+aA3H+524dNJdsUBwOP+S4nVJJUgwvnZ56WIyNqRvcFVp3oCETNsfKpqft 3t7Tam5F0NdxAu3F6Jk46ZhyrZLYjZutMdQKEVHGHer7WulQOBRxSQkWF/fCPRksnWufDGcx GdzZh8GCdo9/n+U9A65C3olLZ7ANZaNFGMM2sOP4fAQyIA8WgtutZK/D7Gidg8tRnb6lX5An mMAgdI1PaIoMNNnl5SjDUzAAD6GC9eJpC7jdjWjPjZBp+BbXFO/kcZHspBXzdcdyEX/74dvu x/36Y3TtED6ccZ1rKULGZEZLprJh1RXuZ2JjAUVFJcZoxPnLuw/nL74vcBjfRMBFJrgK/Kz7 +JtcyFAABj+cTm2vsQgvKOHbXViV9w5lG4okIncumexjQSdyjgSJVzWFoc0xyk2LWpUL8Bl7 rfh3OCX2gvh3iejIm+SajC7qx292bTwv6hhxJ/7jhx//8u1334/9bWHAKD5FwTqrr9XlnJpe OClFjAcjYtPVLrrwyqEgmbY0Bag3IccIqDvK6hSgbncFGf/ASAN47MLAOJjFoxt+iGJ+CL9d DE9mLlPlKwVIZK2hMxAgUW7Uc/15zHBo9fm64J7RNNJ96kyXTDFqsEbfTAXKyY09yUUukcE+ aXir0feqDe9hyg2n/SE+nN59OJxamHVB2hoQqNL7E3hcsxkxneKaK4qUhd7CKvrWmLKma2Kh gpmUjoI22YuFYWX+bVH2Rw+WQa9pkc5fPCkwKGYHh795fPrPzX99c968vcIiFXI6j9SNPHKp PW+J1tbOxf94YWE8Ddccvr+c54IccjZtqjsV408QkKUwVOr3csTNTGvVCZCJfLm0FMxrlMAL BZQvCRhfsZM4YSFnLEu1aCtPIQyWsWCeQpWzIPLT0jUnO6/gmoNukpkMlc5pM5lXjY6LfsG8 qVGslOdNMal4EuGPj8Xm3WkytRtSFqi2DhNoFbOcMsMsJiBrhRwcE3gTU7NzFYOk820QDJUC K6PH9OC0aSad11ZkjNPlVPC7eaCH/0l5tKxpjN6EZhFNoTLP0pqHOK/L+Rv8uY5md13D33C6 wN+g9r5BzU/jT+cOQcOgBDy/Cj0xmfROoyTGTtHWMWyKFLa079z7/19ng8DDZn3eHR6SgUPO ZOJrDN9XmrKFxY0oDvjl0QsUGl7xBb/79ofuxz/+8O9lnil1zVPpuoGAyGBrkq7jXDTccMbW m5Psc/zPetNFxFjbEhg3mBgRMEKiM1yeexcS8o7y5fvodvute7vr1Z2ejtvpVKWxKc9IVeaE Ns+fjtv1TdbZ7HAUT3JU2dIdmdaSYPrI1J0+2JF5G6OeRo1t3OCB7bE/OJQYrJXAKLTbppAT WCRZVERL4MpRCwiWNE99iQaOU8LSIsHGo3EAItUEVegQXMqGYtHA2LEpGgeVsVZiv7NKS2KJ bwg6qVxyAQYvEFBCxkBnFp6gpJ5H01aqdr+wNnaBj3fxAp7IisaPqqd07NaM+n+49qiHtYu1 pamb+osGKTASOzlENooEYaIjLWoF+JVqTXTs+w8fNx93D7eHcmAA2i7GcAYSqAYVFbtOWa7a dVxPI47Ay+EOaTtYa+RDSoHnNIdqHfFRMamE5218E+NlzWPY6up7WVTPmBTQKU1iODSafo7R waoQVpH6+qAhLLItjgx1Ne2FcrYJmXVMGJm8KdlHF/IpG0dWseaRxT3AtYkiqqZOLwfKG8K5 4quM/w6xsdYzsbGkGkaRKTcE915CpR7SkcpKS+aqOO5MmeccLtUYdUfTpIMla/HD9vZPpx4d 9ec8RoEfn24duys2pmOSpgA4zwH0Ug5XWsTNNhYKZVCManJkEUAIDC4lrrRcaJYbxDZet2CV rtwa/YJxFF8wWXZtJKNIBTRYw4zrSVc+0ktqu3LQarwiJ4BbrCjxIaRUHK3YNTTxh5XDkT7b oHIoAXGelAg0pACR+O4zT+ll2X3friX667Z/92Ff68aiFqshIdM1iegz/eHA+bz1xKLnTjX6 xgwspW+Oo+c6M261JyGJl5YKu/1MS7lwa170OO+35/Vpv/7oayS9nvH93bsP93dF06imCe8n 8Q4wm2fmopv9A3GnOaRqUtUGyIzGQtuqDVDpKWkk9Gy15mI2aZOzMLYSdBrGbi3lldPE49mN ov7Vr4FLTkbBmw0l0cmc3M3JueSqueO9Pxx+fnr0Hs1kj7Y5Hk6n++lpXK+9D0AodwugikVJ A7DRqbeLyrkO16bfhffz3FCbT/tgkgmoYwy4psRJ6KUyqr9Qfm787dPq7tPhdJ6eHg80qdHY 1DFGeTYSyDET5J4+6w1bRqKWvWo18uwO7z7sDm9fQPOSJ5tPTw9zY+4ahWx9IS9iWgtA1eAi atBpAKisgUzGGjQLqb2NfqVIPU5OfY4NMgU3P5L41LSOE148im6VdBDjPLq/v/vw+e8t2Lpg IhdzELoi+JCtce2onPL5/Ovq8HT2Qtj1RtqfFQJeYwPmoy8cO2cxCl02FiRPHKM1I2mRmvPV pUN0XRtFIpaiSYqmR6FEIR0CKWNRflw6thZW196RyJPWpbYhZ61A6zOXg/Ei7yzIyqmdowYq aivUFgKsjsnffvyXzv7h3Wa7oHG2oQv1HPGhEq4IkpWXwhXowL4uTxh41ei89zzcDUYO03W9 Aoj99yZ3aroxeSP9DcqdEUMwDgAS6cx0UUGiysy54NP7yMKZh9ye2zltsgQqM5lrWZTBEG+J krkT2VaTH/m4e6zOV1GbT1uhVIkSRgUjl0do/PpR1aS05etH6cT1A7QeIlElbK9bi64DNYyV gpA75DVLAiTv+bvWM6WeNof+9RRZCqCETEko03QwY1WIOLtkJCBwjaCKCpmDquPTVDjliAKm KIWz8DSZdTHFTkZTZzdS9q0NCmrYh16TlNan0CKUxscptBrDZi6cQBVECwrRpUWfYdNwKlfa h6t7yBWMjBtCmITQZUG1MpJFtjQ4AQm/XNGnW5SFBdjYR79WXOMFdJG4BmCsb2S/pV0y4iEk 53Eyy6xRrkMKVnvPHFIAsfaivzVGUe5Lcsw1qVDeF8aZvUiGheSUAWWBEgXB+EDoFuEuYLwi xyeIGQTu5s3Hdx9uSjXVjqHYpEB96wtLAQZmfEKMLqwhLMbistSxaP1XMa8s6XP3nsBdH/+K 1wZo9BihPefDCSyq2sODtXaa4Oi7Fz8FIViFeyzdkrnqSCZgJvtgTVahLpqU8G16rJsJEeab KsGLnmc5FakKJhGZY8UUznOMwrCajuqLi/YmIFPtvaEYnE1wPZJOq0v4FUNw349ZQsB9e08C LQGsbegzFSdUoVlAH4yx4U26TEqq3hjw0FdGw5kVvO7zuKLV5LyZDWJt97IcApvfN0k8f3S2 2Djcv7+xf7nsZ4SRFkkYpVkfiMWoinIli2ZklVN/QUMLIq+vWp2PDkTo4xXD2x7HUDsnRJVQ e1SCN4pDIwmEZ40LI8W4MLft8RrTRK90KqcZbdXxwYA2ies3XApwVDNktl8I78ktdqGsVRVx SrvLsArL6xLRsRdh8KCvfS8GwP2TpYqxbOBsYNLHJULJIijnKiFlffbLYruluWovQQxADSwN QPtVI6G8NoCB/pQQuMSttqWS7GMMOkVOb4ZUFEUb+Bg/yGm0BHviwlBaEIvKsG2ppLwE939j GEdR411OLw0uOEx7C2wOnMs+cTnBFF/PeYsmRX+2OFSEjOnDbfBHlddgHS3KBpDBz8qCL/MK aMLN/VO8nTPDIgHjPpUIIgqhzQF5C+dBbx82+8PJPt8FEhu4tGcWylp+4nIqX8OLg1ReIxin EWUbQoEjsXhVab01mGPppe/e/7o+Hg+/eKJlBar/X/+T7rg9bbehDZQW4nfZ0XHAikadyCWm uX25SmqioJGaFHBI9oK8rPsl5AoUpThbwRECZ+p+er7uh/WP2LSzodGwtOY2QmYona9AWDvA +w+mUuoIzjEjdeqakfm1f5pDgdY8ZPaa3Pn0NN2s44sHtLaKua12wasexODiWZiiuMjF24id jL6JjVsUzltjvA9iKMx9E2xcMtQCEQOVXBSPxtkimde9DNMt+gC0qozwoAXVfzfk84CiD9ae kuVt1ZnPo8j0TLiippW+vhElrTEVOIQWLT65wFqFqojH1L3//wAavTZxmPaCHI27HCF22eeP GDCy5o//+OvDZnvs8nVP4qrAk7FklOeKA1UxGuTWDuNMRq+z/jqdSjaBgVmL636FkEtBkvBw JyvG80LacICHL2p3BYVhJHHjWGZ8kSEw54u6ADjYD8oPVJkYMbj6dDIJnJFyS32rXP9GnShe xYLKy5KkIqvMDn44xSLLU7W17a59ONabfR+XMFwCAY17uUaHpE9IjU5f77FolM2Tz79MABfD d4fvw+DHzSVgabqLZoZpOt5XzBoTW8lSevKC7z83gbQp4rHwEdjv/mgTyPePt/avVT+EtVZS e7ztnh72u4eftze9mbhdb95YTtNpLfeVL0F0hecDNbrZACJdhcQBrcurSMW8bCf6gSn8Ya3O otULnzhOiwTNSu/jy0E44xrvymkwrjuxlOg8doB1k1vtAJnpbQXjIs92BtAh8QLtRDpS+6uU w/ZlZtJKrllUS6G6IdOherYmEZwl1tEVxKH42p4j0+jQtfZvFhx1IdokOPo218oei4FyUOgz A2MKIoUigW4MVGTKbOci7RAQuLtFGdC4ijohvYLnpRB6li8gHbpfaAtxg4iVvN4cflPSfT49 fexLORNyuzQE5ZSpBB1Jo6Ih8xNi0mui5DC2Cqgvc2IOs1gJ42QWK7HxP+beFMaS6YrmuTlZ oatpM4lRk1edBmHRv8Cnc790jB6ba8DBxh1TAThdSkTg6XJnIAB3Bo6QiQhsAkMVZ601lpf0 AI32IyDhrf1gGpFukIa18A8Px3v5v+nF1CVuN25jdEaaLazoOKRGs+bifokYSx7Xv6z6zTF9 vcduVd8pJhgdYReS2EB/sg+RjsboLXsyc4S0t/ExlYspVRRSDxKHolkE0p4pRhrwGEcHCw7x a1ZXZkv8miM9fG6GMEVcg03sifoqeLGvS9CQYzbrElRpZPaqG3qKGkAlw5casb9eZfF8evHO QqQ7kbrNtNQJVgQeq4i9Bi7WhSYFRSnaQ6ShbkRFUG/kRoBCiTCUqqWdS4MubuN007HBSkgT l1AU4yYA47zRjDf6bAxdJUBbzUc2TNWjTAIoaIR7bBNdBUuFXuZy3d92h2K2k3vWot98noc6 a01FlCcWItGhhaTVWPS+Mi5AuhLo9DLU5DKoijOEQUNxTDNpvXwfVGTNRgpF2g2sJDOpd4iK zrBCOih/iCCmcF0Jj2eOVoKeEutdF4y84vWgqUE3Kzb+3WZLmL10OxZv9vOkKN2826AHFMuN ZlwEgEIAt3xfKUMW3hYM+dZ1CF3uA8+V61xbpIYVDG8O6QgxxfHYE+RVK1NuBun6vD75B0nj rKubk7XBw3xIIg8RhrJa6etRAgtRg6G9443hrfiAmsaAUb5UnqfBTvCKUnOoN9UkRCMOvI8n ltjwLJG2WAJl3ZBf0yDYaYVtSJbPMMgI6iKfkfn0JOPiRjAXUWJcEWOG0cbhrpXCwmMJz6Wm ayBdOhIZC0M1wwja4GKihZdsMGfQ5SptshpW6aON+XGqqfq2xCUGCcNL+nEpXl1qj5dCcKEh bGDtJ2/DBh0dLwOczNB4GY4wt/TGYNvBanwq1V+pINHlU0i4pw1TSHpWNLM+QKhm/RU9naHu 6QLaiQsCpEgP4gshBgcjvs8Ybzbfoxr06Zp6uBWzXBjJq+F7ALj8eqkCY3KWwvQmSZ/kX6sJ 8bg9HZ6OG29aahbVBQrtLyyrYNUDGwfMUN/LMC5/qGl02iUu13J0IOXCvOY2DLsi3+SMPN52 t8f1XSqqUISrLUupiy5NoKSgeMPa8XlgYDQardgMoZgerUNwaAxCB9W7zNTBsSxGAkEbTpCT OIJHC81jAyGxCVkg0FHJWIjXbEIKEjAn7AaZYmIQnvPvsp1vsJksKC8gqsO4D0vMZENipGDz wTRvy26sqDhqISECWnfMzs0VrCZv5MFE5e5TwbFKul1Hgyo3qjIhvpDq0UENMYJv4+8GA2ve 4t5wYqIoNysEs+LdsEexiBsgKkCDTRvxERwdr8NX7M7Uqnw/zwOs0CgoQHN2eNOQuQPpK3qR tHv45WF77O6O68dP3VzBR5Vyqfl92dbcnX4i4zkxJXGBBWVEJ3+VCEGg0xKAidjixe3l0ijA 6QXH2+ES5XWb7eUKjtCiDMfTM0egazS9uybacUwL08StpgjdRYhbDCaRykAhVqgDLU5oGSN6 1xkaMiyQVJHQI0A5tzkwZkrsfylgYgnAEV0u97POjZ7uhk8HpTfSRjPeM2fEjRTIyxlx3CD1 QEbx6aRcXpb3pRFPUuPRtcFplY3TyRcmfQ1hmNPj+t0HJx9gY+z7ssFWItAIUIbHYlmC4Tk9 kRTmJBgAmftCGRnI2GWQiTQxWYznObCnfCQh0lD4QsXAcQsCpCHfcfEZLYYIdvg8IYKPwpGL +42uMl7AueHZNAwtQeJy6gzIwuUPIB5O+2P1eL95d3pjQa/9Kit6ech6/GGcoX2sbupRyy2j Jjfn77pRacHkBl0XQ2MLAiQ4Vy3DqISuEyFTCYEponJ/oz+r1jEYMANFyniGmb3h/iq7QHrm q18DPDAzdcbhvhQ2GNCdg1axdqFRNGKufI2O6hhMMk0PmjXhMHSm6zDYFfVIoyXZckvBy7QO 80INIW3qvtoSInyT2+1FzZzW/d9756lUbO3taGQaMsMLQiTwEDS4uVkpa6mW4UiaU/dbhIHH o5nD+TwWJ9PI4pRCtjTt2TcUv9/17Ju6NMUNlnMUKpLF+1KpZvJzTdJcEsUIQxNujvaOn7+5 2X1zcxwQIDEiWAtwf3FmoOssf0Fm0LpW8f1L1hJAd5gE3bPuYjHVLSkFe6WMZLtzjBw1+cMy bPLXdhbpMFEGi4jnTVdFFzK5RuQpVQ9zt3fqZr29T8NOOFIKsqH/oBSUBxQp3UoJJjuNwgRt 1Ryhra3ZUPVCpDaE450+jj3gFmbEq3IVe0oq40Fyxt61dx8+rpys2zmnFJRokuOOQmOhZFYE IqYL4lXDUUA9iB6r3zh5ILx+U7TxudkV/StrggtBIZC5wNFu42Nsfhkxx8OrahiDAMTAThYQ Hgpsul9idh2uHr8YwsOtTgOPWDrUZ4EohW/x3nxaPzxs928/qGXz62a/26x2NwNSTivUaayF LucGVeaoFr2uzJENLWN3WtTFdUrNCdw0TMtYyZfEDIXbwkIoJS5MuQjXq0lKLfSqgdVcpHNf V7OWFOxbz6dRzbpo5JFfTz/SyNfW6vGa9bAjmtQMWR6nQ2WhLm+zoeRFFWmJFdWSdKXnzF0p VUy8Oz0MwhpUtgHlQxdQ06JtcSWUj7uDX4UR43jUGk5GR4mKFpIAgt64DQp4MRs58VdoXue6 pZIGiL9XCtNSsTdJyOVWJ1M/dK3jgJ7o/JCKmH0gF5LDL2Vaj66Z5AXRWqAjj4mp1o4je0EF 2gDPTaIdLB3csrOr2Xz09UAW48Bi/F1k7QqTnwIksWCnxlnExjT+mZ5FSXRrJxpZDTOtqTHR lnIq08epgpYoIcfDgsfWGkj/8ssw5HIHOZ525Aio4UNJkoqSKs+isv+PhElKRaJBqm4WrYlS 6JwhqKPS7X6PwnnFtR6wIYv3m3WmCVuXIKVgZWKSwBz3JjySFCRZxyjzDZYQ419JGUWnlXU+ HMOwxDgLW9MGXI/IKdoPRLNdyS7Xn3GP3RmLkf5AMQaMo2PAJKeDuC2eQcaVaS2jyRaICLmo Z7y0EPK3iNr+8Icf7UrseqZUXDqD11Iob3EoEIP13rcKgwgUqd3tan1zcxz22mF1LIOP67RL WngwrgKDMCGmYBCexUATI0cKVpY3YmjbqdxqXC+91jJCzUoduVRmBT9FLXbSRRIjCUeTNgwb XW0QgvbhmTFC9Z28jFAjaX+1HWTgh9OEFZm+VcaeCP+ZjjTRfzoT8uqpNpcUYqq9cs5p5adK fvPpnHs/P51nFQzcF8J28jktgtsvux6TnJhW2HygjWb19hTdRdMKRah/YKPfRPzgQiOOJdTW lezy+3xoLWmzq+EqIIoeDBOZ7uhVMkglur2KTbWKW3eXhwqoeEaVsvkcPmkQFDpUK5aqvU7s 2PeBRPosO8nQqXWCi2apvqnXFZ2OY3XNOh0WH5pNYESsyTu0J+gNaBsdx7yZC2V/MAyW/WMF Ypw6/fUe67EvIPVZmn+yx+P+3YfjvK8Ajjfw2h0egqzN24lBUqJutBkoS10EObd496iR8M0+ n+99+z9AXVCydux2e+zWx+3Dun+YsmAYpoFnphDj6SBlQZzYQ46GlIzVA38wxFVK1UJc27uC y0gmCHN2V8qHc7/efLrZ+pp1HeahOSEW1TRVGJvV4+l0nukq/7jZHc+/dvZJxbU4IX0ihrVk N2vVBr9oscfwflYvLQnprxDlBEsqgi4dDkcDNcFgAhVJYxbst46m072fU2sRcxmW/oXDEgNi q6ZkBOR5hqv9k+ft/duqXbhbOOxFxg8W2g7lAMdL7n/Zuu8Y3RMiKmAYM6NwRTq1c7SDX5Cw Qj63kiuwBJzs+z9ki4EbchL7ljTyHKkCnJ6UhjEvG46QhjGLdt9SVPVXKrcpQhpf4rtz8syz Bkfaw7OTYTtrNdah2z9QNyiaSJtiVYjIOVbENSOd7wX0zhbVJvfNDtdyOVq083gJrpSrCrl1 UaTIKhvz3Mo72hcMSpzUjURbeYffvHRjjWuydPjNtaI1o/BoTQmiav4UBeV7dJGFA/foFJSt yK9nnASbMU5gxgocrpl5cDviKZwCimaplJLXrOPlw1KdYBZ6Cl6i3yx1ZTGHLCZshqyT3kXX QStcd8E6EmWfBd1ynLLP9Vg/BwjIwTDtyNiZ0EGoJfHrGqOq/UclKSQQWJOrckpMmsVhU1ba qAOrV5hsQwUZuOBZqWGDB8ZqqUzjtQbvGo4P3vWPC114AxZpXsr5uSauQjQ5L/wtQpe/fvv7 P/6f7k9//v33PVPERgfeHDlxpYjVSiNjwcWmsDziMz6RF0gibzfPMdtewzXQkFigiTze+IM3 y/GBVOvXOIFCtU4gRTjBTDfqnaQZHMyF/k56eOL8UUS13nUwY/ESEPlVVOuRgXR7ulkdHva/ zuntiVGk4ABzOqh8R1uuDW3NVoF5io9g0Gzg53afRtYcNDFYxGLXQZoN/Eu39f5xfR7sqj8H xRurz/ceeL9AGaOTOAzFxcJD0FDGAICaztu/Pn95PBwSWSsdTJPKuzaiyIE0kPSaG21w/Nyp Rc6hUYzKChxbbzbbx7ftsayBAqzo2SmNBksXVhN8KuGDtk/rh5t9T/quVVcDKrbaBLqN3fI4 WUrYyN2tAygHJnSk2ABXca4mJ5JGbTA2WBNw93+5esQi44SEEcsEp7d2eNCZAvM5MmfR67W7 87IadScRhirm2WqlSrcoTG9uLsrtYSNbhXEJShkyxmoxS0TwHB3/y/BwSC/VoivZBLuHRx8G OyWoMdUELRsrfbVi2c/n49qT2oRS8w8GE2UTAi/dgWyUeNquObC3nK+/wBrTFgU7B2HzHz8o CriJotNFgS6zwko8Jz+BfmfG79eLaasl3h6OP/vn0ofq7nGcj9t/CGGIznWQ7bLwuEAOKSgU PnlRxKYnIjZus/zRMgAMgRKpLYknrbuCCbIN8mZFl+EJAqlJ2kgWN6myKY3TpHz4YhFC+QCb WvVJOpP2z/Rzu5SxiYqvLgADSWjSjrLXTfYPkPX6RSpS1JiTabyeCzp/ufMV3N4Dnb+8+3D+ sto8Pv3n5r++Odso1v7HZVFEsRHwbuMjwZHYy66YLhUMeJkUsjdQNE4Htb9992F3GKZdWDzC BW6jIsN0bKMwjZAheOPmeE7pLWC2mjKUcAQE2PMIK+4hXYB8v7J56ofOn7Z3Q2p43olMVi0I M+X7AIq1II1r3oXBHBhO+idp37Tmzg388D+q9NLxDW2pM3yN2VUxppQoc9keUDm+CwYk4q/t tmcxu6+QQjMZNOvxiqz9ldQFICdTghZJtI4pq2RC4FOMwDElcdvNRhLI23syHebwxqM2C/3G 8O6YWqaknOAUvxnPfAtKcmQjVRpubq9LS0fTNHH5FH7XrHMk/L7o1F8y+y7LHpJRS9JycbLX tWf3jlT330/3Hw+rrJIizViWw4lgEQyUKBoAF8naewAg89YUnxuD/Sb78mRtTPe4m0xd8fCz s8nKQq7Ay+z7+nF1v9scfT5Dg+rr/t2HfV2fRxVWUHJBMUV8ceSc0Xan0juJtqMd/PjceVBf RcMDGY24FO18m2P54//+9i8/fP/Tt5GTmIHsBGi4GkHeaYCidzl2n9f3tNbNxrADiDriGTzg mjJe7YJNbA0InPfNuaeLUSXnDMVVYgZDWzjUWzy5n0+7O0fM7RehWZABo2DSE9IiC5ADj4wt GzkqjcO+xg28axQ+PRLGa4EffEqGAB01cwQJWuxOWIHLfKrS+10vAOt3Jhwo999OJBVFgk2d mGsaZPIUhmpx4GRcO+BucAj6TET402aOE3q1oTQcgpbCeFoeymaVeBIERlzSbjiEBGrhShwS KOdAJBetGDoXnVIBrdONPciKSarr+Zglk1THUoUp+BZcQ2x6F8QZqvA20azCSgLjUDDA5AmL E9XnqOUifPzQ3R+eHs7FqM6SqF2kaGjNz03+/Jrup5gr+Y/tfv7y1z9/14eILOpJurJDvAFS 8+QP7PuR1sg04y0qSy0s8FJe0Or8xQPdA02u+fnMjWh/ac/0INrvyYID03/+eZO6MRgK9wt0 Bc+B2yOoW7eunWy+EflokppQJDWQJ+7YQD4uw1mGTElPywjhhkmsxVqeYrQMHxS6s73fu5wn 4ZbYY9ClaE+OFm2GsJD2M2rQGfX0x1Ka+/n0kShaYgYPqlW2bhmGp3SQ3avw92oXgBlam1Dc qIs0XLGYc8ZVEXyV88/SnOEIv7aM+jWw1qDoP4O1ulpGnJaiTApydNG36ohVceuZwv2Sp/XO Ud903dxdUN+ASIr4R5E01wbv27CIt9DTK/MZHd7ZT/Cw91HhgStNbH2Wbop1HsWt4YmB7OY0 570bLL0mWNRCei7OQMaevonD6YONoDEBAYdSOqtuSRa1P2xUKkxEVjVx3TPYEgyRU/KxlCDa rVjMrLQUpC4fCG2jdoJZjoT6itlhtC86Sqd1z7rs+eqfbvbvPpzW86Qt2hhkT/hy0H5SzXPB dOPG0MQXCZdA9TB/+uPvS8wNxmIdrukKMwZUucarZSvBKEuqRkPLVrSYxnORS3eaZSjYhsdx oVK3xgq5EsNcRUcAmaroxPzT5vV5Fg0zEYul9rlGXgASs4eF+Fbi6Zhd1n2zeMzeARKzU4PG 7IUqxLJoLWkd1C4YxWYjwqkLXkYx37F8n4Nu9HjUvZ9DywPI3FNU1FsTlukrsS9JUc6y9A8A j1P0WK4pDDeGOeJpE/c5PJ174IeR8WSw00OcJhslmSFpMPnQP/kkakz0SfaypSkC1SNywpFX dKex6kUDtnw4vftwGKZPCquhCmy6joM3ls54xyeT0EA2edrdOASoK3qzWaFtU862KUxh6tm2 Dp3jXhVq7i8WFSnajIqGu/AVvbuTErV3f7u21/F0585c3w9rNDIPxPValy4i7ZcykZYxEuS+ dkE14UE0qD4jeNBlO3lJWyFysmTrZMWufxAkU7msb0tqOeX7dZc1zQ2fMO/auG5OmLShdcSj nPhr/DjN88jXDAfb584w8bb+v54wnUDrw131dzDktjNN8aw/xkIwOk2LUrzb/u+V9XeBc+vn m9SYZ1dq9hTerVGZyaLzi3gcF/FKt1+2m5RijHFYAIIWaVxh9mvouPzmTxlE3fxxiSurzCXd 544RWZrWBIpKLVqVpfqIvdhY3OyORQPFyFKUGHHu/tdFU0CXIBh7P2leb7lsKpujcLPtf878 FrTY26Vx9sss1mCGOfwDduM4kcf97uHnAZ2HcwTTtw8Db9NLnPw5jKDm5EcDQSEwpt3bPgYO tpPENkqpYj8A1yrTJNyU92CwCUiOqyoZg8Jkg3CcLIzH0TK4+X/NvVmTJLeVJvp+f4U/9jyQ dOyAHmhGaaTpvt2abmuqZXPv2FhaVmYkK1u5dS7F5dcP4O4AjsPPAeARkVmSiWQqqsRAYTnr d74PZ4wW1nX2Fs5T8/W5KsOKvv4iY1KKo6LQLnO0nHRRe1PejcjxP//Pf/3LH48BefsUdE+8 uXpjQtfeGCLF4LOOEYUO+hwjcXo2DhDtFzMxbsDmji99FeVMTDCFVi5WGQKfRtQRmCSNI8Xr /I8FNKpDQE0uIEFLlBs3BNuYiWE48wxLSk1dpwCHwnmJvHq6ul0SJu7mM/cnZdLpi9Gx2En1 bzH2CIXfjiLFVvNPzNpz8mzNpVzD5qLRq/ewr3dfv9n01zQjGSzyTG0ptYuXlTlllp9NoMBe 9AaYHpUtuk0TMXuwnGpLX1KtWm0pgNGNkNAx9LAtVLRhofpD+DNV2Ba+xqH86U8/suEa0g5i ckWDgZLdMIVTvcwGLesa7H7Art3eP91dlOMbUlqosRKrxWK00Oqv5demFGj6SSLJRlVHvrqW 4hwoaa9eYMCGPluVwV1FbAy/MQ2Rhb1OyN+ZsmJEhBMcScvYuKL7TXsoafi9eo/akRgVWjvy afOWXdCM/i6h2yj4fOCQlfx9cqQEBhRmTpT8Zz7j8L+QkyXuJJ+jskAVN5cvNfM3fx7mCEzA YTA3pVPcTpUVacJLYcxFnkehSqLuc+Ag3ILyntgAnxtsgOvLCrw5x6+z6h2f3sZUi1o4GlMZ CzQqYkjDrdZw3BjwijIKg1fiXLBCiy1piXMPIfPnsVxbYVLmnMV/npJjy3TuQq1345wcFnMz Si48LWUzyt8uRGhEGGM0Fo9mnDFrwQcxnLEbbQtnjHKiaLQrxWUv8905oIyzQPoOKCNh0cm5 k55ioy4BJds+CVtIZENnJA3hczEmfsrRqDTrAPKL5QUs/leYkkQWDe+1UF3hPf3zUZEIiqBP Vett9Ux7s4miGCtV6/M3hKXhdEMY3zWCdrhX1mLDc8+401E8C0wfY6+OOTQx5kz1ioycIx3S GkuHpGYzx2qAkohpN7xL5Sa0TIe5Yy6sjkmK80bLLSLhc94XJx9ZmLA+twsNCKp+puRJYKmM ZYzw52Swww8w+elnBcq+72j0lqnkcuZLjVuN9qAkRM1ozwmHEK01l35fBJmpDxw83KhU8IBg ISF89DooAttOc7cN58ssFA3nA3Qhh/Ox5WtGDYtLsa7FrCWFV6TGEp7IzqwlLiTMltE+H7fH LniQ2o6TOJMw+rIocpoujGkhPIgHNo7jd7c3F/Pk/rhRuL4ff/f6yz+MkRASACdhwLAiV0vq aYpRhA5sqwH+8rOP5kNuN/cVSubB24sv9y+3v8WAhi8GSQuXgAGOzV0/5v+so49F45qsGGMa P/2fBF8g/ZyVuoznMFNuF6G7E3opg4Wf5q3zaZWJXJqM+bQ+siY5n6TEecVMMBn+KFqXZcPi amn8vavtlPjgjcMq38voHBpRf840NbeXIsHeyY3o89SHlhJwwnoJkCxHbPHoTwL69yX+4ULT /t21s6MALqH153Yay37gd7WA+OPw1z/9OPyhXUJcicfkz3lvuEWVEJ0rovb/+PH3w3UYRj6s x2AlVq2yArstbEzE5K1lIefETKkmlrJYptWc2/lwdemD+mfmtJxyd6Gts9rGYiuTduEWm4xV SMnctAxugydq8K9sSNn/5n8BsK8oo5LIUMTKh3nzMYcZPOmMB+o4StlVvE+zi5ETDhiqYEjp 6vpzfgTFKEnCO2m0XV18PlwuvQ7BFoSJ9El9xJcYayOKx/mjXcokWNV+WsskNbKlRoaPzpTC spuqPcpTMUKFl1QV5nZU+JaEO3/Gk3y5uwiMaxOEfgl0vQW/foomnI3O2QhnEHIKZHwQ4x0g y09VmVha4mFGw07Hb0qAsTwrsAFUGuXix5FK43TnlhFHH+Sk+EPEAbDAl23TnySI9Cz7vSg5 LpVG7vORjmKA4XZVDAi5yiZLYH4X+RbhN909Upj5/NwugrMKt8vXcNpPNz6Mv/zppfZefAZF gCMNW320yxfYclYq+QIfs8ZJKcGjZeDcuKxC7A9tO6IUgdIJwVZWqrcjSlYh9FcmsW/4708w nBGAsoekBWiUZcSMEiu/Hu6Cd9RPU6WhJX4nlGYb/8zDWDDSulYjExRF8btcZ87p64xCixSO 9Kf7pZ3Bjr8olX4pQ0A9TBLq2mm/W9MaUB5NLVxZ/jNvCut7QMyXJVXb1h5s6mvWWlqdDbUq DH3QOzVU6jHBwmo3y/lCRL3K7PupIxr41fKSEqqv9rrKhj76ujpYBwmzS5SUTqqjB5b51npM LKV4G5hnDHT+GdTXlZCMeO12S4u+KiFsoP1lCeGd3BEcTPnp9uHq7vLlZWYxWS8n/+phLc26 hG6GJzAnH0cX6y7MyIWSLiDkfMSEVb60/z+0Z9y3vX4w4x5CNIQnR1gI349nI3ywRA3ulAHa hjVhLO8MJS+ATjVgupjhJ5Vd5fq4CDGQySs83t0dnr8LhnXuKzjK4nJlF2Agcywi8PyBjYnV 0ohYHRA+H0t3fr07oglfRAZnkN3hPtSYtt/HgFHc0V8UNc6lQJ8JmcAQtmxNyC0jDcb8gZ6W 45Qt49G5iBQWNReNJyM8ZZObQhJg7kyBr896gA9P5Xif8wq8BsFKxq+WjjmBTvkagSbTVTIg gYorA5rXLsMCe/ea03hI4Y2DKmIC7gIfP668JMwECORtZOhZZEvHEe1/rHcMHBEnGnY71Lqr gCvF+NeHA/7bv/7Lv3zVVfz5hz8Md5efDndzssQW5ExIIlOpk9nEQMi8i4qMLEETKrGQrJbB RmWwitXKyJVaebgLMKBil8y+NasoN35unKGYFUqE77pSbizKnqyU3h6JT+wdCnlTKqaZ7/qc 5v53JCVH+t8LcWooOaY6nzAziIazIGCjo9/27m6l38XkghkMc1UYT02d8rkoOYb/oJJ0JANn 9FaN/euI03/4xz/803Dwsdbjy9vzARfSWlsbonC+a1i7a0ppHuZO5PeBS5jHXp9cvi5ooGZi h0BJE7csI65kGeaU8gQFDwKbgi6kH6TNtv/MpeVIQuf32fVOJZc89lqWxEZrceHMI5PBEsZB 8ZBo+f1vNZwIPR1J4BKnhzgTtekhjsniinHE680kSTm9G0uVRpT+qFsrlOFzfswIqiNOBZwz ebwUrjpkge6Ivwloh6fC8NOqW03LaACyxjLAGcJ8vkABdkOc+ZKuhVWFBZi5mTR1v+xcDg4t Jm+cb25/Gn78X9T5gDxEgHAr5yr0+QiKXmA5HzlWzodxNdeAvW3XCxeHD/edWCYMfNhlEjpy MviL956/W899cccETiOy5AFB7IjMA75O5OK9XmqKBOjlUmqxPl9d/uRa+H2IX2xZojH10UOM wFlpREvYKiIIS8hm8m0y5JNyiDxKgwXKRjWTchNCKb1pv0qP0jH9OKB8QEO/JB+g0GdiWxa0 PoSa/k2MORlvIddJ9Wvwd9As3yWdVlZiftWb5hbVBmdt2sBJ2jimxNxN/pL7JEgAgnwdinDz j47rhfkDiuLOubp5l0xI44oyQi5DHsbpfFdExt9Y6WI47PfbEAJkYTi4GggwSYmFfI13PDE2 H37xV+lwcxVHRf15IUqXjEtsLUYznBGktZZtUhig+mE9aUvQ1pFcgahz3CLJSjPNZ/HLYZoV GEv0EzLMHpElAWYOprKWnMf/K6QCs0sWEDMFbYjIaWrG+tDYpkK42ROMIXQQEK+YJXgYswQu TmBz3q+vv842poiZX38F9YIo/S0ScDMQ/CTpEP9g4nf6xMcSZSd3Vn1BINPHZEUpcou8CgVW tG9heW/Ta2MjeSmKRpRXMdlAzlGwrHSpvNrhKp5ubu++e7rx/502RJeB3c2F/82Q3APc2wwG sBBimI6Xpg5gaAF87Tz5WCatwHlqkfs1LIVMPjbKAXAsqfLRh3MjuiOTTGLrgPrq30t33+9g yk996JI17GScsw0E6Lr3pUHEwefHl9cpjJrAVQgC5vPVRZAu+jUuadshmIb40WLdCemRLE0A lh45JNSRwu5dSysxkbqkOqswasHvDLMf2FoGtqsAvJ780SU86O3haVVNxawzZ3g1NdJdvS8f pP8tMYZWyxTCy9PlN9/7v11Mv9ZuXa4ONf+sNFWfKdCpPx0e76e/BZqKKQjqE6RCKoAh4yTy zRhNt+TboCBVs8kOvsuBHRlylilGIrMsqTGjj714ep0blOWDh36WIdw54X0RdflofY4YZtNl JXF9pfGFCIfSHJy4kFI2o3hb6N0URG7V+8gbvEsLRVdwAa0b650Ac8UawxDsyOS2mudjQhm5 TlgLFnKeIeRRK4jyRCTuhwEhNwkq4MTY4q5qE+S0mNDVOKcFdc7vnAg93Qwvr5evi0agsSrx 0LD4vgOHJqhEqvi5CW2ZOKOyXgUrm/Hr1ocbsaKwMHILNwxjtgSn6XLVmesaSireXFnOWL85 vtS5mXBZQ1jwdJ+ZNRHr6H8vN/hZKLXF3K5y4gC9RudwSkGQ6TNn0BKsErH/9L6cuJNmW5SO nmfM71+efLYB58knFZi44rRzbjWqkKNKem6nnKDezO2UsezfjRZUfFDDy9Xz26fYaGBpM1ym JhMCdBRdKnhLIyRhxMtpx3JbmCuhoPmXh5efV8yG+zZntxbory9Xl3d3wYdY1hCnUqAXlH6S bkUIEzdNCkdmqM0EyI5d4Re6NQwH/w/Kdm7NJkMVCyWCvzHP4V0eprYU87ZnEX0IjIx26asG rsylRcUDLFvHovto5WKcRe5kjpOJcuekNwoAmN9uDq9Xnye0/SxI8NvNN9//dlNqZSH1hkFj yvHBybA9N6uEAboNR1RpBrgyU7YxlWdTa3q083B0IAfy9jNzwak4AhmqXkxOzLJTes3KqPYc 6oZqGTu740u56+725bUt9m2IGDRWLj4qrZrIA+pplZ4vqeVgzQmIbr0tBI7BcWiBA49JjBnt 2bd+Jn3NNCkXP4WaxN+nXsx//PmH4Z8Py2CBEWKjfcK5VDkeAd8YZs0jVUQHEV/oOS0nuyid oj2nr7IJUzT/lYF3t1d3/tce4ui4zgi3DOQQGqCvecKwcgqZuaWd3gSJEkdQDALZg4CzAX/u RB8yjhRhKyeD5YzfnbWgCY2qAbVSoVSJHIZ3enj9cnsYsGAYkYij5NWFLAfirEpvhOUqJtMq B9bWFeOXkWCenZVSAgoIm83qCUovgZQ7g0wvsqEnqRsZotz50/Pj27F8MGdjyOOQIS/AETb1 GJ8+aYQwLHBvcUq35ay+cEVmrmNMtJ0hR6XGGQUfWX7uqiTlstqmxwDKaoxH3jcehh/TAkQC /nMXhyB9nDkRoG53z8ixpDFYay9wRPVlS6LnX+HIVXGYWjn/GQ4ZXpQDFNQ9eOfD1GLhgMlS 36+Hy+frx58fLm4fZj2JfMRBfzkP4sQ/g7EcZjAJX6aCUOTyRyv+PE2Qvugoxw2ApwXeLGs0 WjLkMuJRWgXkXM5apoK4rqHrdNRbETzz0brRxPwtIA8SS550Oi4v0jXPy1Ci3BV0dtVZuZpd DWqY22fnl+TQ5oiQCzz1fRkGZxytXaIrlEcKi/iZw3PPoeDoqJ/cugIWqIP3V50pQqtdvDiQ n8fb6iYAbxhwSCJOORzmS1ZrzYshR5XArpjqrmCemQuc5f0kz9zR/BMIDpCplQZVplmg14KN Ta3X0iPt8zWChL/+efjX3/+/f/zDX7b9KxgrQcZ52BotQoQsMFRk+hROsyg9/I/bywQkYog+ nve9OIedTDPa7xqszBnUy68PIf+Vi9Q5MheDBSuCQPINsVrSch1lUjNlFCgaDP16dfavl/jX c7S5P2KhWsj0do9qhvrn8+2UYm+kzLbPKmr1rZbj838cupPIGt71Gi0uTFrKhXFADp1/0ivV IYhPpgL1ysj/vOgcjizF5ikKbu4qN+BxJoUtoQF3T/Igg9HO4GBVVg4btSYItyi/vIxslSKw bvoP7JpETMAGO9xlrUwJzoLWCq8kaLTVLbjopS1Cb31LJ86HbWosu4De0/NRosZT8bmfAska 3t14UtqiuBtkI7pyxsm5sfeQxxZyb71TWmO30aganUHtzxCJaPS7/nmu79/mKfcVdRPW1DeL 1/CXP2sbjWMiWPRZWxpqsUoQZM6i5Ls7F2NRsIhbxqJ5kYh+lE+CIRl2DPx9qkcBE8ftaNa6 hzprvbxDs/BMLHivvz6+vf5XfQ24jOXAe0W1YjL5dg2SSaY2bJ25zii42cLEA83oCqaQJhAE wwU+t5w653DLhpzQxLGCCrdNg+lty3W65RJrVQczQtygAsYG1LZJTj96loWc2wx0C2N8exbJ 8HJtotimxdYl9Y4OX0iv4n6mfAjqPZGpKXVNxajdCJAosYJktFLEGALn6FRRxOAEig0cfKKR 3EWNeLItlc5d5+oOnA3pHqXgNuATxvSWCX8Q1qLpuTaGpCs7K0fdMn/DDTp/Y60ua6EhCJLK ouVhH3cv1/xd+c4zM+pcENlUtYMe2qbN78aF1bv84zifyMSq83suezGCltDtNqNeWuRKyixy 7g16XJzgMl4K5+8ZrqMpLT6vl1+WVMjL4tapzctiox5RzgnuP6cgpTTjTgbdsDCBhrcNsRLW qlQN/WisPLauW+4EzJxINot1+gfw8+VtTrExXC1flWWzI3CCSjaIN4rDWf2mPE6IC14UPf/0 px9noysSeSFPeuZs5ECdXggg/ZFIF8usg26npujC1iqv4WFtkn7ufNgKH1baoGQPxOaWvI8Q p5Ga6nJhNVqOl9sraX8lvri/vPp8fQhppNvAA+YM8/XxeZhosF+QqBE8M6o8SHkEjkHLcu8t tCgqo2O4OgNmKUOu6ChIZEejPlBpVPrjW/rD8B8ihB5IdZszp3MZVqYWDc9+WJnAWRD771dJ GOE2OwhaWqGoM/9rfO4Yv1mOMs2GiTAnvSDzAjU8rv7l/yWcvuULq6fgy/R7ZOIO2LClsWaX 0ozlSSwljEBHlyasjuITTPlfQBGKXAX0MZoL3QLh+sABt9qP59fVZNZXyQx//x9/Gv7y7//0 x/oiCNj4cOTY7oaWfCVNqjC6CoMWjk9pDAXalEYiFZgpNt9JyU+DtbQcCLKWDgy2QirYTCq0 nykVWcFucPvNXcSaLDLOCURQUx53Tbc3JJAWDZ9eL38Ky3ABf1x4Ah/+GS0c+lrYnHI5Z989 1I+wB2EFxLA8XN4f2vBHxvA2rN6lv1vGrzxQCmLxq9wSw/DRCZwey7DFu/PWjMj50tOFv2U7 GwHkAXRu6GtQN2JZJUMH9Tt0/zSvM7lvbuGWupCFwd2IC3UR+cC5NxJZ9U7HoWMfqgiLnSWT YcC/jliRqm/kMU9dJ6AMDxwy6Ux1otuxmqHKMwHlSFMeRBl3J0rMyuPFy0NjCIuSRXKdHbey /yhDwLi53JLFWgBXKYDxEbRK98bnARn9q6OG02I1mYxfXyJmFmzv4XUeElmkLXFwb46ywMCr BE88E0EJSymNlBBK/ImLknJ0dRSSb6qs4catulJ5WiRVWcv+GCUdcf36fHl1+O7m8uU1oJmW R9/57ov/84SS6IGxIJU1Gs4fg8cW6g12/D6/TUbIcZJRdv0laFJZfL6nyR2efpzsZ6MV9cGf TFWaJ/+4U2BD0rBPIJziaL8sTHYQK4l+bQrWu5C3mXkl1TsDwSloGOclO01w4YTJaDywzjJ+ is/B/dWn24ebx9RrEcptGNaFtkYh4zM+XOLSdFqgcvpIaT0i00erf9lw/vtRSLO13wzftiCN d01I28LbBjsSDFfYdhQnYsdSWJH5XI5t4kamOEfngn3qHjliWi8WKwqVcKICeMaQ/GKQ8tz5 hSlbghivOqo4nQeHe9dCDku3yCwzS9HtV2aDu3p9urh8CTR9W1BHzMKCpEH2mTzxuoQ6Nf56 RRhT2Jj4QkFPlfwgKwU9Z6xyhSHxX+0tk0a3RM7qdlJsCD7PnmzMdUQrKMU1bBSfcbb3dpHY 5WUUf1bGxEbxfQifhpgzwZS3OWBlIglFWy0ojqKyvoIBYkxbEHiAeKls9wQsIOaE2vs6Aotb yjyURcySCbMoYn6NB/anW7yU+3ErSFRFaRUoP4lCy9x7r+evL18uXubnHWKIQrb0/i2sQQvp Cr3ZSQN8HNFSeyDfDHfGX/d3LSMktg2xcAOQwBwuIJo0tlr06DSEFSUuNWlixL/pMGM8M4sg xtXfQpCj43xtzxQnMY/TTf0Oqqe3V/fT22aLmMzDxe3NVk4Go0xiRKZpd9WzV7GvLgeDiNhX YKyDjOFqAnzsBZviqR/jBAAXP5vVYBAwgST1oWyEe64MPWFZm48uYVgT36V3BQoU6myiJGPM xeJv0QxkFruhsGzJNt0GIi/5KrYvIlY/+5ubD2hbSQ2Sv1hELBhd0H2v8puwhiq/KbYxO6E/ AicITbv8Fl5orfy2EVIklEMwHa3RoG5ECd7ZEzwLUkVKjiFVuM+zYxJior/jUsC5VKORFnk5 eY6GRetsyCleyYbig2Nh6j1aA67Z4kCCLHPCyE91g3WVSsYom6T9zbaz1Iug3idCCuhjVtha zu/G8DiZ1vJo54q1LYrhWd0xaD5wkx/kslYf1TzhTDTHRzbiRHP49+IVBlrRjm2zpHUhjI8l 4KOnEPa+VjWwCDxdh1gukLDKErKmuXGjRpUhuP+F8L44e2eI9WwIOF+USa+fvSF4/fb69lvv CABJ1rIKKUQcq+LK+8gk/yRT9W5wNtUq1nriNpQqV0u/urwPf108HXyG9Pnbz79jJUPon//t L/zHH35cpUgpeuNpZibQD+fjy87bh5rETXZKbY4SzDerUkwCnW8u701HiLlL72OKcGc9j8la F2sqlD6mPzBGJ2Dwjpl1vYy7LVbrxHywlhH+Km/uDYipv9MKgNHjBjd66GgvPtZyUsBfTh5R TssgcCDBUCvMFLf91r8sdi9SH69XsbRKbgchXpZmSz4oAjLEgI8Ds21poc5BGdLcvHDMZVsH Fx2oxBp5gS0FEymmEOQIOcfhFc7azint7REGWS7kCDHQy4gXpaXchQeAPWTNito4wXrAkRvF OKbuOzEBfIhY9YzlMjMiOUl833wb/v7wdj/RXN8eXkBwyzD4zigVtqtac1TTgtrV2H6ZsLDj MhF2dxc+SU59C7NTo0PZZn0STTIh057r9n7iwlE9/VKc3hrvl3ZTvrTwElGjT+nUVB5yGUTD av6QCCyDAEYsYJUldEkWqmhUwtPt02H+sYZ/ZEyiTz03zzuUl5enPhk+O+MS7u6SMtXWwoQm PNbs0UKpHcouq68VxdcKQAmZF6A1xPBm9i1OyYMpS6WasBglS96aAhKC6S2tRcN7AOsEDgEH rH+KiHWjtsxIDawKQbpOjAlujobSBQ8Vr72pFMOnCrpxMxjGWZfCDmuMs0UwFtytmOgzLy1J tMrOqdOxwjgnAD2CcWbbAZEBIp/WnxuSPPfIq1Z2xPJVi9Ivckw0dtJnEIvFZWxS5Vy+G6if LB/ZaWE+gCyltM+nfuLIqh9VZ8fDfUXWGlqdc1fOAPzd0KT8859/+Lfhj//zL//+/80oM67c FuRljFkpwsQTlEaSY3K4atkMXw8wiTYTwwKD9AFrwvFLndRB/ec6Is+YCETg6EICF/QxqL9k NLhbWkzOJDGlMOYbi9rG6DHJ0vJRFHh6nbwN2e+NIdeiwbUKuXQ0TtYl2hjOlDVRMk8wHRF+ wjGlsFfPLS9nCrL4xCToVxZRPt0+5pIFYyq2k0O0vXyzFiKTQIQ58wXeotmY5GCK1Cb8/qYF d2OJxlr7W5T0djWgBoR7CAwWDhkP3e7bh/+ailtFhgW63VboDXdYGP0QbBtscWHdSL4QpEbw /Oi/bsoJ5qT4+fL69pdhSmg+Hy7TJmyHwKbWAy7CKXaEvhCHOgnPxzE97xO+brfp9fLi+eCv 5MtXBvX8+vD1C1Zrkw3+vetvI4pHidClNZQNY71l78NqnNhGenFAfuAJqZmiOyNVpkQQNlZ9 /ccSB+SZ0D5GE4JA6peGjxgXxSvtmz7C+UkGigK33JXOeMmW8OUcL3GOlFydG/HMNeoJNWe6 w3HdX/v/hm/n2JDxACGs0EwymMDniIhRRRnUpwLzKUuB6ebEo/csuC6V7i1htCrOPlwoSps+ Wch+w9kIrQ3a1umEjLEFJXFkggtScLsqByXFk4j9XO9t0jKMyeEXTzgy/2woOrFAB9OEj/KS yoVKYeE7ocafOvVE4JsNZZ154INhhUJsdpNL/J1K11lEOAcFv1zGdSZepJsOXiTGwzxGYXe4 FYYxid5vO8PEBW8KQp8HgKUWZGWFGYnFWUAus7Iec3kyXzgwqSZikW8J/+L6BTKItHTLHifz HXDSOMgHdWYCXkHwMwnyOf91kHKlyODtbSzMSu5mWib/d+/tphtqgxSim/whl6FxppdRajZy Ixac7xzG81AUn66B0ebsy2ZqyX8ntM7ly+eLaQj+28+v+Q4vP6fGQXqRGRCsLIceIxlIlvhn yyMolYq3mVfomJRp4Ku/CoP/7Te3d68Xj09JIwcAANM6fEIGAv/kxYLqOOE1SB3MWdicB/YO 7FZCCdI0BSe4ggYrDlv7TICoHCldCWxie8eWtYKusIbj5jI95p3ucwNUinXvRfZk+le72DIK TFapFByYdoCIU7TXpb8oAcflFJx2WAjDfD61/MsFV7mtzzIJqMh6gNwZdDZxmILjdiqqt0Rf wG9iVWhlIIQ+x8BMULGUw3BG61dStmzpV4IzpA4EQzQtB9jmQtaLSlvAXt3dPvytlMxDyuLc GDTO3Igt5HVsSeqqLJoEqoBhqGw2Ui+GIoiupEeQ+XfGx/3Hj78fXt5evH2+Hi79X8+Hl7f7 w/Dj/1pRuC4booXIQ5RM2lRO0jwDMUraiiChMcY/H/KUwqEFr6VGaqBTxCSNs8R8zn32lcy/ y9NMlvuMBu8Qm1LkGCOsmCp+KGFFtlGrc5B4azaLaW7mqDou7QK+IC4tHggrok0hKeT5+wRu dmwFbkMmBM5ACa3BPBpLUZQWSWWhOE2O0tms9zHclyK5ubo6PC0Hyka57VT4q4xHwOHGzavv SXAR8trR9D84vGnM8AENmkawXFkHgV9Y3rzodeNWIrbJvzos+QJY8q4FQZSEsUXBHUNJiNEl lbDkGwY5unyxuJLJhieDFAuPiSS9bJCtDdKSVOET5jnjgD4LbMfq81UuygRlCiJq8JenuYJV Aoh/+Mc//NNweLi6e3x5ez7AC6PhTFX6Ic7Wz/8B/ZJl01iJakbVB0vRUV6zTQzLiggNWqvE B8FXLu5vr54fvwXEiwnFslp+moILPm35iQuTYSs+aIzCH8JKQ/VytvHAl6g3F3ove6ZGBsK9 UKwFR7x5v5Lry4Pfsc1Ee75MPqAVADGSavjbwlOq+DS4xTeh+4ZbfNjmMlNLC2xIDtrSjHUr SCwDeEMk+AwLxwQQ5YKfK9Vrg0+Gw19e3U3LZhzg4f2HbR47SSAZHOus93fWlF1Z8AZwD0B8 A3j1VyMj2X7RXVs6GfnlMC2hVAVbUM5ftR8i5tTU+9L0fBJynvtwNo+BTGjB+T92TKXldWpq dElQMlXWn17CX18un7/9/DtlioPYgs6tRobyGRMosd9ENDr/jn6J+NXEXBjji72635ZendGw l7/867iwViBeQwZKuQ/i9Jue2dgsRBLBmiYKwzsGUGZ02c3F4fXzVJdWrjqLmR4O1CmF3VYo X0GqLpGsN29ZX0hU0xMsjRU49XLlidNVueWJl1Au8MSVCpK5668LaATmUBrloMo2fc5tq11w xkgkDjRtIhGunFvq2D5P15N1Fkxy7kQ0F2LkCUkyGCUm3yxZtCCJRGloscqEh168yCAsA5VY w79JqcCqIAVWoNF6Bjpaod41x8yzYJxD3oUNw5tAwFKM4eQL1lkqkCJpuJ4vH346hC+cginZ TlimBaTvzj9xB20EoNcwFMYdbVbkWMZyvSO2ZCM0sAAWSgKiDeJwllau5EU4d389rOA5oemg N9mBv9YaFUbiLkqFvy/ddhyXMMrh4xJfJ1YArCw+uMvTv4htdRpFUIDh1S7bWgyvOlurKaPN L4mTlTLbW9zP5YAcM3ijg1H8q3Fc5mcFS/yxejQuMtcaf6/yQDdzMzBQzEu0LtUAzDkpppcG nFkogJd+bFORG8NJAfKu4vMhTgg3dhPrbMulvLnubGO3agIGYN+fSg2bZyno788lOYNzhmmj 5lp8QPHFP7kTXMWEwWfekXByhd0rs05Gs/YB7B6tjBKuMfLg1YgyhNYeWTuQsmUKvFoHs9tj EaNAO8wqN7Z648r1sQjsXuCWj2hxM34WZFICJpXsf0sSKf0V3kh9MTlSoqjT5Ip/9wwAMt7B jXx68sZpAvTMkk7+f3/zvf8bfOdaIYmCBHLe8HPrqHkQOvmFgUkAFPcEJrDsHv8pVlRAYCrZ Umh9gU0HrV+dYLqcBwQuRUXSrsB/Guu0Ro6xzGWsk0nSwK8vbs6yZjUvhMuyTXk2dL4xJDqf Y40hH0Tj74S2n10PVSMzjJFPVFqdOAW5Y3HGjBtj09CRz2hjkpOLwNPeWck7FuCNr1mJFE9/ WOTPSQ0FDwZHXVcyvpnVnsmycLlSagaku6vgA++LhVngvmVMeOfXlyXcD7HMPBd9USbgAvCX pUrKyKH2U0rGheAUJZM6suMPxodwu40rbobcDl9IPWLdIhd9RhsWIEKFZjIfzqhF+0lp/6jl XMcKtEkuSmqw0RvFpZ6zVNmDnx3ndemzh2bMSrUXZITDQKnNXH5usWdiwZlaiEYKJ4x2e/Fs dij1gI8+zMCNVTxvpjnoJ7nUePMHnm4+Y0as82keAzN9zmm0hKCY7MIGN0pxBeRDS7GtT1pA 1pJw4QEhSRQGUGyW/+EijLuGDpktSRlXY7DO8AUz4JSS88iQUNIpO1dBzcitz1birBpnqUu2 zM+YeSFSl3iG80h8zZN9n7/5fnoe07IP4V0sIzaRKUkIG9Mr6Zef5YZsbpxIm2Q2M7nYknaR Q605OlaKLDMKZiKoywRt73hBRSp9BvlDFY9SWgKOG2hg3wEFMTHD5aoxLCbkpCqtlAsAVM2Q JP8HEBQ8sILP86ZjnjvYTN32kMSTBMlngrdrOfz+Lbix6DQ1kuRwu5oWTaFLIifeamWT8We+ Tm47sLX4TO5jy0yFN0fp2l8gmz8NpiHbBjsBH6SxmxjunORXi8tyZszVhD7VW/p437dq9dc/ //hvP/wh6cmgKmlsVUSGPIjUGrZ8RmANrhzRxAn/8N0gBDEpVG7jsjNp0Yho+qZtmhUmK7Br zoMNJRZwzmLVl+vDl8lOLHopX66/+T58dnH9ekeVp8Bxrcjc05+jwhZLSvFUCCSQ8eOU/LOc uioL+DRMsqY+GyKEz40+q+98ubu4vvS54mGWTpxd6PWLN/4raU87F7CE1DIlR9zY6L+mz+Of SWk5rncyjeiW6rBn8ftL+Rv3+3AZw9pHwJ/hbaY6GFviyUxO5ha1sE37ApKVpDNXsG+QTE0g TiRej8QKJyBwm2VCa/wlIvuDNE8SiBozKNYsBxvwKKnNv17IVEogiEQiVZFZUOVlmXRYzRBn 9Atbld6zLs9IEmacc4BjkiyVZjYjL5elajPHCqpGcKxboBT7GJ7QFKppS4VqX8OD/seffxj+ /8eHr0wY/nK4u8mBYr7d2XWt4A3JmAmeSHTLFeg2K4osO/oArSsD3G7+V3p7uXzf1AWPD59b NUYNISftiBsgW462nOkWWY3eIsuy9EdsIAgJmwk8AtoqQ/gqlOfpIXxbBLZwECewW6X6bQwr QrsA9MZEyou9F4hOp4ywy/M7ad/8b4l4SLFQybzef/P96z3wltOIwrxaE60Ys2zMYEyTxQxA H6YMyU07NXBIppSJNZRdUkVrU7zmjdRyCUMtmqcSuTJ6Hf+oWM0K8wBn99p8QSd1eW0Y4OJA QEZiKNsNYykQQeAwPQE9AYvlaa4z2bBWed7G2YTJV5YK3hR/x+BNMTR4AzdrtWUaH2gLwqQ7 trKgQx3lDLB6vb2Gnn8BVmltAT+UjENTUo9Z+a6Mw4N9ilEQHYf/MomKyJEjnHpfxxFdPb29 HBpZHE7VmCUdW0tYoi/93c3z5U86xIElYfbtk/4vBMwjLUReZAXM1edlQpTK2/ggUHkTFMNu AmiA5vE0Bslwc9fBaqomfFZXCE36ElKVJn36TuykUABQiKGp4PWojrIr+UgBLBmMLSRfKAxU DEyxj3SWYt2kSeFzF1Qg2INmVY664nYPsAaSOM2Z9tPtQ2S+eHt+eHm9vTv0MH0RrY9j1arV FDvltbxcfT5cT+sYdHUlJP8Z2dEqEsCAX3u+fLh+vF/+cfH58vnL4eV1anROwn95WSGmenz6 dVh+S8FriSbHq0oXmDQlmQPJzn6yCIpJarNUWkyexUyj5kzkYXnh8jSXTSyY6z4oc6WTzeJT 0+2dcH1TWbU4vekXh6vPl7cP1dPjKyIZSEpJnZ7ec3oTfu2k0xvwOmUq0G/WVxcsnUgZ0IOT 1X0yFq4jfe5Te+oW7VtH/bzgbQbnJcEwGUVtUfLQzxco/LEvvLWeCqaT/Bu6J6bRy6awBf29 7BXLBzPFa0eNUC5nJ0kA53QOC5VOGUsg58GWwoM37r/G3Mkd13g7A8jHLEoaiHrSVIlKa12v TzBUUAQe26RjR+yVQPpccde0GbNMDVMmdQydEeixCVkO0mWDeOs34tsogYQuhTduEIGuFLtd 6uzFfIpOLkUiJxTfqk8vMUCdEyPbpU41LeVq6gc9vj3PM8ei8GCvF59/XlW4ME2FEWdjXjhi zrIzvSZnDVDPr89SRAyMhsYd0rawUZNv3dVNoCIA8/j9HSoh/hKHGe2OtIAOuTQDwKIdkXdK Sy5mbISEFPcAFRKS+9IMCRn9lvzxJ6zEtBdJ5VhROVAYSKO25fZ+cQ5Crt1UgKb7X2mgZHGq Tz5SvUWSX70n9mudD2HsNNWlQd3lT3dzxGfIa8KwdYDcBfL/g9SZzGMkRuDXG0JgDgC+X+Jn MunceWeJlArhZ7JQ4ic1lgYjLMFEz0riArCWn29nim7OyHgzH9KW/yJiG9Jq4udKUYNSHGP2 K56QHF31CSE0PCz1P8N/QItNUemTwxYCb4uj98RVgqjB8NxeYzojRbTEMe/ClkEeshTy3mJL yQQEIwAbZ/KLAK/D763Au56rpZBmBVtK+skZEPpmAb2AV8MPyKn2Aak9nhlaOAL8r6jXfEJC 0NALINRPh/2TrkSxpj9IoAAuvYNhzV1p1o0yB54FrBIsAUaVkwJ3QkGPracAISR5UNnQ5Spp HHuWNvNHha7zMn0guJAo/Icz4zpiBTVxH/QYuoxAtdkVAk4ixiRu/jkvJQjyzqTzqHuAxmMy eLigSGgP6Ro7w+56BWQgGJkUo2Ip0j22w+5G2ohOQA95Vqt/a3rCOoaZmgyHlBwWqCMvMBOO iqc4eYOrB9Vb0iPJeNmuOLOjZIYGmhmdoGHNLOFMBGc4ImcKL3psjeSmbWswNghO3Bu62Ekn 1b8+TAGekGtfWS8NARxr/GFSrUyfJl8e0If4opRpheOVAAurgWRoBIfDKS6DIyQRSjC0q98Z YMmaNdYM1Dl16guxUeGZNeMllHBPgIXtSnYGxoH3nQEb/htxw+c9w/EBVl4KOs1F6PekksgG kV2rN4Try9cmOEI9X1o0Rk7BQc/EXhPm2ggbXCIvML8dQKM1v42UVo3Ic9M88X17o6UVGl4J hU+OdJZWGz6b6IUNlvKRrdJUJerkyFWJQ30BtgpYueIgacD048V5pVFoWGeV11bNLqndRQ1W VMxus8oLfFK+tuDRAp/E82yo1ESZV6C81HuivErZeWqewr1JPyuG394plWo+IzvWw1/MWY+Q Ui1zImuDd1CIhWydtWYd5UyE/Mo5jThrn2uzXulRqiBv9PrybAvy2NNmsA4CXRY58Y/6xu3+ qKPLiVS5dzfmqVnubSS4As8M6CorCX7qCcfrcIR1wxia5l3Zdkf0q+rroGZDTWeO0lvtrcbg a95xgPUWxHYwFJjWW+y1tS2RCmWm8SaGIBeueMf1i7bSHfGiyWqV7CwR7alWmVoMM0qbvp7n XbKSwDh5N96qbNZimG3knXHkkgPmZZa4xAbDjUZvzMTTfnTgIBonRP2828SBzFrRy2lYXJxZ KRNonzOOaVgXysoNe8qbHTii7r1xOE5HkyUZmtitr1qFFh/gocDyGaDSddRhkV3rdrmqaveG /Y3Z4z01VjiDgR3MH8EwJCV8UQ4o7SycHXeHWWfzvNtTt5415anP3Zet1RGHtS4l4CthlqhO 6VOgXaLikEK7CqCsgODYSGA+JF4M76sEMaxqly6qEKBfLbMOh7IE+ZjVJ/Ta0LWA9gkIoXhm uDTjiJs5VQpb7akFtSwcGb90Qu73xC8tHBfhGdXuXK1d96gbW2PRbpvkVEJNB5lvPoIMNTI3 rlGs4fMOhDYjanXnQ3BhFg4tegCgJs9jPlITFi50NE4rejSvLqUX+G7hS8v4OnhtQEFt3JPe 74lfGgGeAGsDn3PSKZ0Acou+AM4vx68URqNQRK0FgYtpP26re6rOu2IGEg5+dG8r3xdk4kxq fBgkzxn186otA1/KjB2bghBYSAbZAzPbh3TEAQkUbtfZQVI1/ygs0MmSuVeSKVzWS2Gjajez yLgBW0qmwDCgfw5IPKWRBNKhw1V3TH0gS1Er+5unPq0gyK1UKWm8J2rAlgKDWhyMQuoy0vf2 5vH5bxNetQP5ASRA4z6MIovf8Tx85iMs/C1zWzLR7Sp61HvVBRId1s/eoXMjtgWYWI4LaDae jognbUhuncHFjZWkoX/tAkyjn0Xh84+tMVc7N6wexAArl6nu/Hsmqi/CoY9oRxBT99DcQFeU 12koDZqSMGZ3xNCKqaiJwM5JqR1Vhka9A5MMGSoAY/rW9FQZWug7alv6Z6P7IoZGX+LDqgx1 8DcQ+WFg5IUT9p+h0qe9RYaWdSHqYnilebslW7c47qoBwa+HtwTWYYjHvKtbQ+AzEc7n0UIr l4QpGLeE0pjGrVxfLIdljRlurQGE1ui8FIPPrDHZgfzu6HVm2ALoXFk4UAKh151yY5uoxXAy UmiZWQLN2zuytqfqguEqcgClc6mD6zQLahXBWS4EOhnfG0Cp7b1NJk1akwVYWZbQsorhgHjB 0CJdZ9CCQSoyNl+DoCW1OcPAAjFnY1VFx+w2ayofla9ShbH9woHtAEojzzmLvI5gjI9nVQuf axAlXRxnuCN+qpZSh/XoD4zEO03uETWgegpg8Y6RpkarKxIAnS2sRoMECMbDFEpRQF5UdaOz BnTkZEevduiO4BLrCbPtNxaf0/K7J2G/G3VvshzVz13aWY7qbx2B1g2j1CtOYgWozwk4tDLG hCTInytXZTG/jBXnc4564VnQQM3pwswByYFCuXZ5WzjulZhA51x6O2qt4SgiJyKovtqB9/a6 YNMK2PgaB0kA4A0XWuGhncVxh50dtVZHmuqo7SL56IvtsAZs9IgcEsVwlQberdV4TVcEj358 bNdqkBPXpXdweU9t7MiUUVA3t1HkqBakanMbfsNBizxTEkhDyK6HqPjEplrdCUQFj2Ud6SdD AmZJRH5n/NKoeAi8ycdJSaXWtamEL5hPArd2VXyBUjC7m8LEtWmOYKa9cAY6yNgkDtRQVKGO njOn9qV/VIyo7xKa6oR7bBelco0OwO/Tl1lA/JZZPAdH7YhsdgIqRQ9dsbtAJXGYhlnij8IQ N4WjYxOdbhpL2HLXSAChYJ3ftiP6I4x3dI06QNWIk/b/ZrAreSlCE2ks1yfMmGNLSUUPxvK4 O88q9NZvEM5eNJakvfu6Rq2AgUjveyFsuA/QqgPtvc3xpTB5PsxvVMLuSo6zcHJVCq/sqcJU O2r+lnDMHWlpicAOV0muRg31GDMvyuAinkyTjEEIO5j/59Xjw0v8Z7g4gcFvtaD5l4Yvt9eH x7UjQAqsXMvsiWAEoYhMSZRSQ7sjCMxh55U5BZKlXGU1gtKUxM1wuZ5AFH5U3LkeuclrMyRu lmyE9hZkqtdIWDz4NMSkbi2KWMaH9Xg02dPettbx3b4GHdcKHgQcGXlKJ/FrtqDWp4vJ9BVk jqRGJIHWRwZWDby3wlH5Uu1GNdReEB78Lsk8Y2PcDGakzaKMOhNlewell4NbE8dzi/csShuj CnKwvKKVd1hGhblNJpYZbZMkgLYmLoMbydw6b2LTT8qNeHCzWZLqWhJqiQeoWgqh4KSuQwu7 VKvnMYShJV8WzYA4GM+EaoYTDFA96KWOOicWh2qAXpIZIaks0QcMwnfHF9FqOYvkgPpP5JDY UABNNbaRbh2DFBgofbTghuQCiXIED6sYT4jOWwEx4ZbeY6auyovFNNiVQeQ+GKMGMPGZuvJZ 90xIpRSFx/yImzGL//q7EIvBTCkV17lGjApdygte3l9rufz9/uni/vLq8/XhaaqPFyQkL/dP w/Ph4frw25fHt5aGDYXE3k2JMjNKBK6BI+AGfLSg4ilzN1sr/EEJ1hx/qSV2WJLZcY3jrvSj e5dd0Xt2JQEAg0YNoGdJnIBGG9zi8XA9j87oYoufZcbyxeF4l+9S6Tmw78WeqPZ/MpS7nAVx 0O4LXAY4Oy4wozqT79BhN9V8zgICSSBNrEYi6JOyK7apVYQbrVKC/bRX2HBPAfbYYaVjmZ4r +UqDx0zguANyzI6eVO3JV7ZYX5ijKKCpJ3l2XhQoQ+DlxtaV2SUsQU1lnps9DEtdwLNicB1A voscn6LZWPEL010kpzA8+0HQ6cJouaujDdsXkIYPTEYyihhWnULo0JpCJx41MZmJ78zm9gq+ nrd7vb4I6jeHl9fDNeK2E4ZeahCTZ4FbrQ0hast5kwpqFrY9qkBDcaGei/WzAYiLX+mjBo1c YO79+a7pzPW2aDJBwLYFYBaz9eUqIdJ8bIxjfkV7Kqdavq8dkb8vKFJFKpKta6uB3Dii/kYc 9fOxUOhAoLQDmJfSRiXHTOXAEsB2sE7iTUGuNGl52yFnC3nwge3+OtRrrYKQn5lx1CMiA/HO YjRWYYTBHc6cLiQVxpzAb9FyAxSxPVWLFtg7ahc7sdovLEQDiwtaKyQrbHP2sOYZqxz7PnkH gWV6Q0LbEY/rgg5May2m4xEhCs1CCniL077IkRht5k2+/0r9rsZz5DMhmS+qTdxCzFscdCnM 4JiZvvodlqwB5wPAtTxDeSQn4kuFk9P21e+wXckFVgtUfHPXlI2UQpkSJ5BK5KXkR5v0iLRN 6AsuXELaGikNVqQSTOK89ttauOrJRzY0gIxZhrkA79xMrywhsSK9L0PKqTQbAS6D5YK0Tjwl GzvTRE9yduzQn8JNL9ediEUCgFA0trEIHIxXRwyTHrMQdDA18Wef2cb9WxPUhiHgrtNyxwsl 7gYvnjDYqyrJvlSGg1peeuc2yD1jS+EW7V70ojOqtSFpHQgfEqmNUpZK8zW5lomL79st6X5j EiStSnOoPJpicsk4gYPgeEq7wR0UjwpNIzElgDEXYgKJQlz5OI5oPDP1N0+r41VDPQ2FadiY 7pMRxGAKPhC+3R/O2vuTTyavQTq8yc5j2p/tYNMkX778LQ7ucLF2EjeXL6/5NzSgr/CRA4ky imEYPbE9wXkjWch8jgO4Y1yQw74km0w7OD+28Hr+yZ1WDRgq+cKmKok2xUKu7spra6KJmsw7 N09sC7FCJU+7Oqbt5KlBtanwayLIa9KmzaKD0AYXBxT4yyQhgSUEX4s5Cf1QdZRaQFR9Ig8H EU2xGIE6ym0bWbQjLLQ1Sct076oUlcZX7DW+4OoY/OrQE3m7VTWI1YCB1rg7TADaNRFHMQZ/ WoSGhelQ5upoYSCDyD6xA09bp2NjghiA8EHh8dkur2W7xoAnZbN0mhAjGmJ5/9WGtXfMTGK4 jFGDC5zrAUyNxK6wtsgIaWiQpaSpFGYhhicZlyDGTRTGcdplGJAzYY7mFfhANuq6o9YOBMBJ rKdi8xSq+IpAZzpsHqhkxpMRMkNnfAgVFZYCYgSHhoxNiZxa06CB7CTUZzmjip2tpnoln0SA eSmGkqPIwjTedORiDcfJSHm4EEfnk1XADFcrFEa2O8rgj7oSgMd0MugaHtk0oGi634G0ytbM rtAu5yViTEsxgrgsip3KuYDZXmjk8GKI7IUlo3mSFmuvtGM8W+LBjNivcdeBUGkwI60wX+Dz d+DP2vZTIMkxbO0IgOBOQqPp++NPJyFUGnAZcoK+88p050lYgpKEv5gEYQNPRlCJFE4U63Al +mybFGixqy0JKRaAhQFJLQktPQmDkS1vvqBpVYzj0TcXmmpit62daj/nCCFLJFXMOjUmyR5p 5eKnVehjoBOLlgWRI8LcrShdwijJcW11jrsCRiCjiWQJHpTp8dbIRL9PpDO+1SW+DucsjjsT 2p4yuNjgLZH4vvBdTZXOIAbhdUwMNj66zXOKYRgjwQ1Yoh0uohiHtgx6q+KNyKEhr9e/Lcnw BmI9uJjfHu8/3Xb1Lai63X6tymYU05JhoVhAd/M/pG0RRfG5e1scfkSGaKxXwu/OSm+jfLeq m0FZMAqh0ppyrUQwLeUr6sbsPqZmANPKZamaZuegZG9NszGZKOAECOS9JJqj8hRWJIwvHNDc gH4by/Q/zOA9pUGcQnFQow30the0k2TSrmejI2S4nD1hiArLj7LgtwPQFDvmpVjcsjDbwSvQ MTmKgTAM6OqLMTf9NKENbE5R9qjrZXKiFs8tVWI4ge2y1Z2g5rl2ocH7qlKNrgD5857p6+pr 3gMrkDgykRNVzJMiqI3ddyPkuMyPyIksQMtLRgwZf8Jv7lRkeL58mjam44wQUeBhJYUOPlfj bhcEvTMrZIFbKsXZ1mkoeZgGaZm0BNPZZuhudwGm/rTdCkWayTpoUgxSRTSelTqSXUEI9AZn BYv+s1qsjLas45jQ1wT5fcHGqf0zOZ1RXSPiRSVPJkwKsZ5T6lJHBt+7BRva/fs982SA6QFv Ig0n1qWaE2UfVZhqAQn2op+PbOA36nQrslhwZUkG871sa/3ekY8Se8/MSUaV4HchwrHyD4J8 zlZ+GCCFuLLE25Edg/Ed5UukFyxg/K8yx5ghup6M8XbXc5e2H9DTk+BTSARNtKUJNbC+oLu6 FJmYc4YJ05t2ZSTqc5KfIIzTMPkUrzA5T1Ec0C9WT3/dPl5cPt1eTVHC+oRur94aloSYdiFS kJOCbYZxSqRqpeY5d+YuQrAZl46qVqJz573VykYSwuEewajhWG9cq1Yi4/jxK40SGfroc/dI a8ONwKty3JoTtHlawiYUgO0YQRE2MtM2sug6yHbrbsfTLlQ6JNrPF9WAJhrPFDrSEoqYAm2I 74n2GzaFESHKDoXOt/vLi6vH52kdowvGLRzMshI5ulZNkOChGvZEj9uOjBR2vRJ/dJdXry/V 5IfmGtlfXr+9n23KVNgA67gNv/LcU16niML2ZKpfbl4uwuaEHTFTMQEs5cvD4/Xh4ub5cLi4 u3159cth2olRl8uR47iKVNbLkUGc3QLW577lsKnkE7bh/qfn3wVc3eH5yAnUTmXF+8drb+hf vvvt5uW7b78N/726vr77LiB8nm8/fff4dHh4eby7fL59+e7t9cV/fn//+PDdzfx/+O32Lixa TR7zl9/5//n40zff/3Z30Z6Hh5dq1b9Zv7HcLLeNN2bWx3j/6e3m4j/f7j89Tp51h00Gjx/n 35xs5+7FHDmS3y9PUzU5rGVwiDdORv7172fStr+fQzML51pAPMXWVQ127Am8+F+79A6gsQtn oSHMkZxgxTF8uR9uH7zFvf1yGMJv64DpnvrAWzfj6WZ4eb18PWQngAinDqSkYCIJ3LcMPnHl 7rigpCzzHicELK2YhuWef/6df5qPn/7TH8vr4bmDZoX6edf4wc3l2910P7Te3I8Oa0WEJ8Me HzgFB1/uD/dTn0GUyzjcD8H7dXhkKsrvrUdCeqmn+6k4qnzePZ1N+N/D05dhccSrnYbfxh1E J+cL7DSp6sgbOdiozMlPFyPwmHbsWM0xNrXfOxfFHQI8chZXQA7Z+7Qvsm9RS/7hU/7J81/e 3YUc7UMtmR4LSxbCpQ4bT/3cGVqfFDBde2sTDlIvEdP1wzffXz9c3L/+Qk9TD8EL4vmA6GV3 wh6Z4YZ+ZO9l/Kqu6C//+O9//OG/T2vw13Tc7ISx3I5UoMIDToGL9aQyugZYPnZbA+y9weHq 9WK1HQL7SuJQUvzSmqdcvSQV2JP8aaQFNNwxUXOKEUzfNgSTETpAvOGGkM77sHZDyOebiKAP NShFsZjYj1q5IvRAdCOI3nUgWrv2OhjyWtk44t13Hrvva7Lk7TrqWeHbzUt1G0BFdv31ptPO br++PJIfp8R0gGYLPRCqxxJ/d+uWnmZr799CiLV4qMXSvrw+v/nH/fwzOEVmcs0wC1RnjM0A xhmFYUQPmpVdiMJRdtymvEWAh2aFKFytiUiNSpE4PMAq8rPwC40AvGHt9sUNiu/Zji4n1JkI nMN/ezfDKP+9x1Wc8CRHiZSturMYumx11HKEVoUrv398e3hduVCMkkCMBGQqWspWVhkP8y0d ZvjpZv7rwu/J08sUKvJsPv0J/nx5i7h2SNgOYEISgArKeCu+OV5OruHxVunYmm+OrMoQWc3e gGvK8z69Xv505JM7CR4k2BzxvN1e3z7cPA6fL18+H5n4fsjTv3+7mCbkGZtwVujj37Xs42vW qrhJs1cbpke38spoPZ8TGLjjjZG3G9VSSn5Z0PDAKAl+zla/7Kj4cYnkp8clwnR0OBMfuYZ0 5h/eXg7P/+3IBtgeBurX118vnl4nWaHyUPwvta8G9XN/SgyzmYlr7pffhT/8AI0LllMODr+W vBcj2ml7g8TH7jIbxw3fmdcWZjura0NTL0sUiVWnz8rWcLonUzlgbMbXeGeOWEqvYV77h5k0 F6/Jol9P9UlJYSVEn+vt5VP46+Lz26cZe1HENYfPV7epuc9we0akfdHutppwiLMWUsue4kgm sBjBQGFuYvtAGxe4m/SWjx90bAkF4xaekUj0U+bnkLUAoWIBTyfPDfPREnnDKTDnLTiGAfCj g7zbgMLCkfBQGlXcRPG24KFUg3/PmBj0wZaeu28sZUV6BS/LbkBxgodK23FbtkWlQKkHTEqq 6WtJERIwWmT15efb14mVZqcQA7RjxCFR7Ai4d44pL01JjrW+ABwSkEWyxLOnjGV42Ctw6cxN oU3IjneELEhYaF4SUl+MloIO4aZuW/njJBNh6zVRP+8mV1oviBb0rO8Q5BrNXpL7e0xYGpzR frMgzY6Eo7MVgwTAtJK6gySjxXpB6lirc5YjK3NKtWuEGO4J7B+CzxnB1die3ZIVDGfDeRNW x+3emTZs8kgSiV5UCHFrKmJcG/ZnxSEnd6b+VxzUZdIxzb81UH+M8d9y9IhdY6pidUzgZ0rQ 4yR0a2umgkLonQgXVOtz2jEZRCJp9uzOHoXl+mlZAQoMOTo1eiTiLdVFjVXRyEXiHEABKDSM c7K9MdR6aKbRDk3aqg4hMxBCn/dJcEoNG6fp2qG413IN58ObtrTt6srBdH4dycz2roTWk6vG xVKDVJslKJb3vsQZ8aZsZU0ip7YrXHGFeAVusgjr7uyypktTZX8SDBSxWCK3CWhdiqmrmelW BBtat+W9rHA/xF8C+Uw2aphEpEeuR0Zkdk1OzRrjff1Rc4sWaTJH4Gm5VOfUgz9+mLqkXEo7 0g+QgyAd/OkNO0eM/old0dW0lg7u3u1aVBpAZ1oA32SSXrr1aQNuczkqFgwHZGqUmo13JIgA OKadHVDTfFVcaXKxq4JBCpMi8Pw/0+4oR9VFyC3p4EY81iV29irOgfKXEyCmjfLvyvL2tpo+ 3T5OjiK0valOLm6UT0xhzjIfocz+nTtHgRCmF2w0AsVqYowQJ0c+D4fXW//Xd69XTz5Bv3yZ 2i2yqO2/PF48X305LgUlCxfvcH7ClvMtt6+//PS/b//Pt+Gf68PchYP6yD+EZLr7D/EuzabT uu/TVFbIdaZuhGphuvaGaFELp/Fn2NgjfUx3k3zbx80XboDQT7dPM3aTWQF4JuM3ScbwaGyY z1Jb3UIonnSWL0+XYdl2Giz2VuDp8uLh8v7g/3l16LGMBPR1V9j0BbSBWVDsSl7l+vDl5kjx dZJLeuf5/e3l9qcAiTkyJtjdt0keQpWowG5UHsFQzHpZPU66UF/8mU0bqWe78OX6m+/DZxdh sOmUKOUjcEXh+h+evdX4aQoVZHwT33wff+X2pesPQZjnXqXu85yAta0TwB0khezqtMvn8y1a 2H3grr+fuyP59u48Qs/OgEZsXivnI/FnmC0q5EarJVzLMAHjrrAiyzTBEGaqh56JxxNRj9mw XR9erp5vZ9DAEoPf3N4dwsfDHDW8PafFSAQlpZzGfTVbimDgdb3P+XqXuOBH+ewu71+evvn+ /mX9qJBJr+Hrou/vD6+XL3eXnyb/Khm99HfJG+cbOQ/STBH8aIjpmvn/z7iGTO5Z/c9YCPeC 3owLl6j52M7hdL9BF79NVqdnYI2KFndaGAILd/OyAPWCsOwZB2Kp1bEjV8cbkTYe7xMYz/1Z 9MsM1FsSeX+Pf6NSpoQJUwZMYLJU7TOjIwJFjlYbe6v2NQ0JwaRDDkpI4SiOA3Qreqv2GI85 AJAAFo/sfIQJFQpsLUq3GX7pqj2yFgXogABWLeHjgpgOISY0NGWWa0Xy6r4oBp1Nmqv2f3pJ mD2Ubbi3Ot3gmmfIfQn/ITVHjiSDboiUa2iWAS+ypjwB2fHvKJJvt4QlprVAWZCz5tTsCnpG xBRumBw5vkhelYThQiCI+amhTR0P2UDuqE43rspZapW7YCLIpUnS8UxwCy5KYkTTwli0RMBG 3XzTFSxuY3MEMeJEoPfadreGxUXWAoX1DIxlsqi6oFAZKLqyE4tblzRyFmZ86WcfKlHVRFqf sYnFPdLS6U6sSq+la1HwUD/vaR93QoIbOrjUADPVOKbDph5IcE14kDMHJ/6TNbbeuBJzuJwW eOqABNc9tDA5SWTpZy0ERVWH8yvuweLWQykHBcHTNLqQQaMHPSpBikz3YnGxBQGaUii0l7D/ 3FKE/APqJHdBXxuXmXjeZFDVyVNQgb4e6y53eYQ97rKxIEOMi7jO0uEe8Cv22PPXc4j/B5+n pXSRSnQCPI89ptPgPJVjaun5EE2bXqroPTyzWFQO7LCEiQqkFCf2BhVIrPGpNlmr4eHg0p4D SY5zKta0apLZCCeeRIq4jKLwcaZrPRWsKfKmgICCBBkujEaNpvbnFKxpVUWYj1DZLY9mCUOx rltaSKEPa/oRtrgPa1oPiD8Sa9raE+rnPdGf/+Ei9KHngflivjX8wvzjkSvZ7ZiWFqeQRSU2 djh/en58y/PSOCSmMWbaYheaFvN2+3R14T99PvzXlP8v09FvD08Xd7cPfytBDO+XKMBdkZKg XFrvyvt5yWJXJB//LnZF4LvSsRZCfomYBcY8NewfmPL9MN7SfzrHZpSnYku+vRPgW7te8Ovr r1PxXRbz6R0MC0ygHQCu5Xv3ZydCnhDZlJwru4h6iP7FB6IXjbEFbmxq7K/Wvg9CSPHD1R7k xJaq5jZieIevz4dD4+sphrxOFPHNxE8Q/57AQsrJfVihcz4AIfAN2GcD9mBv57BvIvhnmlv8 61GaDEJTLLnRFsTs7vbTtILL5yv5fPlw/Xg/oTPWjyn84gkYDeoVbQt81XbuTwEO+L4GucXk 9OPw1z/9OPzhhG5yZ3DXwq79S/dCznY9F3+tSi7PLgKeE3nPz2Bgpe5jz+85xSPp2jfXya/l 4u7nT13whPMyx2+Iwfcs5cQr5Zfy8vPl01QimVIoyUoaef/L3uh/eEis7DF5wjuE5hNlEtyR sKL2Kzvvdd0YHfyOvF9UUkbGJblnuSn7sCmnrKTMFPqO5z2yhZKq/GtclE3idA5j8m7l32PJ GHbT/neWObFydD4fB+fjclnPCEp5TXWpNlXKnJgoHRipX9Eeg3FGqiZNtw46ypzYWuBIPbjH +WfBCK3uIDp5Wpmz1TqgZgV2M3g2y5zYzsBvpNKy3ex5zTJn7YyY1EDEnCVGIy0F0TkNPCTH g/Nqu8KFMUjByEdCTr0HOK/6rgXToO2WaHEEHwnuRemaRHYVcN52LTyfCxtBb0nlhFJzinRa tgCUNXBe9Q0pDrmKU6ddaU3JGaIUIr3gvMZ7VrjqpSCZ7EgkWgcqDrm7OoNcR7AWnu6O4YLo rzPX0lasoeKq1oWzlQIAnNkeV78178spqLiaSCpjDCgm8dR2U0IY/B052QS5VkBoLZVUPJgh /VHTvtRAaNhaACEkoLMDY2OcUVrqNDdaG4RWd0ZOwXAzhVf+hlLh5gkgtJaH/kBCyHpgtxIZ gK7T7Q6ketBfdS9tJcjXEvGMUnYkIhdHsjD0oL+qOyPtKmSIV1eZcSRYMnDXuAf9VffVCrb2 E0VQoB+k3hLZTu9Ff9UXNEJS7ZQUcKUp4sPT0V/1y8xWQjk5g1JkRtkH2KugvxoBsMbr4oy0 OSeArY7NCt4Nt+2qe0PJXGXJ5r17U0F/tfaG+nn3MSXRW3d8P/kcin9TgebFf9fhdcIpFzW0 y6urw9M7C4BtN0Vt5Cj7N+Uc0J3JY/76cnV5d/cyvaWiYLTVMUHXQqlTUkNEPWtRperSdi14 hk013jozbEjdkup5blHL9B/ePhwLCeksw/svf7q5vfvu6cb/N1xVW5QSn24u/nb4tUMv5OwL KVWLtgvBi86UoWXleqpHor97u37S+VCsmw/FfwoPBb8V1F50mlZyCbOIy9PVp8BYccKZnGLK SjmUNTDlXZhg4ivx2wGr3sHGb0/kXYzpaa3HWeZgIdLx/+ub7/3fLm6+DX9/eLu/OIR/y+Gl o43xThspd96rE3czLuP26eLx7fVpAkiH0ZOPOs30vm7980orYNK4/iV8/V62Vep8HHGn7Zxh rH/nzka2thoTf/3lm+9ff7m4enr731f/59vXqw/YjG2EFRBwR0ZYBg872X4m8naqIqsedD0X gizlnHMhrT4chcmjEF2njhtg6wF9OEhEngsURlDyYvpUamuJOIRcpZEr7aZcn9Dk4Dgt4dHu w2FrAbtE8Jwxm46mWMsJCaVClpJbglrCKYz0sxgFUYK1Xcd0Qjv54yYf6odEoqXfYfIBW0kq 2UstspVhacBeSyZxg8cFKhDU2RKs7QrPLS74ObdcUbpJ6K70tgSrJkaITCo98Cz4oiTRPtDN VmmtJYisJTOOsxGQVfE8QaR9pITvy0l8HdU35GNUOAYXTaDSknrTukkRXGkJYmuBxXH4ngEs gmwJkh6yoyW4vbsstUeZ4S7fXZX2yEpNtAS5JNfS0RLE3hFQTIIyh5BFn6RxPkEypKqXx7nO BweiCKOoeIoWt+roTmLXJXcnFbguPCFnlHI4YSPDhbZ6u5MNV0QQfjOzqw7d2Z3E1gK6k5Cu A3YnqS5/Sf21pzu5vbk8z4uyEQyuZ0XBcHGpPhw52t/uTraChXOkJZ3dycZSLHFZ3DFxS7M7 WQ0Y/I0HoUtyUlpIApO30Yjf152sPelBGgfkDXX2RqROnO4jOqh0J+thg4TRd0JDcCcIUYEB Dxv2dCfrC2IwpkqPm2tHsHcEpaATu5P1y8xGvFavxj04nl3dyY946NOF7pNle+fltIYL/E38 /VsoI3/o9M4Gvy74acvYfThh72+meaqZVHQ6i8DVewB1ocCNty3G+EhAUwlb2A5hbIsT9ery 3rvrl9vpbxeHh6uLl8NExO3YXDjzH51CjHmU+vF2CAYbZcIT2L269AQH5nHsxZ/ebubpvGnr rj998/31p+OR9sdd6Y109PZhoRrkJBqYrX6ZXAUm3TwuJfOGdPP7WRkm7VFWhqDhIUlVTots auUqtiK6Sl3sQVnD8T1h7KTIBqObSctSo8bqMspaS0Q2uMrXnsimuiCh4axBqr8KpiSFSz5Z AbdBk3kW17BLAbe+Q0xBZFoOtYwmdqhTtLgSarV26BxyCbvATg3VTsZwELeituj04O/YS7R7 JLcv+MNMEPhaCilHNhJOaD+1duYcDcJdl6dFFvnRw2DV5z64Fe4zc14xio725GEw5O6Aeolw aBlnMOOet9XbhKp3fhSohg4yWSFODveczHl17GU+f+en8cI/sPNTD3fgoBFLQhD+OVg89sL1 D3s7P7Vd4YpJxFNxM7Jenr9dnZ8PiStWa9l3bwGzqIE+IL5vIZ3C7606qQuFMUznrooF3dyk kRnkQtwultPOLlT1jKSFrM6pzKa0sNQgLknG2NGFatwXhdsWQYIjTtE03bwjFjQslh/9203x A+NppsVoIfB3xJsklbUuVHVf/EVDOSqNIuwLXsvvbf3UfDVjGpRveFKlUJISNnWmOeBZaf00 7gvBEsRIbspTWj/IWmDMIGF1OMUPnFHxOKP9dLP1U3WMbGToHCNzZJODBos0Wz+NE5LozQ2M zsRSil35xerpr9vHi8un20m+ka9P6Pbq7cxOaCd/k85V6K1aIuhDlt8fL2kLvl3yyjCpZFWO qPxznsgh1ip0Jr1IegUnhgFrhbBSICxU6j5qUmnRSrPlbMXLT5/ebnqmTqif+zmG6tqdN0sd /t2fBDyQ4kEsim1xCRoL3qneVdwJ1SJ4O4eGqJsZ6ZLk3dPz41OPvhjDcYGCmiptvWjDyhGE LqIoAkna6/w2dmWSOVvdpnCppx8bbhi3cInzoP99z9TisqSJKm4UsQr8lXMzrj7qXwWv32t0 FYxAo0iyJ9tYhRLsiL04X5K+gqTXZC0BCYngoOKeh9v4SClW82YAQsdCGGoKlJZWYiQ5FrLU SPEpsdDfkURLYykGt2B83D2y0NMsqh0RkxyyuicKDqUkYcgYWuzvbhZhOwOaRWCImCVRR5/c EFdXdCqiVJpF1QUJCY1rUr7zroYi3+hshVSaRfUFcWjhgEQLCS0+fUi/fpmpUEKSWOeTJVqw BYFLvIr48+c0aLQ1blJpPbRszolc4rsr/dXLM1iABR+ylzCcArGqrvVUKv2Y7cmZulgJIOUA QpNCYrTtaVf6q66KaQgezTZRjNQEHkoRtafS37o7J4I8dlT6q9DngU5hKbLLGqdMvdJf9VVq tKDSn2nfTFImKfbktBmP2q5wqRk246G15sfSMtUq/S2nAMxcAqoLJok6T2jrHl9d366Fm1yt HcG+iKy/rJwggl+JvqPO6nr1DSkBNVlTl1NpTVSRgw7l8dX1xntW+CsSpJjBKTMe27ubqD1D jwywROmETjdCE10qZk+Z8ajaXS5XEm+Q9o3ykU1qvkp1vfaOGNMM0uElwiqtifkkZ0+prrfu C15g4Iy4u037UquuH5s77Qav3t8GVXluiyrif17e3vXUOk5dSKuC95d//Pc//vDfP56tgI0S nXhPWvYjDGpzxC0ZjBKW1zNdnCmZxKOmucixIC6dnHUpvtwPAXP5D28vh+f/dlxk0qtG3i0b bxqy8Ygv5FKBGCEiRgbOHREjiGaMUPOFZrMWAWIUB2Jame2/FpR+8Em+cLuWVWotIKgvU6Ba QXWayVi7wxdia4G2jaBApajvThIGNxv/o5PP4YrnDJ+PEZnAAn8vbmctih7p9YXbtYB94asz yp8bTaKIT/GF2BmlGEFYEDvxNJGkpWTofWHegp3gCxv3ReCx076CWa8vrK9FrNoPudM8GqrT 3Jzs63nSESTBwBisy6G2/zwPH7pdOvKd1dXGCZ0FodtZXW0sxcCvB6aQ5BE+qbqKvei4F1xk xHvwRpFVaVTUGB0/bciw7gIg42huDPjolggpwwTRidXV6oLkSlE05WdCKSrGPX3IsHWNzxHo 7oLiV3dICDAXCuvPnMqOxMnl3voOsVW3EaSRpCk+mZO1cWQaPzK2i36/s9xbdd/Dam7iLEup 0BW0LjIBeyGi4MpF7qw8V+/x4GBRPuMvjaSGbizN1t1XecbWkytTK11lgH+35DU+BWOOXZuc sXAB3niecpaKYv/QXXtTqTwfawR3j5I0K8/1B0VXnjv5CXdUnmtnxHzGBu5vmgkwwhJ9QGZb mOFa5bm2K0LqEakMCSvN0dFwrfLcuCuUUiTZwdkJb7N9w9UnAi5KslifzJSq1720tdTPnStp 7cdf//zjv/3whz+mReQH0vNs2OqXa9uxsDI+6Ivn+5BgB2WXUIt5vry+/WWYZHA/+9j4yGM5 Dm+3UQR7vXp6/fnIvGQ3FGeuWoX4YEpjA6Vz+GDwhtf/xtfD/fDyyxqDOP/pBV/TgaWCPP45 S78yxg/OOG++6BovWnMJUvTy+vx29fohUt2tC95Lp0AIuos9PmpPfIOJdufrbVZMVWltRlOd frzStyO+wdYDO294fGMUhcogK0kd8Y2txjcM4NGGHOtIkvXN0BWTvvimJbD+cZ11bGcQK0St 5IzxDbYSIAQFMnGWkBDaUJ3S4A6Oj29qu6KctYjNNlor8R6Caq27Qv38HjN0VRsjOJhxyUBX wUdCqEC2K7KVzgayFgZ+UjCfSzGgoRAHIa85vrNRPSPFJDiX5FKVU5x4z3Q3u6Oz0bgvmBhg 2CKKpeqkLj+yljz/KgyYnRYxTxHMSYKaT9HySx2djbwWnAOa6GaPOexZ78sJpRHA3YD5I8HB eeU6iffVFE1g875UuizbM+Lp7XA1gvti4ztmXCXypvVaeCgxHd9ladxdAnEgqHfUtHW1Lgu2 FqKvzXOXRVPikZwsVbe7LNXwhY2rJ52WyKmqeQi3ju6ytLwRpRJALAU9oc4uS2MpDk/8+X44 U0+XpXZEkjtQA2YxkuGCpZdWRi/oxEN3l6XqphUDzyjrSOpxJMoz3J5MeFSPG+wKw57ofIwd CZ90Ooa9viANmxrJGPvj0kRQJU6mcqxfZgZxyLBqTWEj6DGV3qbGsWHnO7H5NPZnRLCuYTHk BPwJPZZjd+Y83Hdlfe1dkoIW51yxCIYW+TRR5IuV6tYIZ/IF14eby7e7gELT3o7+QziE+5+e A+rq58vbj2Xf23B8FjshsRHvwdaHWWX3TkSvOMWWcR+uD/6XjnTLlPnADGxxFFb0Yt/OeBzZ jn16m2TsWDGE6TdjaNNTnmMVc3z9+BDK8bLUO/wft5cPx97M42rPmzqlt6UfQfu6KsU/XX2K YVKvnN953+d2F0Z+LAtwZwG+Fywa5tB2PxjG8FIgCdA8xbHV9VO5wY3I8UupZMiucWbU2Npu usXO4jq2HpCBjTAZ7FCGV12qTZXiOnZU2ckoGA6B8QVHpBungQewteRihhUguM/wUcUp9VKD SqrsKK637s7HFdfrD2r/2NrxxXVsJam4bpwBIMAIcR6MoSZSQ9p6fHG9tivKQVhv+twww94F PFC9u0JCIrYkNSOY0fi+iCaoolbQRu5ttiVMwCqPSudlJSPuLZ4kdxa0q/ZOpSbdtMbE3+wE JTNumjDwSkG78Z5x8rNBOiqOIm1dR0F7uxaeKaOFyiAyJiO7CbeO4cSpXKC2rreg3XDXjICk 4xo8w0numrG0FoSJnQsB0/PsH8mC9tCUsqoUtLH7kvRBpAJEiyr6JsYlw+WjJvDb8QXtxt2l CtrvMkKHrSWnq4mBZtqPdF4+4aWaDrSfbha0q2aXjavKQS5oC4KRPUzmH13QbkULVBFjj/BZ Z0G7tZQTUyWiBng0TIFxPJLi5l0q7LU7I5WGHLsxzA3zUoRanqDlz3oq7NhRpQVoLvOtYSnm 1U5Tcww0mLi3wl512NJq+LjjgoThVMJ0eoW9uiBhFFxQfOKB2J+wfJ08OpUKe3VBEx52QD63 pLs8ucJ+7HN/pwr7R1if+bEv2tH8eO3od1iMYn8/iwm0M6vF3IZfef7QtTy8Pi3LcaZAtX58 cXPTgKCAtWAwCdJZJOfAHcnC2DmYVDF5qCRHroQYiw04cz2SXcVOE0PbvKZICF5HU2RJ72Sj 11jRYIiu/bj7ArfrnWAtIHiIP0gDZB5YhqgqTZwXb8LZHB1QtPZl97jUqbrw9cs8rNWJM76O Ubxz9KhmZ8kTA00B4LDQYIcAbj3Rx2x2iKTS6ah51hFcTMNiTUZFcqcJRIM7tejZvD8EkJci D6s4qVbVE90b+JWNpZyx7Fk9JiYNAGbm+p4Wo8BBXEEo9/i6Z3VfuDQCHtfys0+/FYWCQfel t/BZf95CQhr/JL4tpLH4YlSbaKhS+cQWA+YfHdBqgnoYluJzaeqn1Eqf9ZekjMBSKaUloV03 iFMEMbDFQH6HFX9Y/lxQkwsnoXmRxbB0aZnKIu3B0sVj8gk4sRhOq/p1VD/rVsZn4CiDi1GW 2plTmEqqrykILmSTB+rF3szgvsm6U6hKWq5A4JNJ3XTcu4qOyGI0SLUBFTQbM0kgHwnZHQKm 31d1ROI9NkIzA+K93EFlVlI+kh5zbpYdm/6a+vkdyKAboR4b8dIIJ+XgTqrzoQ87DwJp0JPL EDNlKcLjE7VD63sjHehus9RZUMpa/M6E+35ipQ/dn+wgV2J5md6LWbJi3aT/qdxi1rjFH0i6 01qLxcPOd2LdAavBxvUlILgc0riFHjlRrh7o0ZeuW4ztDeTdAWtMVlg5jvfEJsjPqbe4uiJh oEJw1sDlI9ntJhOo7upNfUWcQUx4/NznHhRV6+mY8MaNZozqfFMJQ6cKbqV603rv1M+7s8y+ mjX6yuCbJ5RHju6B16pJrb2hKjiURz+5glO90INbTSXmKTNOKeDqU0Fr2GkBfKOEFCbgc0Nq o5BxRk8Fp2agA6EK1JzNPWhHze/rLs3ZWgXnWF96jCZHo4LTeFaksmqEU+xdSqWCU/WjiitQ Gki5qNaKQJZw3Cr3VnBq++KXghH7+edEypI3M6tqBafhQeEwdpS68E6JESVIhQp3dldwtotR OellY56QHLRLz1yP+1jieis41ZekFDR8oILDCMpDHK3bXcFpPGuFTy6JXYX97gpOXkw8HJ2s LtMcDNDLhE2yjKIODbHzCRWcqgXmfEVWl+6SUURqFYDOJ1RwsMXEL2VcAhGZNAUYqp448S0L OfMJFZzWncFBSayX02pfBWe7GAaqwyvhZIB/dCRY6wRh07CW/+f/AgMG50m/FwUA --fdj2RfSjLxBAspz7-- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 15:17:31 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3838A7B3 for ; Thu, 24 Oct 2013 15:17:31 +0000 (UTC) (envelope-from prvs=1009ce2dcc=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CAB3424DE for ; Thu, 24 Oct 2013 15:17:30 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50006495039.msg for ; Thu, 24 Oct 2013 16:17:27 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Thu, 24 Oct 2013 16:17:27 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1009ce2dcc=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-hackers@freebsd.org Message-ID: <5677D36B8E1B460186377C6D1C8B888F@multiplay.co.uk> From: "Steven Hartland" To: , =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= Subject: make buildworld edits contrib/unbound/util/configparser.c on stable-10 Date: Thu, 24 Oct 2013 16:17:26 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 15:17:31 -0000 Was just about to commit a change to stable-10 when I noticed at the last second that just doing a make buildworld had edited the source file: contrib/unbound/util/configparser.c It seems its overwritten it with a generated version specific to my environment e.g. svn diff contrib/unbound/util/configparser.c |more Index: contrib/unbound/util/configparser.c =================================================================== --- contrib/unbound/util/configparser.c (revision 257053) +++ contrib/unbound/util/configparser.c (working copy) @@ -15,7 +15,7 @@ #define YYPURE 0 -#line 39 "util/configparser.y" +#line 39 "/usr/home/smh/freebsd/base/stable/10/lib/libunbound/../../contrib/unbound/util/configparser.y" #include "config.h" ... Obviously quite a nasty little gotcha as I could have easily broken the build by checking this unknown change in by mistake. @des: sent to you direct as it looks like you did the initial import of unbound so may know whats going on? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 16:52:21 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3ACF5D95 for ; Thu, 24 Oct 2013 16:52:21 +0000 (UTC) (envelope-from satan@ukr.net) Received: from hell.ukr.net (hell.ukr.net [212.42.67.68]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C07E32B1E for ; Thu, 24 Oct 2013 16:52:20 +0000 (UTC) Received: from satan by hell.ukr.net with local ID 1VZO8w-000La5-DC ; Thu, 24 Oct 2013 19:52:18 +0300 Date: Thu, 24 Oct 2013 19:52:18 +0300 From: Vitalij Satanivskij To: Vitalij Satanivskij Subject: Re: FreeBSD 10.0-BETA1 #8 r256765M spend too much time in locks Message-ID: <20131024165218.GA82686@hell.ukr.net> References: <20131024074826.GA50853@hell.ukr.net> <20131024075023.GA52443@hell.ukr.net> <20131024115519.GA72359@hell.ukr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131024115519.GA72359@hell.ukr.net> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 16:52:21 -0000 It's me again :) Some little more information Using simple script like - #!/bin/sh time=`date "+%Y-%m-%d_%H:%M"` sysctl debug.lock.prof.reset=1 sysctl debug.lock.prof.enable=1 sleep 1 sysctl debug.lock.prof.enable=0 sysctl debug.lock.prof.stats > locks_stats_$time.log and run it exactly when zfs flush data to pool eg. disk1 7,53T 8,78T 114 24 2,38M 1,36M disk1 7,53T 8,78T 16 4,18K 307K 60,9M disk1 7,53T 8,78T 146 7,18K 2,73M 48,2M disk1 7,53T 8,78T 372 60 7,71M 2,99M disk1 7,53T 8,78T 119 18 4,36M 940K disk1 7,53T 8,78T 115 33 2,56M 1,53M disk1 7,53T 8,78T 35 942 929K 11,1M disk1 7,53T 8,78T 58 5,64K 971K 89,4M - here disk1 7,53T 8,78T 271 2,63K 8,10M 10,9M disk1 7,53T 8,78T 218 21 11,7M 1,16M disk1 7,53T 8,78T 86 15 2,66M 786K disk1 7,53T 8,78T 92 15 2,29M 841K disk1 7,53T 8,78T 68 27 1,80M 5,98M get more information first system part of load grows up to 60-70% even when overal load not big second using script #!/bin/sh if [ -z "$1" ] then echo "Usage: lock.sh file.txt" exit 1 fi count=`awk 'BEGIN {NR>2}; {sum += $4}; END{print sum}' $1` awk -v col=$count 'BEGIN {NR>2}; {foo = ($4/col)* 100; if (foo >= 0.01) { printf "%f\t %s", foo, $10; for(i = 11;i <= NF; ++i) printf "%s ", $i; printf "\n" }}' $1 | sort -nr > res_$1.txt found next top's 26,812861 /usr/src/sys/kern/subr_vmem.c:1128(sleep mutex:kmem arena) 21,292701 /usr/src/sys/kern/subr_vmem.c:1230(sleep mutex:kmem arena) 13,661495 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3541(sx:l2arc_buflist_mtx) 13,099382 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1597(sx:buf_hash_table.ht_locks[i].ht_lock) 4,028993 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2009(sx:arc_eviction_mtx) 2,972297 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mru->arcs_locks[i].arcs_lock) 2,558227 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:862(sx:buf_hash_table.ht_locks[i].ht_lock) 2,328091 /usr/src/sys/kern/vfs_subr.c:2101(lockmgr:zfs) 2,294326 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:396(sx:vq->vq_lock) 1,418128 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mfu->arcs_locks[i].arcs_lock) 1,407655 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mru->arcs_locks[i].arcs_lock) 1,176428 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_object.c:41(sx:os->os_obj_lock) 0,936418 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mfu->arcs_locks[i].arcs_lock) 0,848378 /usr/src/sys/vm/vm_kern.c:400(rw:kmem vm object) 0,845051 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1381(sx:zfsvfs->z_hold_mtx[i]) 0,808122 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:129(sx:db->db_mtx) 0,680536 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1136(sx:zfsvfs->z_hold_mtx[i]) 0,616060 /usr/src/sys/vm/vm_kern.c:344(rw:kmem vm object) 0,609296 /usr/src/sys/kern/vfs_lookup.c:707(lockmgr:zfs) Another run 36,917638 /usr/src/sys/kern/subr_vmem.c:1230(sleep mutex:kmem arena) 32,872199 /usr/src/sys/kern/subr_vmem.c:1128(sleep mutex:kmem arena) 5,242466 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3541(sx:l2arc_buflist_mtx) 4,885420 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:891(sx:buf_hash_table.ht_locks[i].ht_lock) 4,626170 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1597(sx:buf_hash_table.ht_locks[i].ht_lock) 2,125377 /usr/src/sys/kern/vfs_lookup.c:707(lockmgr:zfs) 1,831535 /usr/src/sys/vm/vm_kern.c:400(rw:kmem vm object) 1,202869 /usr/src/sys/vm/vm_kern.c:344(rw:kmem vm object) 1,187653 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2009(sx:arc_eviction_mtx) 1,183960 /usr/src/sys/vm/uma_core.c:2097(sleep mutex:zio_cache) 1,015616 /usr/src/sys/vm/uma_core.c:2605(sleep mutex:zio_cache) 0,679882 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1136(sx:zfsvfs->z_hold_mtx[i]) 0,577229 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:129(sx:db->db_mtx) 0,555283 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mfu->arcs_locks[i].arcs_lock) 0,545886 /usr/src/sys/vm/uma_core.c:2259(sleep mutex:zio_cache) 0,510988 /usr/src/sys/vm/uma_core.c:2605(sleep mutex:zio_link_cache) 0,503242 /usr/src/sys/vm/uma_core.c:2710(sleep mutex:zio_cache) Buzy on drives not grows over 20% Maybe somebody have any sugestion about what can be going on or at last what to check/tune ? Vitalij Satanivskij wrote: VS> Forget about profiling log VS> VS> VS> Vitalij Satanivskij wrote: VS> VS> VS> VS> VS> VS> EEE forget to notice - old system have none compression on main zfs VS> VS> VS> VS> VS> VS> VS> VS> Vitalij Satanivskij wrote: VS> VS> VS> Hello. VS> VS> VS> VS> VS> VS> After upgrading system from old current (r245701) to fresh current r255173 (than switch to stable/10 r256765M) VS> VS> VS> found some strange system behavior: VS> VS> VS> VS> VS> VS> Diff from r256765M anf r256765 is VS> VS> VS> Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c VS> VS> VS> =================================================================== VS> VS> VS> --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 256765) VS> VS> VS> +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy) VS> VS> VS> @@ -5147,7 +5147,7 @@ VS> VS> VS> len = l2hdr->b_asize; VS> VS> VS> cdata = zio_data_buf_alloc(len); VS> VS> VS> csize = zio_compress_data(ZIO_COMPRESS_LZ4, l2hdr->b_tmp_cdata, VS> VS> VS> - cdata, l2hdr->b_asize, (size_t)SPA_MINBLOCKSIZE); VS> VS> VS> + cdata, l2hdr->b_asize, (size_t)(1ULL << l2hdr->b_dev->l2ad_vdev->vdev_ashift)); VS> VS> VS> VS> VS> VS> if (csize == 0) { VS> VS> VS> /* zero block, indicate that there's nothing to write */ VS> VS> VS> VS> VS> VS> VS> VS> VS> But same situation was befor patch. VS> VS> VS> VS> VS> VS> VS> VS> VS> System load to high VS> VS> VS> CPU: 6.8% user, 0.0% nice, 57.3% system, 0.8% interrupt, 35.1% idle VS> VS> VS> VS> VS> VS> hotkernel (dtrace script) says VS> VS> VS> VS> VS> VS> kernel`__mtx_unlock_flags 292 0.3% VS> VS> VS> kernel`__mtx_lock_flags 315 0.3% VS> VS> VS> zfs.ko`lzjb_compress 349 0.3% VS> VS> VS> kernel`__rw_wlock_hard 686 0.7% VS> VS> VS> kernel`spinlock_exit 1050 1.0% VS> VS> VS> kernel`vmem_xalloc 7516 7.3% VS> VS> VS> kernel`_sx_xlock_hard 8664 8.5% VS> VS> VS> kernel`acpi_cpu_c1 9737 9.5% VS> VS> VS> kernel`cpu_idle 20189 19.7% VS> VS> VS> kernel`__mtx_lock_sleep 45952 44.9% VS> VS> VS> VS> VS> VS> VS> VS> VS> VS> VS> VS> Trying to understand where is a problem I'm build kernel with lock profiling. VS> VS> VS> VS> VS> VS> and get some data (for one minute ) VS> VS> VS> VS> VS> VS> (file attached) VS> VS> VS> VS> VS> VS> using agregation find most lock's is in VS> VS> VS> VS> VS> VS> 14,159818 /usr/src/sys/kern/subr_vmem.c:1128(sleep mutex:kmem arena) VS> VS> VS> 9,553523 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1597(sx:buf_hash_table.ht_locks[i].ht_lock) VS> VS> VS> 8,386943 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3541(sx:l2arc_buflist_mtx) VS> VS> VS> 8,110206 /usr/src/sys/kern/subr_vmem.c:1230(sleep mutex:kmem arena) VS> VS> VS> 5,909429 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mru->arcs_locks[i].arcs_lock) VS> VS> VS> 5,452206 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mfu->arcs_locks[i].arcs_lock) VS> VS> VS> 5,050224 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c:303(sx:tx->tx_cpu[c].tc_open_lock) VS> VS> VS> 4,232819 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:891(sx:buf_hash_table.ht_locks[i].ht_lock) VS> VS> VS> 4,211348 /usr/src/sys/kern/vfs_subr.c:2101(lockmgr:zfs) VS> VS> VS> 4,011656 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:862(sx:buf_hash_table.ht_locks[i].ht_lock) VS> VS> VS> 3,823698 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2009(sx:arc_eviction_mtx) VS> VS> VS> 2,697344 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:126(sx:h->hash_mutexes[i]) VS> VS> VS> 2,343242 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mfu->arcs_locks[i].arcs_lock) VS> VS> VS> 1,752713 /usr/src/sys/kern/vfs_lookup.c:707(lockmgr:zfs) VS> VS> VS> 1,580856 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1136(sx:zfsvfs->z_hold_mtx[i]) VS> VS> VS> 1,534242 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1291(sx:arc_mfu->arcs_locks[i].arcs_lock) VS> VS> VS> 1,331583 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:129(sx:db->db_mtx) VS> VS> VS> 1,105058 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:427(sx:vq->vq_lock) VS> VS> VS> 1,080855 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:396(sx:vq->vq_lock) VS> VS> VS> 0,858568 /usr/src/sys/kern/vfs_cache.c:488(rw:Name Cache) VS> VS> VS> 0,831652 /usr/src/sys/vm/vm_kern.c:344(rw:kmem vm object) VS> VS> VS> 0,815439 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1845(sx:buf_hash_table.ht_locks[i].ht_lock) VS> VS> VS> 0,812613 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mru->arcs_locks[i].arcs_lock) VS> VS> VS> 0,794274 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1529(lockmgr:zfs) VS> VS> VS> 0,669845 /usr/src/sys/vm/uma_core.c:2097(sleep mutex:zio_cache) VS> VS> VS> VS> VS> VS> VS> VS> VS> VS> VS> VS> VS> VS> VS> Short system description VS> VS> VS> CPU E5-1650 VS> VS> VS> MEM 128Gb ddr3-1600 VS> VS> VS> VS> VS> VS> Storage subsystem VS> VS> VS> VS> VS> VS> 36x1Tb WD RE4 drives on LSI SAS2308 Controler VS> VS> VS> 3x180Gb Intel ssd 530 series as l2 cache VS> VS> VS> VS> VS> VS> VS> VS> VS> POOL is 18 mirrors, two drives in ich mirror and 3 cache devices VS> VS> VS> VS> VS> VS> eg. VS> VS> VS> .... VS> VS> VS> mirror-14 ONLINE 0 0 0 VS> VS> VS> gpt/disk28 ONLINE 0 0 0 VS> VS> VS> gpt/disk29 ONLINE 0 0 0 VS> VS> VS> mirror-15 ONLINE 0 0 0 VS> VS> VS> gpt/disk30 ONLINE 0 0 0 VS> VS> VS> gpt/disk31 ONLINE 0 0 0 VS> VS> VS> mirror-16 ONLINE 0 0 0 VS> VS> VS> gpt/disk32 ONLINE 0 0 0 VS> VS> VS> gpt/disk33 ONLINE 0 0 0 VS> VS> VS> mirror-17 ONLINE 0 0 0 VS> VS> VS> gpt/disk34 ONLINE 0 0 0 VS> VS> VS> gpt/disk35 ONLINE 0 0 0 VS> VS> VS> cache VS> VS> VS> ada1 ONLINE 0 0 0 VS> VS> VS> ada2 ONLINE 0 0 0 VS> VS> VS> ada3 ONLINE 0 0 0 VS> VS> VS> VS> VS> VS> VS> VS> VS> POOL have two ZFS VS> VS> VS> VS> VS> VS> main with options (diffs from default) - VS> VS> VS> compression lz4 VS> VS> VS> secondarycache all VS> VS> VS> sync disabled VS> VS> VS> VS> VS> VS> Data size for it around 6Tb (compresed) eg VS> VS> VS> disk1 refcompressratio 1.56x - VS> VS> VS> disk1 written 5,99T - VS> VS> VS> disk1 logicalused 10,8T - VS> VS> VS> disk1 logicalreferenced 9,32T - VS> VS> VS> VS> VS> VS> VS> VS> VS> and another with options VS> VS> VS> recordsize 4K, before it was 32k, but internal software use data blocks mostly 4k so we try to change (without real data realocate ) VS> VS> VS> compresion -s off VS> VS> VS> sync disabled VS> VS> VS> secondarycache all VS> VS> VS> VS> VS> VS> DATA size on it around 1.4Tb VS> VS> VS> VS> VS> VS> ARC configured to use max 80Gb VS> VS> VS> VS> VS> VS> top most time looks like this VS> VS> VS> VS> VS> VS> Mem: 14G Active, 13G Inact, 94G Wired, 497M Cache, 3297M Buf, 2214M Free VS> VS> VS> ARC: 80G Total, 2010M MFU, 70G MRU, 49M Anon, 3822M Header, 4288M Other VS> VS> VS> VS> VS> VS> VS> VS> VS> LA's - around 10-20 depend on day time. VS> VS> VS> VS> VS> VS> VS> VS> VS> zpool iostat disk1 1 VS> VS> VS> capacity operations bandwidth VS> VS> VS> pool alloc free read write read write VS> VS> VS> ---------- ----- ----- ----- ----- ----- ----- VS> VS> VS> disk1 7,45T 8,86T 546 1,49K 16,4M 13,4M VS> VS> VS> disk1 7,45T 8,86T 272 3,91K 11,7M 27,4M VS> VS> VS> disk1 7,45T 8,86T 344 2,94K 7,26M 25,2M VS> VS> VS> disk1 7,45T 8,86T 224 2,02K 9,91M 21,8M VS> VS> VS> disk1 7,45T 8,86T 244 2,35K 8,23M 18,4M VS> VS> VS> disk1 7,45T 8,86T 245 2,54K 6,44M 23,4M VS> VS> VS> disk1 7,45T 8,86T 114 2,94K 3,28M 13,3M VS> VS> VS> disk1 7,45T 8,86T 288 4,43K 6,09M 33,5M VS> VS> VS> disk1 7,45T 8,86T 157 1,26K 2,98M 15,7M VS> VS> VS> disk1 7,45T 8,86T 94 842 3,07M 13,6M VS> VS> VS> disk1 7,45T 8,86T 327 1,71K 9,63M 8,21M VS> VS> VS> disk1 7,45T 8,86T 237 1,81K 5,73M 29,3M VS> VS> VS> disk1 7,45T 8,86T 247 3,47K 5,17M 29,6M VS> VS> VS> disk1 7,45T 8,86T 165 2,37K 3,22M 16,7M VS> VS> VS> disk1 7,45T 8,86T 155 3,23K 3,27M 23,9M VS> VS> VS> VS> VS> VS> Strange as timeout seted up to 10sec's. VS> VS> VS> VS> VS> VS> What intresting - after reboot system work fine for some time, at last while ARC not become 80G size. VS> VS> VS> Low limit for arc is 40gb, strange but old system can take memory from arc eg like this VS> VS> VS> VS> VS> VS> VS> VS> VS> Mem: 32G Active, 8797M Inact, 78G Wired, 2370M Cache, 209M Buf, 3977M Free VS> VS> VS> ARC: 43G Total, 2204M MFU, 28G MRU, 135M Anon, 7898M Header, 5301M Other VS> VS> VS> VS> VS> VS> On new ARC getting to it max allowed size. VS> VS> VS> VS> VS> VS> So for now question is, what it can be, what we can try to improve system perfomance and so on? VS> VS> VS> VS> VS> VS> VS> VS> VS> VS> VS> VS> _______________________________________________ VS> VS> VS> freebsd-hackers@freebsd.org mailing list VS> VS> VS> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers VS> VS> VS> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" VS> VS> _______________________________________________ VS> VS> freebsd-hackers@freebsd.org mailing list VS> VS> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers VS> VS> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" VS> _______________________________________________ VS> freebsd-hackers@freebsd.org mailing list VS> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers VS> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 17:15:00 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 8968273C for ; Thu, 24 Oct 2013 17:15:00 +0000 (UTC) (envelope-from prvs=1009ce2dcc=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E0B652C79 for ; Thu, 24 Oct 2013 17:14:59 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50006495893.msg for ; Thu, 24 Oct 2013 18:14:55 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Thu, 24 Oct 2013 18:14:55 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1009ce2dcc=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-hackers@freebsd.org Message-ID: <6B49A70ECCAB453F8A4EF540EFF1A3A0@multiplay.co.uk> From: "Steven Hartland" To: "Vitalij Satanivskij" References: <20131024074826.GA50853@hell.ukr.net> <20131024075023.GA52443@hell.ukr.net> <20131024115519.GA72359@hell.ukr.net> <20131024165218.GA82686@hell.ukr.net> Subject: Re: FreeBSD 10.0-BETA1 #8 r256765M spend too much time in locks Date: Thu, 24 Oct 2013 18:14:55 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 17:15:00 -0000 The following proposed changes to illumos may well be of interest:- Webrev: http://ma.nexenta.com/borisp/arc_locks/ Webrev: http://cr.illumos.org/~webrev/skiselkov/new_buf_hash/ Issue: https://www.illumos.org/issues/4218 ----- Original Message ----- From: "Vitalij Satanivskij" To: "Vitalij Satanivskij" Cc: Sent: Thursday, October 24, 2013 5:52 PM Subject: Re: FreeBSD 10.0-BETA1 #8 r256765M spend too much time in locks > > It's me again :) > > Some little more information > > Using simple script like - > #!/bin/sh > > time=`date "+%Y-%m-%d_%H:%M"` > sysctl debug.lock.prof.reset=1 > sysctl debug.lock.prof.enable=1 > sleep 1 > sysctl debug.lock.prof.enable=0 > sysctl debug.lock.prof.stats > locks_stats_$time.log > > and run it exactly when zfs flush data to pool > > eg. > > disk1 7,53T 8,78T 114 24 2,38M 1,36M > disk1 7,53T 8,78T 16 4,18K 307K 60,9M > disk1 7,53T 8,78T 146 7,18K 2,73M 48,2M > disk1 7,53T 8,78T 372 60 7,71M 2,99M > disk1 7,53T 8,78T 119 18 4,36M 940K > disk1 7,53T 8,78T 115 33 2,56M 1,53M > disk1 7,53T 8,78T 35 942 929K 11,1M > disk1 7,53T 8,78T 58 5,64K 971K 89,4M - here > disk1 7,53T 8,78T 271 2,63K 8,10M 10,9M > disk1 7,53T 8,78T 218 21 11,7M 1,16M > disk1 7,53T 8,78T 86 15 2,66M 786K > disk1 7,53T 8,78T 92 15 2,29M 841K > disk1 7,53T 8,78T 68 27 1,80M 5,98M > > > get more information > > first system part of load grows up to 60-70% even when overal load not big > > second using script > > #!/bin/sh > > if [ -z "$1" ] > then > echo "Usage: lock.sh file.txt" > exit 1 > fi > > count=`awk 'BEGIN {NR>2}; {sum += $4}; END{print sum}' $1` > awk -v col=$count 'BEGIN {NR>2}; {foo = ($4/col)* 100; if (foo >= 0.01) { printf "%f\t %s", foo, $10; for(i = 11;i <= NF; ++i) > printf "%s ", $i; printf "\n" }}' $1 | sort -nr > res_$1.txt > > found next top's > > 26,812861 /usr/src/sys/kern/subr_vmem.c:1128(sleep mutex:kmem arena) > 21,292701 /usr/src/sys/kern/subr_vmem.c:1230(sleep mutex:kmem arena) > 13,661495 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3541(sx:l2arc_buflist_mtx) > 13,099382 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1597(sx:buf_hash_table.ht_locks[i].ht_lock) > 4,028993 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2009(sx:arc_eviction_mtx) > 2,972297 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mru->arcs_locks[i].arcs_lock) > 2,558227 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:862(sx:buf_hash_table.ht_locks[i].ht_lock) > 2,328091 /usr/src/sys/kern/vfs_subr.c:2101(lockmgr:zfs) > 2,294326 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:396(sx:vq->vq_lock) > 1,418128 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mfu->arcs_locks[i].arcs_lock) > 1,407655 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mru->arcs_locks[i].arcs_lock) > 1,176428 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_object.c:41(sx:os->os_obj_lock) > 0,936418 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mfu->arcs_locks[i].arcs_lock) > 0,848378 /usr/src/sys/vm/vm_kern.c:400(rw:kmem vm object) > 0,845051 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1381(sx:zfsvfs->z_hold_mtx[i]) > 0,808122 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:129(sx:db->db_mtx) > 0,680536 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1136(sx:zfsvfs->z_hold_mtx[i]) > 0,616060 /usr/src/sys/vm/vm_kern.c:344(rw:kmem vm object) > 0,609296 /usr/src/sys/kern/vfs_lookup.c:707(lockmgr:zfs) > > > Another run > > 36,917638 /usr/src/sys/kern/subr_vmem.c:1230(sleep mutex:kmem arena) > 32,872199 /usr/src/sys/kern/subr_vmem.c:1128(sleep mutex:kmem arena) > 5,242466 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3541(sx:l2arc_buflist_mtx) > 4,885420 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:891(sx:buf_hash_table.ht_locks[i].ht_lock) > 4,626170 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1597(sx:buf_hash_table.ht_locks[i].ht_lock) > 2,125377 /usr/src/sys/kern/vfs_lookup.c:707(lockmgr:zfs) > 1,831535 /usr/src/sys/vm/vm_kern.c:400(rw:kmem vm object) > 1,202869 /usr/src/sys/vm/vm_kern.c:344(rw:kmem vm object) > 1,187653 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2009(sx:arc_eviction_mtx) > 1,183960 /usr/src/sys/vm/uma_core.c:2097(sleep mutex:zio_cache) > 1,015616 /usr/src/sys/vm/uma_core.c:2605(sleep mutex:zio_cache) > 0,679882 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1136(sx:zfsvfs->z_hold_mtx[i]) > 0,577229 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:129(sx:db->db_mtx) > 0,555283 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mfu->arcs_locks[i].arcs_lock) > 0,545886 /usr/src/sys/vm/uma_core.c:2259(sleep mutex:zio_cache) > 0,510988 /usr/src/sys/vm/uma_core.c:2605(sleep mutex:zio_link_cache) > 0,503242 /usr/src/sys/vm/uma_core.c:2710(sleep mutex:zio_cache) > > > Buzy on drives not grows over 20% > > Maybe somebody have any sugestion about what can be going on or at last what to check/tune ? > > > Vitalij Satanivskij wrote: > VS> Forget about profiling log > VS> > VS> > VS> Vitalij Satanivskij wrote: > VS> VS> > VS> VS> > VS> VS> EEE forget to notice - old system have none compression on main zfs > VS> VS> > VS> VS> > VS> VS> > VS> VS> Vitalij Satanivskij wrote: > VS> VS> VS> Hello. > VS> VS> VS> > VS> VS> VS> After upgrading system from old current (r245701) to fresh current r255173 (than switch to stable/10 r256765M) > VS> VS> VS> found some strange system behavior: > VS> VS> VS> > VS> VS> VS> Diff from r256765M anf r256765 is > VS> VS> VS> Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c > VS> VS> VS> =================================================================== > VS> VS> VS> --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 256765) > VS> VS> VS> +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy) > VS> VS> VS> @@ -5147,7 +5147,7 @@ > VS> VS> VS> len = l2hdr->b_asize; > VS> VS> VS> cdata = zio_data_buf_alloc(len); > VS> VS> VS> csize = zio_compress_data(ZIO_COMPRESS_LZ4, l2hdr->b_tmp_cdata, > VS> VS> VS> - cdata, l2hdr->b_asize, (size_t)SPA_MINBLOCKSIZE); > VS> VS> VS> + cdata, l2hdr->b_asize, (size_t)(1ULL << l2hdr->b_dev->l2ad_vdev->vdev_ashift)); > VS> VS> VS> > VS> VS> VS> if (csize == 0) { > VS> VS> VS> /* zero block, indicate that there's nothing to write */ > VS> VS> VS> > VS> VS> VS> > VS> VS> VS> But same situation was befor patch. > VS> VS> VS> > VS> VS> VS> > VS> VS> VS> System load to high > VS> VS> VS> CPU: 6.8% user, 0.0% nice, 57.3% system, 0.8% interrupt, 35.1% idle > VS> VS> VS> > VS> VS> VS> hotkernel (dtrace script) says > VS> VS> VS> > VS> VS> VS> kernel`__mtx_unlock_flags 292 0.3% > VS> VS> VS> kernel`__mtx_lock_flags 315 0.3% > VS> VS> VS> zfs.ko`lzjb_compress 349 0.3% > VS> VS> VS> kernel`__rw_wlock_hard 686 0.7% > VS> VS> VS> kernel`spinlock_exit 1050 1.0% > VS> VS> VS> kernel`vmem_xalloc 7516 7.3% > VS> VS> VS> kernel`_sx_xlock_hard 8664 8.5% > VS> VS> VS> kernel`acpi_cpu_c1 9737 9.5% > VS> VS> VS> kernel`cpu_idle 20189 19.7% > VS> VS> VS> kernel`__mtx_lock_sleep 45952 44.9% > VS> VS> VS> > VS> VS> VS> > VS> VS> VS> > VS> VS> VS> Trying to understand where is a problem I'm build kernel with lock profiling. > VS> VS> VS> > VS> VS> VS> and get some data (for one minute ) > VS> VS> VS> > VS> VS> VS> (file attached) > VS> VS> VS> > VS> VS> VS> using agregation find most lock's is in > VS> VS> VS> > VS> VS> VS> 14,159818 /usr/src/sys/kern/subr_vmem.c:1128(sleep mutex:kmem arena) > VS> VS> VS> 9,553523 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1597(sx:buf_hash_table.ht_locks[i].ht_lock) > VS> VS> VS> 8,386943 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3541(sx:l2arc_buflist_mtx) > VS> VS> VS> 8,110206 /usr/src/sys/kern/subr_vmem.c:1230(sleep mutex:kmem arena) > VS> VS> VS> 5,909429 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mru->arcs_locks[i].arcs_lock) > VS> VS> VS> 5,452206 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1969(sx:arc_mfu->arcs_locks[i].arcs_lock) > VS> VS> VS> 5,050224 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c:303(sx:tx->tx_cpu[c].tc_open_lock) > VS> VS> VS> 4,232819 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:891(sx:buf_hash_table.ht_locks[i].ht_lock) > VS> VS> VS> 4,211348 /usr/src/sys/kern/vfs_subr.c:2101(lockmgr:zfs) > VS> VS> VS> 4,011656 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:862(sx:buf_hash_table.ht_locks[i].ht_lock) > VS> VS> VS> 3,823698 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2009(sx:arc_eviction_mtx) > VS> VS> VS> 2,697344 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:126(sx:h->hash_mutexes[i]) > VS> VS> VS> 2,343242 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mfu->arcs_locks[i].arcs_lock) > VS> VS> VS> 1,752713 /usr/src/sys/kern/vfs_lookup.c:707(lockmgr:zfs) > VS> VS> VS> 1,580856 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:1136(sx:zfsvfs->z_hold_mtx[i]) > VS> VS> VS> 1,534242 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1291(sx:arc_mfu->arcs_locks[i].arcs_lock) > VS> VS> VS> 1,331583 /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:129(sx:db->db_mtx) > VS> VS> VS> 1,105058 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:427(sx:vq->vq_lock) > VS> VS> VS> 1,080855 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:396(sx:vq->vq_lock) > VS> VS> VS> 0,858568 /usr/src/sys/kern/vfs_cache.c:488(rw:Name Cache) > VS> VS> VS> 0,831652 /usr/src/sys/vm/vm_kern.c:344(rw:kmem vm object) > VS> VS> VS> 0,815439 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1845(sx:buf_hash_table.ht_locks[i].ht_lock) > VS> VS> VS> 0,812613 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1256(sx:arc_mru->arcs_locks[i].arcs_lock) > VS> VS> VS> 0,794274 > /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1529(lockmgr:zfs) > VS> VS> VS> 0,669845 /usr/src/sys/vm/uma_core.c:2097(sleep mutex:zio_cache) > VS> VS> VS> > VS> VS> VS> > VS> VS> VS> > VS> VS> VS> > VS> VS> VS> Short system description > VS> VS> VS> CPU E5-1650 > VS> VS> VS> MEM 128Gb ddr3-1600 > VS> VS> VS> > VS> VS> VS> Storage subsystem > VS> VS> VS> > VS> VS> VS> 36x1Tb WD RE4 drives on LSI SAS2308 Controler > VS> VS> VS> 3x180Gb Intel ssd 530 series as l2 cache > VS> VS> VS> > VS> VS> VS> > VS> VS> VS> POOL is 18 mirrors, two drives in ich mirror and 3 cache devices > VS> VS> VS> > VS> VS> VS> eg. > VS> VS> VS> .... > VS> VS> VS> mirror-14 ONLINE 0 0 0 > VS> VS> VS> gpt/disk28 ONLINE 0 0 0 > VS> VS> VS> gpt/disk29 ONLINE 0 0 0 > VS> VS> VS> mirror-15 ONLINE 0 0 0 > VS> VS> VS> gpt/disk30 ONLINE 0 0 0 > VS> VS> VS> gpt/disk31 ONLINE 0 0 0 > VS> VS> VS> mirror-16 ONLINE 0 0 0 > VS> VS> VS> gpt/disk32 ONLINE 0 0 0 > VS> VS> VS> gpt/disk33 ONLINE 0 0 0 > VS> VS> VS> mirror-17 ONLINE 0 0 0 > VS> VS> VS> gpt/disk34 ONLINE 0 0 0 > VS> VS> VS> gpt/disk35 ONLINE 0 0 0 > VS> VS> VS> cache > VS> VS> VS> ada1 ONLINE 0 0 0 > VS> VS> VS> ada2 ONLINE 0 0 0 > VS> VS> VS> ada3 ONLINE 0 0 0 > VS> VS> VS> > VS> VS> VS> > VS> VS> VS> POOL have two ZFS > VS> VS> VS> > VS> VS> VS> main with options (diffs from default) - > VS> VS> VS> compression lz4 > VS> VS> VS> secondarycache all > VS> VS> VS> sync disabled > VS> VS> VS> > VS> VS> VS> Data size for it around 6Tb (compresed) eg > VS> VS> VS> disk1 refcompressratio 1.56x - > VS> VS> VS> disk1 written 5,99T - > VS> VS> VS> disk1 logicalused 10,8T - > VS> VS> VS> disk1 logicalreferenced 9,32T - > VS> VS> VS> > VS> VS> VS> > VS> VS> VS> and another with options > VS> VS> VS> recordsize 4K, before it was 32k, but internal software use data blocks mostly 4k so we try to change > (without real data realocate ) > VS> VS> VS> compresion -s off > VS> VS> VS> sync disabled > VS> VS> VS> secondarycache all > VS> VS> VS> > VS> VS> VS> DATA size on it around 1.4Tb > VS> VS> VS> > VS> VS> VS> ARC configured to use max 80Gb > VS> VS> VS> > VS> VS> VS> top most time looks like this > VS> VS> VS> > VS> VS> VS> Mem: 14G Active, 13G Inact, 94G Wired, 497M Cache, 3297M Buf, 2214M Free > VS> VS> VS> ARC: 80G Total, 2010M MFU, 70G MRU, 49M Anon, 3822M Header, 4288M Other > VS> VS> VS> > VS> VS> VS> > VS> VS> VS> LA's - around 10-20 depend on day time. > VS> VS> VS> > VS> VS> VS> > VS> VS> VS> zpool iostat disk1 1 > VS> VS> VS> capacity operations bandwidth > VS> VS> VS> pool alloc free read write read write > VS> VS> VS> ---------- ----- ----- ----- ----- ----- ----- > VS> VS> VS> disk1 7,45T 8,86T 546 1,49K 16,4M 13,4M > VS> VS> VS> disk1 7,45T 8,86T 272 3,91K 11,7M 27,4M > VS> VS> VS> disk1 7,45T 8,86T 344 2,94K 7,26M 25,2M > VS> VS> VS> disk1 7,45T 8,86T 224 2,02K 9,91M 21,8M > VS> VS> VS> disk1 7,45T 8,86T 244 2,35K 8,23M 18,4M > VS> VS> VS> disk1 7,45T 8,86T 245 2,54K 6,44M 23,4M > VS> VS> VS> disk1 7,45T 8,86T 114 2,94K 3,28M 13,3M > VS> VS> VS> disk1 7,45T 8,86T 288 4,43K 6,09M 33,5M > VS> VS> VS> disk1 7,45T 8,86T 157 1,26K 2,98M 15,7M > VS> VS> VS> disk1 7,45T 8,86T 94 842 3,07M 13,6M > VS> VS> VS> disk1 7,45T 8,86T 327 1,71K 9,63M 8,21M > VS> VS> VS> disk1 7,45T 8,86T 237 1,81K 5,73M 29,3M > VS> VS> VS> disk1 7,45T 8,86T 247 3,47K 5,17M 29,6M > VS> VS> VS> disk1 7,45T 8,86T 165 2,37K 3,22M 16,7M > VS> VS> VS> disk1 7,45T 8,86T 155 3,23K 3,27M 23,9M > VS> VS> VS> > VS> VS> VS> Strange as timeout seted up to 10sec's. > VS> VS> VS> > VS> VS> VS> What intresting - after reboot system work fine for some time, at last while ARC not become 80G size. > VS> VS> VS> Low limit for arc is 40gb, strange but old system can take memory from arc eg like this > VS> VS> VS> > VS> VS> VS> > VS> VS> VS> Mem: 32G Active, 8797M Inact, 78G Wired, 2370M Cache, 209M Buf, 3977M Free > VS> VS> VS> ARC: 43G Total, 2204M MFU, 28G MRU, 135M Anon, 7898M Header, 5301M Other > VS> VS> VS> > VS> VS> VS> On new ARC getting to it max allowed size. > VS> VS> VS> > VS> VS> VS> So for now question is, what it can be, what we can try to improve system perfomance and so on? > VS> VS> VS> > VS> VS> VS> > VS> VS> VS> > VS> VS> VS> _______________________________________________ > VS> VS> VS> freebsd-hackers@freebsd.org mailing list > VS> VS> VS> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > VS> VS> VS> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > VS> VS> _______________________________________________ > VS> VS> freebsd-hackers@freebsd.org mailing list > VS> VS> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > VS> VS> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > > VS> _______________________________________________ > VS> freebsd-hackers@freebsd.org mailing list > VS> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > VS> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 19:09:55 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6BC88197; Thu, 24 Oct 2013 19:09:55 +0000 (UTC) (envelope-from outbackdingo@gmail.com) Received: from mail-oa0-x22a.google.com (mail-oa0-x22a.google.com [IPv6:2607:f8b0:4003:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1B74B23B6; Thu, 24 Oct 2013 19:09:55 +0000 (UTC) Received: by mail-oa0-f42.google.com with SMTP id k14so2891656oag.1 for ; Thu, 24 Oct 2013 12:09:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=noI8hKN35QDmnCnLkPnnuxKVvS5Bk34ImchDGsEpE5U=; b=ytPs/CYG2a0H5enMCt9P84pXpnk4bAGNjcp7bn9bTAq1sS44lgBtw9D6cBcTbNZNa/ kR/rt4R8qu8VdzTeRLh4UkgnQI7sbn7NZpgGnCL9/6nsLWvsNXUp/EiFdQFgkmR3GQNc qkp5XkYebW95YYN+dWqmUvUYVazYuSqK2RimQRILNepv36kMvm6LmBiJV6fRtwhoUPwI 5z7DjYb7XLrPiw9PI/W7Zr+ef/Meo1oh+5B2L1Z1TcKmUx1pp8a4P7PAW0afTL4WaxX2 WBQL0W/VodM5si2/ntXRm7WLR5+kI9kE9HyURXPR2h4w4yKaSlFWpi723PnQE9Ugy6SW 1K4Q== MIME-Version: 1.0 X-Received: by 10.182.142.229 with SMTP id rz5mr3336186obb.12.1382641794264; Thu, 24 Oct 2013 12:09:54 -0700 (PDT) Received: by 10.76.19.115 with HTTP; Thu, 24 Oct 2013 12:09:54 -0700 (PDT) In-Reply-To: References: <1382192148.119437035@f257.i.mail.ru> <1382286899.92499.110.camel@revolution.hippie.lan> Date: Thu, 24 Oct 2013 15:09:54 -0400 Message-ID: Subject: Re: determine drive's SAS port From: Outback Dingo To: Alan Somers Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Alexey Egorov , freebsd-hackers@freebsd.org, Ian Lepore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 19:09:55 -0000 On Mon, Oct 21, 2013 at 12:42 PM, Alan Somers wrote: > On Sun, Oct 20, 2013 at 10:34 AM, Ian Lepore wrote: > > On Sat, 2013-10-19 at 18:15 +0400, Alexey Egorov wrote: > >> Hello all, > >> > >> I have a server with LSI HBA card, and when I remove drive I can see > following messages in log: > >> > >> (da0:mps0:0:5:0): lost device - 0 outstanding, 2 refs > >> (da0:mps0:0:5:0): removing device entry > >> > >> Is there a way to determine physical port (number "5" in > "(da0:mps0:0:5:0)") when drive is inserted? (I need this to be able to > create device symlinks based on physical port). > >> > >> Thanks in advance. > > > > I don't have hardware like that to play with, but when I plug in an > > eSata drive I get devd events like this: > > > > root@dpnand:/root # cat /var/run/devd.pipe > > !system=DEVFS subsystem=CDEV type=CREATE cdev=pass2 > > !system=DEVFS subsystem=CDEV type=CREATE cdev=ada0 > > !system=DEVFS subsystem=CDEV type=CREATE cdev=ad0 > > !system=DEVFS subsystem=CDEV type=CREATE cdev=ada0p11 > > !system=DEVFS subsystem=CDEV type=CREATE cdev=ad0p11 > > !system=DEVFS subsystem=CDEV type=CREATE cdev=diskid/DISK-10MS109LT74Z > > !system=DEVFS subsystem=CDEV type=CREATE cdev=ufsid/51fabc51ea1a923b > > !system=DEVFS subsystem=CDEV type=CREATE > cdev=gptid/c057d696-fae3-11e2-b79c-5404a6f2f88a > > !system=DEVFS subsystem=CDEV type=CREATE cdev=diskid/DISK-10MS109LT74Zp11 > > ^C > > > > The pass2 dev appears in a camcontrol devlist, like this: > > > > root@dpnand:/root # camcontrol devlist > > at scbus0 target 0 lun 0 (pass2,ada0) > > at scbus2 target 0 lun 0 (pass0,da0) > > at scbus2 target 0 lun 1 (pass1,da1) > > > > Would the camcontrol bus/target/lun output give you the number you're > > looking for? It's a pity there isn't a devd event with more info in it > > (similar to what you get when usb devices come and go), but perhaps with > > some scripting you can make the connection between events and devices. > > > > -- Ian > > > > > > > > > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > > If your disk is connected to a SAS expander, then you can use either > SES or SMP commands to figure out the port to which a drive is > connected. But it sounds like your disks are connected directly to > the HBA? If that's the case, the problem is much harder. I don't > think that there is a standard way to get it, and the FreeBSD kernel > doesn't currently help much. mps(4) accepts some custom ioctls to > look up configuration information. To learn how to use them, ask your > LSI FAE for the "Fusion-MPT 2.0 Message Passing Interface (MPI) > Specification Guide". The best solution would be to update the kernel > to report physical path information for da disks. I think that > information is currently reported only for disks connected to a SES > enclosure. > I need to replace a disk connected to a SAS expander also now, there are 44 drives. How do i use this ses to determine the drive, can i make its light blink ?? > > -Alan > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 19:22:45 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 1C78E844 for ; Thu, 24 Oct 2013 19:22:45 +0000 (UTC) (envelope-from outbackdingo@gmail.com) Received: from mail-oa0-x229.google.com (mail-oa0-x229.google.com [IPv6:2607:f8b0:4003:c02::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DD5542493 for ; Thu, 24 Oct 2013 19:22:44 +0000 (UTC) Received: by mail-oa0-f41.google.com with SMTP id o9so2918334oag.0 for ; Thu, 24 Oct 2013 12:22:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=soh9BIjAmjAT6f7fmNEOeg+JBjc1cmudzAR/XcnH1ns=; b=DRVsKpEPBeM00LmL/1ETkcZ1kKMA1winW/RZC2UhXcV7YEXNs5zPexwdQIodZn6HxZ kw9SkNl/EGRhul342d74LFH2d4F3Ie4/MKGJm+vuLoTIm+TX/P8R38x/sTHNiG6ZfLbl ddeEX1CLAAU8Tpj46kJXvqwUl/0fjWIoDLjyaU/7o2r2LuzC0K9Y7elSqn4POncyOXxJ Snd/hhqgC63Vnaj/E3uKZd32a8MwX5Kz33re0tDCjI891cbEyR9XYoUGYmixwyZ5OAt8 X/5GjnS71gD+sOyS5znLLza6DOHDthAvxOEVW1sRwibo5zOFkczohet81KXWaciTvXGV 01Ww== MIME-Version: 1.0 X-Received: by 10.182.214.98 with SMTP id nz2mr3347620obc.37.1382642564245; Thu, 24 Oct 2013 12:22:44 -0700 (PDT) Received: by 10.76.19.115 with HTTP; Thu, 24 Oct 2013 12:22:44 -0700 (PDT) Date: Thu, 24 Oct 2013 15:22:44 -0400 Message-ID: Subject: How to blink a sas drive light? From: Outback Dingo To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 19:22:45 -0000 I need to replace a disk connected to a SAS expander also now, there are 44 drives. How do i use this ses to determine the drive, can i make its light blink ?? im getting da19:mps0:0:35:0 SCSI sense ABORTED COMMAND asc:44,0 (Internal target failure) From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 24 19:57:30 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6C159E31 for ; Thu, 24 Oct 2013 19:57:30 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-wg0-x233.google.com (mail-wg0-x233.google.com [IPv6:2a00:1450:400c:c00::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0A9562709 for ; Thu, 24 Oct 2013 19:57:29 +0000 (UTC) Received: by mail-wg0-f51.google.com with SMTP id l18so2873406wgh.30 for ; Thu, 24 Oct 2013 12:57:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Px3FYKBym3mC5Ym53IcZwneHCBaRXA+MCPLt/K0ajFU=; b=GtzhH3MEKMsAwWzl+CVzmA27qfU1023hDOAyeNCZfU/HuanjkmmofVtlwnw6uHOOEQ cRz/ZXUkZulcf8GDWpEA5PzimVAi2k5etuuQ3fLzboUVyDfsqOfAyqpzZ0Ma+n0yCJg7 5CPkaKgVjn3XzK5e2zaoz2yd23kkWARO8i4m2TInGMXhUfLqvVjI2RfPvkQhNq55FTJM oK0VRr+bSs2uVirFsOMZKJNOcjT5OLN1CI27XzzBWLprwbAs+euv3Y7zkcMt/Zi2AwSl U0fcWbFmT9l+ZyL6Xznacy6og5oKmlMqosRpuVn/IMGl6zToV+b5oUN9ubKBS4gtYtQp Ogyw== MIME-Version: 1.0 X-Received: by 10.194.94.167 with SMTP id dd7mr4079972wjb.43.1382644648494; Thu, 24 Oct 2013 12:57:28 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.194.171.35 with HTTP; Thu, 24 Oct 2013 12:57:28 -0700 (PDT) In-Reply-To: References: Date: Thu, 24 Oct 2013 13:57:28 -0600 X-Google-Sender-Auth: mw_QDdSjsRpgLQDweVFRmH8ySjA Message-ID: Subject: Re: How to blink a sas drive light? From: Alan Somers To: Outback Dingo Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Oct 2013 19:57:30 -0000 You can either use sysutils/sg3_utils or ses(4). Using ses(4), take a look at l/usr/share/examples/ses/setencstat . If you set the status for a DEVICE or ARRAY_DEVICE element to something bad, then the SES processor may decide to blink the error light. It's up to the firmware, though. Using sg_ses from sg3_utils, you can set the FAULT REQSTD or IDENT bit for a particular DEVICE or ARRAY DEVICE. That is more likely to cause it to blink, but again, it's up to the expander vendor. Read the man page for details. To identify the slot where da19, resides, you can look at SES page 10. Using sg3_utils, do "sg_vpd -i da019" and find the SAS Address of the offending drive. Then do "sg_ses -p 10 /dev/ses0" and find the index of the Element whose SAS address matches. It will help to read the SES spec to understand all of the SES configuration pages. http://www.t10.org/members/w_ses3.htm -Alan On Thu, Oct 24, 2013 at 1:22 PM, Outback Dingo wrote: > I need to replace a disk connected to a SAS expander also now, there are 44 > drives. > How do i use this ses to determine the drive, can i make its light blink ?? > > im getting > > da19:mps0:0:35:0 SCSI sense ABORTED COMMAND asc:44,0 (Internal target > failure) > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 00:21:24 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C0ACAA3C for ; Fri, 25 Oct 2013 00:21:24 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-pb0-x22b.google.com (mail-pb0-x22b.google.com [IPv6:2607:f8b0:400e:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 930D12A26 for ; Fri, 25 Oct 2013 00:21:24 +0000 (UTC) Received: by mail-pb0-f43.google.com with SMTP id md12so3126555pbc.30 for ; Thu, 24 Oct 2013 17:21:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=K8u+oJn+OKk7IqrkVX8S7sQuFSDUZdDz2UWLufD/c38=; b=qYtV+UvcmXY6ZCwYj2QFpJ8v7A2UZS1MVR3zIuY8VUwprdboHFT+wuU6YnxYVFxdj6 yZc0iGA79iqXXGW+UnrzMmUrtj+lqD3etQevIFg53+T7mmL9K/0eNXBW+ia10S4ySgl1 3MLt+6nS8dQn09tWeDU6iMJC7UnwXN/i1JEBNqAa3OFWf+6e1O5bUa/zFEfgitnpmNQO DSsq5MfrtlLZnwgA/OO3U7prwKf5Ya2kdcgURhTV6BxGbb8R6jGHAryu50k/DaxLiAKy hzB5PCnrZ+Ih8DpswoXISyJ42OXfZFCIhrYNxXiSxy2Mq9nNW/u9p49Yt1THkRKcot6n ELjw== MIME-Version: 1.0 X-Received: by 10.66.139.166 with SMTP id qz6mr6330136pab.88.1382660483733; Thu, 24 Oct 2013 17:21:23 -0700 (PDT) Received: by 10.70.92.79 with HTTP; Thu, 24 Oct 2013 17:21:23 -0700 (PDT) In-Reply-To: References: Date: Thu, 24 Oct 2013 19:21:23 -0500 Message-ID: Subject: Re: How to blink a sas drive light? From: Adam Vande More To: Outback Dingo Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 00:21:24 -0000 On Thu, Oct 24, 2013 at 2:22 PM, Outback Dingo wrote: > I need to replace a disk connected to a SAS expander also now, there are 44 > drives. > How do i use this ses to determine the drive, can i make its light blink ?? > > im getting > > da19:mps0:0:35:0 SCSI sense ABORTED COMMAND asc:44,0 (Internal target > failure) > If the drive is still working enough: dd if=/dev/da19 of=/dev/null Otherwise wiring down cam devices in such a big array would be useful for situations like this. -- Adam From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 00:43:15 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 7B8E4335 for ; Fri, 25 Oct 2013 00:43:15 +0000 (UTC) (envelope-from outbackdingo@gmail.com) Received: from mail-pa0-x231.google.com (mail-pa0-x231.google.com [IPv6:2607:f8b0:400e:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4DB262B4A for ; Fri, 25 Oct 2013 00:43:15 +0000 (UTC) Received: by mail-pa0-f49.google.com with SMTP id lj1so3251503pab.36 for ; Thu, 24 Oct 2013 17:43:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=sjweKvI8v3AYwzbYN/H7YaDfE9lNOYx1OPw8O568Sq4=; b=PRi+3TdUpnxrQB/SGqqCG8JjQJKaLm9t5RLJZ+Z21EgEhqSrmdFL/pjqgxuHHptg1u op81vd0R8J/NacKTb2MVD91edYmzC5jMGSw/zwGSfMbfNBX6lOictb7L8bFjzsItu71P 5861d46vAe2/CpDq/c8kI+/xvRIBjifcIvMvV0bgAfvRB6AKtHPItu7wehhRCcnQF8PV LxrixuREmWca7Xh4LTX+VbbqJmJBuIq2UQyB8CZ5+ApQillbdrba1wqNClL7YYfZq4g4 3DpQx0cgIF3tbgAmLT817w/Iq/pn0jXn+lOKj+fcFvJqHfpAZGMSety5Cf3MoppnDVr2 vy6A== MIME-Version: 1.0 X-Received: by 10.66.121.131 with SMTP id lk3mr6383469pab.61.1382661794765; Thu, 24 Oct 2013 17:43:14 -0700 (PDT) Received: by 10.66.126.141 with HTTP; Thu, 24 Oct 2013 17:43:14 -0700 (PDT) In-Reply-To: References: Date: Thu, 24 Oct 2013 20:43:14 -0400 Message-ID: Subject: Re: How to blink a sas drive light? From: Outback Dingo To: Achim Patzner Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Adam Vande More , FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 00:43:15 -0000 On Thu, Oct 24, 2013 at 8:34 PM, Achim Patzner wrote: > On Thu, 24 Oct 2013 19:21:23 -0500 > Adam Vande More wrote: > >> Otherwise wiring down cam devices in such a big array would be useful for >> situations like this. >> > > I'd rather think GEOM labelling and letting nature take its course is the > best invention since sliced bread if you're dealing with more than two > drives (you might try fighting the squid salad you're facing with 24 > unmarked SAS lines coming out of some port extender or trying to remember > which drawer was connected to which link). And with a large number of > drives investing some money in a controller that will help you getting > things done (usually made by LSI these days) wouldn't be a mistake, either > (also considering the tools that are available). And as Alan Somers already > mentioned, there are those sg3 utils (http://sg.danny.cz/sg/sg3_** > utils.html ), too. > > > its got dual LSI controllers in it, on a enclosure, its a high end storage unit.... > Achim > From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 01:11:25 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2CD47113 for ; Fri, 25 Oct 2013 01:11:25 +0000 (UTC) (envelope-from sean_bruno@yahoo.com) Received: from nm11-vm3.bullet.mail.gq1.yahoo.com (nm11-vm3.bullet.mail.gq1.yahoo.com [98.136.218.158]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E35E82CCC for ; Fri, 25 Oct 2013 01:11:24 +0000 (UTC) Received: from [216.39.60.180] by nm11.bullet.mail.gq1.yahoo.com with NNFMP; 25 Oct 2013 01:11:18 -0000 Received: from [208.71.42.199] by tm16.bullet.mail.gq1.yahoo.com with NNFMP; 25 Oct 2013 01:11:18 -0000 Received: from [127.0.0.1] by smtp210.mail.gq1.yahoo.com with NNFMP; 25 Oct 2013 01:11:18 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1382663478; bh=VRC2K1BhShQN2KdwIHOW6MZVPdA4aCo2gqkr1vSK+tU=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Subject:From:Reply-To:To:In-Reply-To:References:Content-Type:Date:Message-ID:Mime-Version:X-Mailer; b=yqUxzPaO8VCY/NeFl0cQNqR41qpwlEc7onymfxafLGzBjmWW1/9JSwtLOTqsfQBRVNSXl1Jk4k0K3oc2QqyZYxVwl0SxkySXsSqjNzz/rdpVVoD9fcnf77JaZ3H33GnbWVFxgjgNt2bnuhgcRLpBnbFKJVvAeVuw1TT1azDw0iM= X-Yahoo-Newman-Id: 559524.65149.bm@smtp210.mail.gq1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: pq.97ekVM1l5IgmoBB0tgcNBSXcjaZIrzwSZHKtlkaLNJ3a XebrovPVAjHnLbCR5wOuc1SC_MxYylxvNNnsOGvmRdOWPIuDhLDTbxZ2PAN7 _U3N.rTBpOXiyIpGTEssUAPg2b1ql3lwQjbTnwr0oIxETq_2jXF5dQFQ5t5e Eev9edNqV4whsUsYFLThIY9W9gR7.ArydOEngIm4tjxpfAlIGvQ4BhCwbkuo yp1faJnNgziTLlpYUP0V5DAK3ZpGzq6Pd1SdA7nD5SMAvbjnG1EnOB_6CDOP 0PKsK0lJD7XCa1ZdlMm3oUOZCuvcGhgShKbyPwJ627d4OVmersmPKDbmo.nS At7GYoJT9VYDpEaKiHX7SMCGZjgDwMZXYg8tYva9R08ULqTOryhJtlFo.i1j 7GNXDuww6qwraMuleDRX7FZ8vuVfxX5PKfJ3tAEGId2Y3HVjxfCHAT3aYRmU Liym.OP1HPL._Lj.lWnBHyBY8i3W_Jqo.m22iB_gRYW07MlbTHRjFeQOjcdt cc_Wvd6Am.tj0dBpDRB8NT4aramAfrsebArECuPFT68xpF4H9A2avAHI6e4I J9Br1BM8UNdpYjvzjVd75GlMJKP_YFCyBxBLEsJem1h7.s.Rg X-Yahoo-SMTP: u5BKR6OswBC_iZJVfGRoMkTIpc8pEA4- X-Rocket-Received: from [192.168.100.5] (sean_bruno@63.138.121.126 with ) by smtp210.mail.gq1.yahoo.com with SMTP; 24 Oct 2013 18:11:18 -0700 PDT Subject: Re: gperf -- #define for if (0) ; else for From: Sean Bruno To: "freebsd-hackers@freebsd.org" In-Reply-To: <1382327705.2610.9.camel@localhost> References: <1382327705.2610.9.camel@localhost> Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-AqiqxfkeUIz5FoIv9ZY0" Date: Thu, 24 Oct 2013 21:11:12 -0400 Message-ID: <1382663472.2498.2.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 01:11:25 -0000 --=-AqiqxfkeUIz5FoIv9ZY0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable On Sun, 2013-10-20 at 23:55 -0400, Sean Bruno wrote: > I don't even know why this is a thing in our code base. Its generating > a lot of clang noise due to -Wdangling-else >=20 > /usr/src/gnu/usr.bin/gperf/../../../contrib/gperf/src/search.cc:417:15: > warning: add explicit braces to avoid dangling else [-Wdangling-else] > for (int i3 =3D imax; i3 >=3D 0; i3--) > ^ > /usr/src/gnu/usr.bin/gperf/../../../contrib/gperf/src/search.cc:39:22: > note: expanded from macro 'for' > #define for if (0) ; else for > ^ >=20 > I see no reason to continue this and propose the removal of the line in > contrib/gperf/src/search.cc=20 >=20 > 37 > 38 /* Assume ISO C++ 'for' scoping rule. */ > 39 #define for if (0) ; else for >=20 >=20 Commented out at svn R257085 sean --=-AqiqxfkeUIz5FoIv9ZY0 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (FreeBSD) iQEcBAABAgAGBQJSacUwAAoJEBkJRdwI6BaHHtsIAICBdJVQuWgFGoj7lT/dRSw9 KpVARzx6t14bOkE8jW0oUqmwC8qsvb3KBkRVqhvHYLwmP+2OmIulEbiLwkwKzw8W nGdXSv2Nz3IoZR1AxLCyo6dxvd2f0b+D5Mo0K3u7IuMSARuL444VN7HCoTj/Y+IO xPpV49aMM3O9LxuBqPwGVyQkGgzP43nskZRaQTHZVrODNBilhKQRhj72JdlLMF4J GJkHEC2BlDWqZq7a1r/jdVJ7QY94Cu2eoVIa9fEchyZhYYY8bFrNcK0vNCcL9EwM gZuTNQEq7s2AfTWUqliknIhhnfTxlAihFOLvVKLTAKNm04/P0qB/mZo4dZercoA= =cOef -----END PGP SIGNATURE----- --=-AqiqxfkeUIz5FoIv9ZY0-- From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 01:11:42 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 619E7211 for ; Fri, 25 Oct 2013 01:11:42 +0000 (UTC) (envelope-from outbackdingo@gmail.com) Received: from mail-pa0-x22c.google.com (mail-pa0-x22c.google.com [IPv6:2607:f8b0:400e:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3A5FB2CD9 for ; Fri, 25 Oct 2013 01:11:42 +0000 (UTC) Received: by mail-pa0-f44.google.com with SMTP id fb1so3292780pad.3 for ; Thu, 24 Oct 2013 18:11:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=/GMCIxfDjLwFU4bGcKMvnS5ySTxpW79tCasm2FeOPNQ=; b=NGXV5KBYFSivkgZgSHdwOO4iCQ0E87U7jzdMxim3/pfBLVy9fU7IrrgkPxILSwg/Qa Q+e/I096+NqxFvpOeABHn76rPYjlRlhWyszM4vR2zR1pH/7r6eIwjOffHignxlmp+ffZ E6Plcce/ZDVNB5rUxrEbK30qQLlQPJe4trpTDUdUDFpj0vYSaWxMypLfE4KSsdclUa6z uLB8B+RfuXujPV/4fa9idsnbaINbKKIjmFgn1R/69qZZGU35E4fIXfSJ9G53tJcrGoFb bVXPrQX3FkpH/uYYwCGm2oFZf9N2+KVILzy5o1gxEAXpnl92Xkpz99N+JIrMZuXiKJIU Nsjg== MIME-Version: 1.0 X-Received: by 10.66.150.69 with SMTP id ug5mr6639742pab.55.1382663501694; Thu, 24 Oct 2013 18:11:41 -0700 (PDT) Received: by 10.66.126.141 with HTTP; Thu, 24 Oct 2013 18:11:41 -0700 (PDT) In-Reply-To: <82839443-30E1-4243-A82C-E0AA12E56607@gsoft.com.au> References: <82839443-30E1-4243-A82C-E0AA12E56607@gsoft.com.au> Date: Thu, 24 Oct 2013 21:11:41 -0400 Message-ID: Subject: Re: How to blink a sas drive light? From: Outback Dingo To: "Daniel O'Connor" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Achim Patzner , FreeBSD Hackers , Adam Vande More X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 01:11:42 -0000 On Thu, Oct 24, 2013 at 8:59 PM, Daniel O'Connor wrote: > > On 25 Oct 2013, at 11:13, Outback Dingo wrote: > > its got dual LSI controllers in it, on a enclosure, its a high end > storage > > unit.... > > What driver does it use? > mfiutil has a locate command, I've never used it though. I have used > /dev/led for AHCI but I don't think mfi(4) hooks into the led(4) API > (sadly). > > its using the mps driver...... and im not so sure mfiutil works with the newer controllers its way out of date > -- > Daniel O'Connor software and network engineer > for Genesis Software - http://www.gsoft.com.au > "The nice thing about standards is that there > are so many of them to choose from." > -- Andrew Tanenbaum > GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C > > > > > > > From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 01:14:02 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 00D154F8 for ; Fri, 25 Oct 2013 01:14:01 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 630D32D0A for ; Fri, 25 Oct 2013 01:14:01 +0000 (UTC) Received: from ur.gsoft.com.au (Ur.gsoft.com.au [203.31.81.31]) (authenticated bits=0) by cain.gsoft.com.au (8.14.4/8.14.3) with ESMTP id r9P0xR46009847 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Fri, 25 Oct 2013 11:29:33 +1030 (CST) (envelope-from doconnor@gsoft.com.au) Subject: Re: How to blink a sas drive light? Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1816\)) Content-Type: text/plain; charset=us-ascii From: "Daniel O'Connor" In-Reply-To: Date: Fri, 25 Oct 2013 11:29:27 +1030 Content-Transfer-Encoding: quoted-printable Message-Id: <82839443-30E1-4243-A82C-E0AA12E56607@gsoft.com.au> References: To: Outback Dingo X-Mailer: Apple Mail (2.1816) X-Spam-Score: -3.552 () ALL_TRUSTED,BAYES_00,RP_MATCHES_RCVD X-Scanned-By: MIMEDefang 2.67 on 203.31.81.10 Cc: Achim Patzner , FreeBSD Hackers , Adam Vande More X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 01:14:02 -0000 On 25 Oct 2013, at 11:13, Outback Dingo wrote: > its got dual LSI controllers in it, on a enclosure, its a high end = storage > unit.... What driver does it use? mfiutil has a locate command, I've never used it though. I have used = /dev/led for AHCI but I don't think mfi(4) hooks into the led(4) API = (sadly). -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 01:27:41 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 95B7FB76 for ; Fri, 25 Oct 2013 01:27:41 +0000 (UTC) (envelope-from sean_bruno@yahoo.com) Received: from nm45-vm8.bullet.mail.bf1.yahoo.com (nm45-vm8.bullet.mail.bf1.yahoo.com [216.109.115.79]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 42EF72DD3 for ; Fri, 25 Oct 2013 01:27:40 +0000 (UTC) Received: from [98.139.212.144] by nm45.bullet.mail.bf1.yahoo.com with NNFMP; 25 Oct 2013 01:24:27 -0000 Received: from [98.139.211.192] by tm1.bullet.mail.bf1.yahoo.com with NNFMP; 25 Oct 2013 01:24:27 -0000 Received: from [127.0.0.1] by smtp201.mail.bf1.yahoo.com with NNFMP; 25 Oct 2013 01:24:27 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1382664267; bh=RUHk7LDKgwKq/Hyi9y4K39AJfA8H649oSuTN9fiKdso=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Subject:From:Reply-To:To:Cc:In-Reply-To:References:Content-Type:Date:Message-ID:Mime-Version:X-Mailer; b=SNFLxV4VrQTSWqIkUJE/VtdacaAlphGnqYHsMAzRZC3tCCwseq5U4KiXgmDKyCOpx2EdlKkUiCCr2SyHc9E86YGyDQyYaJjIM8VFHR6vgEe6VzNt2Mgco4erT6uRO0VApe7+41tPEQ+c6oJnpw2OwLD258xptzQrz0Rw0Q/DsDY= X-Yahoo-Newman-Id: 839144.98921.bm@smtp201.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: qUgMqbsVM1kSwOfUHXOzTWfGzhg8Isjz3oo4dJNb2w.5oBB 9N7Cj9szJgAqaouVz8lYE1qJgjYh4nPAPk37jlg6SrizbTUXjpuaSHhBVF7c Nkb4m2YbKu.TwzhvEfQyR8szDUJQjojEbYQMjNTJZaNb8OdyUH61SS8q4YPV 1yvCb0kj.lr8Q.MhsF.YDaJUs5BeQAZeuaa_n.vsJERFNmUWe8ZBnvWN59Oz SjrM_jyTsIgI6sX9m4sNMuce2C4UhNVYJy6lqSN6sgRHkKGqFyFU5..0wh3a wChhpSqJ_xH_chYpCltMcT5vBH67eqVpQDVR0fvq6QA6tnGa2BIRbRsLRPlS pczW2o69pbQweanwK.2XECigSi0NICvsSlUfAQRuIOaf5gzhH_E4lzKX7fsk Pekda4qVnyacdL1rZ6LzxMXB.spZokCeqHgWb1f0YDgcWwuqUHAxm8zo9NxL tgP7kzaHo76NRqSib5JUiSVoOcG8aCnHH6A88sNbxwf8iq1viohQuyEhYb6y b5SO1mOHp2Sc9k1WhqkUG9HQAn1Lta_vWikdRFdoHVKGP4KXDSBmCYs2kBCZ SIwpgy9pZXz_fm_SNCtLxOnEpXnG4FQ_jWv3fBaNwXYAyWM_V7CgQPuGnl.B gpF3HE3aiK7XNoeGJyVAMhxsPo_E0_Q-- X-Yahoo-SMTP: u5BKR6OswBC_iZJVfGRoMkTIpc8pEA4- X-Rocket-Received: from [192.168.100.5] (sean_bruno@63.138.121.126 with ) by smtp201.mail.bf1.yahoo.com with SMTP; 24 Oct 2013 18:24:27 -0700 PDT Subject: Re: How to blink a sas drive light? From: Sean Bruno To: Outback Dingo In-Reply-To: References: <82839443-30E1-4243-A82C-E0AA12E56607@gsoft.com.au> Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-TLOsSc9xmuqWshbGz5wX" Date: Thu, 24 Oct 2013 21:24:24 -0400 Message-ID: <1382664264.2498.10.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Cc: Adam Vande More , FreeBSD Hackers , Achim Patzner X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 01:27:41 -0000 --=-TLOsSc9xmuqWshbGz5wX Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable On Thu, 2013-10-24 at 21:11 -0400, Outback Dingo wrote: > > > > What driver does it use? > > mfiutil has a locate command, I've never used it though. I have used > > /dev/led for AHCI but I don't think mfi(4) hooks into the led(4) API > > (sadly). > > > > > its using the mps driver...... and im not so sure mfiutil works with > the > newer controllers its way out of date=20 mfiutil --> mfi(4) raid controller. Not what you have. There's no direct mps(4) tool in FreeBSD, but LSI *does* have something on their website you can use for Science on FreeBSD. I just don't know what controller you have, so I couldn't look directly. http://www.lsi.com/support/pages/download-search.aspx sean --=-TLOsSc9xmuqWshbGz5wX Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (FreeBSD) iQEcBAABAgAGBQJSachIAAoJEBkJRdwI6BaH/3cH/2jdNEUpZhmkv2TAlDSYUddM tLRN6hrysB0bjEZxmICIIkgV0hH1f8ZyVNUo5DMEXSIwbfpcYXq9cLXFsXR4en1H cS+vx5nipOVjICyckf0p6BnSC7MdHw23m0TyGc+LOfymtnY/Q3LjjTo7wKz6RsNE tE3RllO+oBwsMdEIuMzyoCRWHaXjvOb064dJP5MAvz0vgnelFTjPPCOfp4QKpm/J PUI6Xe/pSc7kSLbFWtHzy2ONQFmozoW2eqEDTCRCovWosRrkvioHoGf5fI9+t6Mz jL3CWPaE3UN8gLppkDgsyBoQKnuJ++CIvM1GZfQBwvPNoSdCVH1+ziz0DZMUIJ4= =n2hu -----END PGP SIGNATURE----- --=-TLOsSc9xmuqWshbGz5wX-- From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 01:30:43 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 98F69EBD; Fri, 25 Oct 2013 01:30:43 +0000 (UTC) (envelope-from outbackdingo@gmail.com) Received: from mail-oa0-x22c.google.com (mail-oa0-x22c.google.com [IPv6:2607:f8b0:4003:c02::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 542552E35; Fri, 25 Oct 2013 01:30:43 +0000 (UTC) Received: by mail-oa0-f44.google.com with SMTP id l20so335918oag.3 for ; Thu, 24 Oct 2013 18:30:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=+P8HW50HZZFRxxExhpi8wuOYVb/RkpPB0zE8Jt+O9rE=; b=n0rCwZ8FwRVexMuT+0QogkolS6e/AdALTRYWmGLyKTfgTae3TwcUm9R9Ztz5WFRG8F Jvt0IrLVaRCBH7TvIQZ8nntg3x0iADjLE05XIvEikD0E5FlQn3gMiNsz6w8r9D/4IBEw oYIzZ3EuI9DK5pi07Kq2/xo8ONHdXF0a2aaXSpm7B7AFqzEMWQu1JQY5vAN0H0Rh/dnx DPC2waqDO9Wf0Ms2PBfgUSYONvx7o1aCEeMl8bwfSOq8qilq+MQoYfQGQVsqqsjANuDW nGTvhoioEwPKG5vwSeMfILHn1DaEOBaNjcd7sKCXTCZw9A8vjidSrzmpibcvCbtHUGbA Bu9g== MIME-Version: 1.0 X-Received: by 10.182.213.97 with SMTP id nr1mr986707obc.48.1382664642461; Thu, 24 Oct 2013 18:30:42 -0700 (PDT) Received: by 10.76.19.115 with HTTP; Thu, 24 Oct 2013 18:30:42 -0700 (PDT) In-Reply-To: <1382664264.2498.10.camel@localhost> References: <82839443-30E1-4243-A82C-E0AA12E56607@gsoft.com.au> <1382664264.2498.10.camel@localhost> Date: Thu, 24 Oct 2013 21:30:42 -0400 Message-ID: Subject: Re: How to blink a sas drive light? From: Outback Dingo To: sbruno@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Adam Vande More , FreeBSD Hackers , Achim Patzner X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 01:30:43 -0000 On Thu, Oct 24, 2013 at 9:24 PM, Sean Bruno wrote: > On Thu, 2013-10-24 at 21:11 -0400, Outback Dingo wrote: > > > > > > What driver does it use? > > > mfiutil has a locate command, I've never used it though. I have used > > > /dev/led for AHCI but I don't think mfi(4) hooks into the led(4) API > > > (sadly). > > > > > > > > its using the mps driver...... and im not so sure mfiutil works with > > the > > newer controllers its way out of date > > mfiutil --> mfi(4) raid controller. Not what you have. > > There's no direct mps(4) tool in FreeBSD, but LSI *does* have something > on their website you can use for Science on FreeBSD. I just don't know > what controller you have, so I couldn't look directly. > > > http://www.lsi.com/support/pages/download-search.aspx LSI SAS 9211-8i Controller...... thanks for the link, what asset type or utility am i looking for ? > > > sean > > From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 01:34:37 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6AE78297 for ; Fri, 25 Oct 2013 01:34:37 +0000 (UTC) (envelope-from ap@bnc.net) Received: from mailomat.net (mailomat.net [81.20.89.254]) (using TLSv1 with cipher DES-CBC3-SHA (168/168 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EE6382E69 for ; Fri, 25 Oct 2013 01:34:36 +0000 (UTC) X-Junk-Score: 50 [XX] X-SpamCatcher-Score: 50 [XX] X-Junk-Score: 0 [] X-Cloudmark-Score: 0 [] X-Cloudmark-Analysis: v=2.1 cv=Ool2lUPt c=1 sm=1 tr=0 a=/nRDZZJyYxTE/j0HnTmKXw==:117 a=/nRDZZJyYxTE/j0HnTmKXw==:17 a=y6j5TWOJ03kA:10 a=Cln8rwbaxfUA:10 a=IkcTkHD0fZMA:10 a=ZDwDCB9QlRsA:10 a=JysbXFYnAAAA:8 a=dAPAsP0gAAAA:8 a=h2mDgNjbVZwA:10 a=pGLkceISAAAA:8 a=uAKUDkwnAAAA:8 a=8x8qVowgzpivQ6JSBCcA:9 a=QEXdDO2ut3YA:10 a=dGBlhIIZPtEA:10 a=MSl-tDqOz04A:10 Received: from [194.39.192.125] (account bnc-mail@mailrelay.mailomat.net HELO bnc.net) by mailomat.net (CommuniGate Pro SMTP 6.0.5) with ESMTPSA id 63388327; Fri, 25 Oct 2013 02:34:18 +0200 X-Junk-Score: 50 [XX] X-SpamCatcher-Score: 50 [XX] Received: from [192.168.200.188] (account ap@bnc.net) by bnc.net (CommuniGate Pro XIMSS 6.0.5) with XIMSS id 6991507; Fri, 25 Oct 2013 02:34:17 +0200 X-Mailer: CommuniGate Pronto! 6.0.5 Subject: Re: How to blink a sas drive light? MIME-Version: 1.0 From: "Achim Patzner" In-Reply-To: References: To: "Adam Vande More" , "Outback Dingo" Date: Fri, 25 Oct 2013 02:34:17 +0200 Message-ID: Content-Type: text/plain;charset="utf-8"; format="flowed" Content-Transfer-Encoding: 8bit Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 01:34:37 -0000 On Thu, 24 Oct 2013 19:21:23 -0500 Adam Vande More wrote: > Otherwise wiring down cam devices in such a big array would be useful for > situations like this. I'd rather think GEOM labelling and letting nature take its course is the best invention since sliced bread if you're dealing with more than two drives (you might try fighting the squid salad you're facing with 24 unmarked SAS lines coming out of some port extender or trying to remember which drawer was connected to which link). And with a large number of drives investing some money in a controller that will help you getting things done (usually made by LSI these days) wouldn't be a mistake, either (also considering the tools that are available). And as Alan Somers already mentioned, there are those sg3 utils (http://sg.danny.cz/sg/sg3_utils.html), too. Achim From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 01:52:39 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AEB0AE79 for ; Fri, 25 Oct 2013 01:52:39 +0000 (UTC) (envelope-from sean_bruno@yahoo.com) Received: from nm48-vm10.bullet.mail.bf1.yahoo.com (nm48-vm10.bullet.mail.bf1.yahoo.com [216.109.114.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4D8DD2F8D for ; Fri, 25 Oct 2013 01:52:38 +0000 (UTC) Received: from [98.139.215.143] by nm48.bullet.mail.bf1.yahoo.com with NNFMP; 25 Oct 2013 01:50:42 -0000 Received: from [98.139.213.14] by tm14.bullet.mail.bf1.yahoo.com with NNFMP; 25 Oct 2013 01:50:42 -0000 Received: from [127.0.0.1] by smtp114.mail.bf1.yahoo.com with NNFMP; 25 Oct 2013 01:50:42 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1382665842; bh=zkG8dshjdXFf6LSoqMAZ2ET2ViDbvnyv8BlzvQjhjaQ=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Subject:From:Reply-To:To:Cc:In-Reply-To:References:Content-Type:Date:Message-ID:Mime-Version:X-Mailer; b=R34Th3R2V9uI594oY8C1vWwpdPKygNKxwPrTgurSkEPTjf1nDUmWSFkZpODlU7lWEVGQhowsV6lTPSzzKTihwnQ6ICA+eDnwXESdonaGUTc/+HUPirtWqHbySTaYh2LL1tT1n36IO6a/KvGlIl6Cotx6OdEG4HEDPvUi+LPgQMs= X-Yahoo-Newman-Id: 309698.59497.bm@smtp114.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: pzLZOuYVM1lKrNdiOBlAjNiCRK1wTPs44arZ3svvdylbFdS yO9fL4kPKQ9WaC9GEbuT5qtEDKuX2IpcmbUvrOow997ed.vC1zZzNSb..lUc qA_CCMss9jskVYEorb6.yC1cDnrUjS5IqIGgFy6uJ.AlDpv4.1ZD.0hGQA3U tn3KpOQWNxaYmQPiuQwxZ2xrDni9nRHUfzz7yKXi17H_UlgmmODsAu1Q3Hf4 mDsza_kuvIc_UPmKfH7b4TZDZJURZRcxVU2ybPPcMd7YBkVqStIvBcDgMx_M Oo52RkYbQXm_GTp4WJ8UrCRkG2Z0QHJ5X.X5xPFDzC3icRECc7pFXxk0Dcx. NquVlrtbws1udZiG96gwfTsJwEqBDfKVx7NgJ5xyM_4oEU7nAFPfKRNoPDd. fKIbMDDYM6j0K1MazCh9aCXROSx88x5OoXDn1oCWctAPqfAWuORd.u0kSdvq xOLd_npA48vj.btCqAUBNOn7f88bnyNg5.R1VXNi2BFn0U_1d2UMuHVQ10s8 VukibwUDwdx_DAEnv7kWcD9hGI.0ADutqeG.yf8_ntm3gCnzbaDVlXllksXu RNsXib98Ir6UCftU0b0PlJd2dVPl0.BJ4eLEOww8v.W65SEV5SiQn.A.SkrD 2J6skAQaT7oYs0h0mZy4bFSMuEH5igiA- X-Yahoo-SMTP: u5BKR6OswBC_iZJVfGRoMkTIpc8pEA4- X-Rocket-Received: from [192.168.100.5] (sean_bruno@63.138.121.126 with ) by smtp114.mail.bf1.yahoo.com with SMTP; 24 Oct 2013 18:50:42 -0700 PDT Subject: Re: How to blink a sas drive light? From: Sean Bruno To: Outback Dingo In-Reply-To: References: <82839443-30E1-4243-A82C-E0AA12E56607@gsoft.com.au> <1382664264.2498.10.camel@localhost> Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-sVZfCKejJYbXdmgYX+ox" Date: Thu, 24 Oct 2013 21:50:41 -0400 Message-ID: <1382665841.2498.14.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Cc: Adam Vande More , FreeBSD Hackers , Achim Patzner X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 01:52:39 -0000 --=-sVZfCKejJYbXdmgYX+ox Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable On Thu, 2013-10-24 at 21:30 -0400, Outback Dingo wrote: > http://www.lsi.com/support/pages/download-search.aspx >=20 >=20 > LSI SAS 9211-8i Controller...... thanks for the link, what asset type > or utility am i looking for ?=20 Its going to be something for "UNIX" in this case. sean --=-sVZfCKejJYbXdmgYX+ox Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (FreeBSD) iQEcBAABAgAGBQJSac5xAAoJEBkJRdwI6BaH0dcH/0zrdsgLoZVOMdM0aTDCkOVs onnuD1xWUM1hARxnil5D9OYUzzd4v39uHw0kN49WImoVEzbez82fwXWuQjF4SfsZ yuwy8C2IMTuW3NFaGwPNEEuvCRcpSilqaBc7vOvuvtx37uUYrQ9N6daTEFkK8sa9 bjDZB21FvC1iTKGlC2n+U1lsJrd03h/OUya/9SZ1rAnXAsafE+xbnV45Robjlnc5 UVEcY9j0FNUmhSoEkyVYDoreBDYgWYhFBG5Cbq9sz2dOlMzUpPLjsFkvcviCrcws V9WMfYwdX5uX+ZLAYaws8FZ4VLsxBxwHcE3vuwQonBvXebEREDLxvAC0vVjH5kM= =7CFt -----END PGP SIGNATURE----- --=-sVZfCKejJYbXdmgYX+ox-- From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 01:57:42 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 05C7512E; Fri, 25 Oct 2013 01:57:42 +0000 (UTC) (envelope-from outbackdingo@gmail.com) Received: from mail-ob0-x22a.google.com (mail-ob0-x22a.google.com [IPv6:2607:f8b0:4003:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B483B2FC0; Fri, 25 Oct 2013 01:57:41 +0000 (UTC) Received: by mail-ob0-f170.google.com with SMTP id wp18so352439obc.29 for ; Thu, 24 Oct 2013 18:57:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=IQAXAbkQjDk6eLcOaqaeCGnFulWu8Hd5AhWEexCAY9w=; b=jZ5DLvVXs6iAwzkOsd52+mldeXt9wdBH3hkd+qYz6C82kaOUZOFs/l+dFEVO5LduWG xC/fZzPP9C07ZJy4u8za3/bpoTcRjop+rNRGww0zcBGiwUID/iJ2ca2vnTbQ1nkE6MKP eNt+CsIsBktCqsQHkR2yaKaRpMK5xdcs42EUYL3zGqbUwjEbx8NvbEbyeFnCVqv0MWMb k8QeKYcjHfRDjVqc8xgHDl57HTj5Q+exFSQBwqHKZSzA7sOz0xnUIpcOg69mg3Mvq8iF SQY57uBtgeVDFjnd+FE2BPsHRBt9g74c5u7nKFwdy/pzUidYMyPXHI7vSs97jiAaCk7B dYxQ== MIME-Version: 1.0 X-Received: by 10.182.85.167 with SMTP id i7mr1066850obz.60.1382666261052; Thu, 24 Oct 2013 18:57:41 -0700 (PDT) Received: by 10.76.19.115 with HTTP; Thu, 24 Oct 2013 18:57:41 -0700 (PDT) In-Reply-To: <1382665841.2498.14.camel@localhost> References: <82839443-30E1-4243-A82C-E0AA12E56607@gsoft.com.au> <1382664264.2498.10.camel@localhost> <1382665841.2498.14.camel@localhost> Date: Thu, 24 Oct 2013 21:57:41 -0400 Message-ID: Subject: Re: How to blink a sas drive light? From: Outback Dingo To: sbruno@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Adam Vande More , FreeBSD Hackers , Achim Patzner X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 01:57:42 -0000 On Thu, Oct 24, 2013 at 9:50 PM, Sean Bruno wrote: > On Thu, 2013-10-24 at 21:30 -0400, Outback Dingo wrote: > > http://www.lsi.com/support/pages/download-search.aspx > > > > > > LSI SAS 9211-8i Controller...... thanks for the link, what asset type > > or utility am i looking for ? > > Its going to be something for "UNIX" in this case. > > the only thing freeBSD wise is see is the sas2flash utility, which honestly is pretty broken anyway > sean > From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 02:24:02 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C84C2D9D for ; Fri, 25 Oct 2013 02:24:02 +0000 (UTC) (envelope-from sean_bruno@yahoo.com) Received: from nm47-vm10.bullet.mail.gq1.yahoo.com (nm47-vm10.bullet.mail.gq1.yahoo.com [67.195.87.188]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8B7B3218C for ; Fri, 25 Oct 2013 02:24:02 +0000 (UTC) Received: from [98.137.12.62] by nm47.bullet.mail.gq1.yahoo.com with NNFMP; 25 Oct 2013 02:21:10 -0000 Received: from [208.71.42.214] by tm7.bullet.mail.gq1.yahoo.com with NNFMP; 25 Oct 2013 02:21:10 -0000 Received: from [127.0.0.1] by smtp225.mail.gq1.yahoo.com with NNFMP; 25 Oct 2013 02:21:10 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1382667670; bh=5XW8hhwxuKL7HORuqqcyt/pc1Gxcx1vqE8pkfdta13w=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Subject:From:Reply-To:To:Cc:In-Reply-To:References:Content-Type:Date:Message-ID:Mime-Version:X-Mailer; b=SIuU9wIaSEgQSORY3k8QG03dnbYQ4oX3EEg4zavr1rUHrcS0UspsMSvKmQ4smfhBAwr4IOkIX3VhSVhKSG3PnuR7y9CH+uQlurHcqKTY5wXg+0EnJgJX7+aVR7UyaQQcYPdzddyS2BQGTGYCrewuEZSP51PcAQtAWlrEmAxkVOM= X-Yahoo-Newman-Id: 664133.43908.bm@smtp225.mail.gq1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: ydKlyMIVM1kOY_XieiBuEEEWW2DjWXESy4sOsQUWPchkhW. Pdi5_TGohuQTqGa4iaXoJvMOwuyEybscLiNlmMiIpVZNDGWykXVcXSUBZAFB Zpn_o4sxAh6St8w8hah85Cr53L8_9Kzo27nDPq_9GBacn0l9xGyII9EQnBP9 6nCeDDXjrcJ4eshFb.h31n1U2wRJ69mHfWxcE.ZYeqlzHQlJrAWMQpuhQN.p lhEG6gDVOo4bethXPHwc9ymyttGsDOt4QUGTz3OPkzoH0GTFpFyA0DfNLsUO mjl32VW6_o_YOVxv53DfGQB_cU921BHKCY_ovTMUemO6zvXuyyOw0.xF5K0D 9kNCWAWEpWPhWX5L54CspnIE7.4xtheul5TkImnIlRIex7rN43vFV0NjUYhf A78si63orVtMk9E9xSEILxLFzJDOMCj7ksAh3bULnOXk7MfX4RjtFsDwXQiu WT8z1QqOEX2jnCoNZJdOdVHwHXTNQmW4YfeA6xHnHlTlKVkCrFMJGcZ4jMba qUmtQcWzvlZLaDEio51lgeeMCKx.a9IGGQ5ceAxvms_7XwZuXUkMTWUwathw - X-Yahoo-SMTP: u5BKR6OswBC_iZJVfGRoMkTIpc8pEA4- X-Rocket-Received: from [192.168.0.101] (sean_bruno@63.138.121.126 with ) by smtp225.mail.gq1.yahoo.com with SMTP; 25 Oct 2013 02:21:10 +0000 UTC Subject: Re: How to blink a sas drive light? From: Sean Bruno To: Outback Dingo In-Reply-To: References: <82839443-30E1-4243-A82C-E0AA12E56607@gsoft.com.au> <1382664264.2498.10.camel@localhost> <1382665841.2498.14.camel@localhost> Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-a40YXc/UYwTDRBtR9eyZ" Date: Thu, 24 Oct 2013 22:21:08 -0400 Message-ID: <1382667668.18382.1.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 02:24:02 -0000 --=-a40YXc/UYwTDRBtR9eyZ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable If you're up for Science, can you go upon you host's /usr/share/examples/ses build the tools there, and post the output of getencstat /dev/ses0 ? sean p.s. if you've already done this, just send me the output privately. --=-a40YXc/UYwTDRBtR9eyZ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (FreeBSD) iQEcBAABAgAGBQJSadWQAAoJEBkJRdwI6BaHlH8H/jHuOvRoNFHw1NknYNIt/5ga tsoNRDVQdORZ3h8IBEG/52UDlqpDmiaYm2vUIaUzufUse01QLCZsQqcDrk/dscde RcD+SFE6/gvhZC0Qy1iOZyGUHL+hSYfszvLkzYTu4OF/xmrxL6lyPjtW3E/Na/R1 vqJMWqURmytMqoxX2so/rl4dtfVp8ysyd3Db8Xb8DmlKxaRsiPyCDvOHkjPyD+9h UzLoNhELDB81M2cUWzsg7324mv5OOFzkLgr2djnuRiat/6h/l9kAze9v97bo34DR 3rG0KJkgQt+sbXVtrKcnpgv6vPAmRAiDQprAibgMMPdZYX5Q90/30ryNlisBVOo= =I/2X -----END PGP SIGNATURE----- --=-a40YXc/UYwTDRBtR9eyZ-- From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 02:28:16 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id D460BCC; Fri, 25 Oct 2013 02:28:16 +0000 (UTC) (envelope-from outbackdingo@gmail.com) Received: from mail-ob0-x236.google.com (mail-ob0-x236.google.com [IPv6:2607:f8b0:4003:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8F22D21C5; Fri, 25 Oct 2013 02:28:16 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id wn1so378121obc.13 for ; Thu, 24 Oct 2013 19:28:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=DOq9kF1DVsCVdWx6IA1eLhFBTLC8OgQ5U07xekS9Fq8=; b=M/zQOFg6247GohfJJ/JvSSn8M8rgUSlGcSnEFtgHnQi2JWMBFdUy0Iorh1CPjh6If+ yMAPgnsTWJwoIOwbgpPAPGIzp7QnqyIlOMOw7VUCzpu8ewixEI2rvQcRt13qAy/au85P ccsgKpcKnvWYScojZHWu96nOMzdqjrCaASy68HVuDczNkUUDL3qx5ctyJQVpIgoS6IIS GksdAVCzk4Imo0RWm5ep3qv6+XjHphtl1v/ehSoRyMA4OKE8HG5C4B7ybccjwhJ+NGSW iEVHrPwQesoQfcAN/5rr1WgrSN9j5ncno2oWJ4ihOixgwlcZw3sJy1PhpXPCcNnOAE00 ngDA== MIME-Version: 1.0 X-Received: by 10.60.133.233 with SMTP id pf9mr1169503oeb.46.1382668095898; Thu, 24 Oct 2013 19:28:15 -0700 (PDT) Received: by 10.76.19.115 with HTTP; Thu, 24 Oct 2013 19:28:15 -0700 (PDT) In-Reply-To: <1382667668.18382.1.camel@localhost> References: <82839443-30E1-4243-A82C-E0AA12E56607@gsoft.com.au> <1382664264.2498.10.camel@localhost> <1382665841.2498.14.camel@localhost> <1382667668.18382.1.camel@localhost> Date: Thu, 24 Oct 2013 22:28:15 -0400 Message-ID: Subject: Re: How to blink a sas drive light? From: Outback Dingo To: sbruno@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 02:28:16 -0000 On Thu, Oct 24, 2013 at 10:21 PM, Sean Bruno wrote: > > If you're up for Science, can you go upon you > host's /usr/share/examples/ses > been down that path...... its a wonder theres no "simple" way to just say blink disk 18 from the cli > > build the tools there, and post the output of getencstat /dev/ses0 ? > > sean > > p.s. if you've already done this, just send me the output privately. > From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 02:36:58 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3B57C4A0; Fri, 25 Oct 2013 02:36:58 +0000 (UTC) (envelope-from outbackdingo@gmail.com) Received: from mail-oa0-x231.google.com (mail-oa0-x231.google.com [IPv6:2607:f8b0:4003:c02::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E89D72239; Fri, 25 Oct 2013 02:36:57 +0000 (UTC) Received: by mail-oa0-f49.google.com with SMTP id j10so373590oah.22 for ; Thu, 24 Oct 2013 19:36:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=obWnTWDc/UA58blaQ7nrr8miswWSFMTwJViR3N+7PSM=; b=HxFFD08QIF7XRa9z9oovzDBXiyJ/gpg6j5NuLSOlgmaQ1LYfhnvkEXDUPf7U//wt2j H0UkZ4XCPnEjwbQZ3t2YF7HR2YXFJ559yeMNlwswWhf8ucomcS7qbh/kv3KTvNFqKKYK ruIsrZNYEPtUtz76r2tmKNdbqZEzkWWOcQvZfVPBi220SAmr8QX4S6Wiidpwmiorftnt sUxMG6k+Z7ZGbz4vRxBV5pmVEelHtluEgR9Jn45MOYJRHzTzbbHqEJiTCLlTwm6aCSqW u2N7vQkhpwl/0K0B3gm/dFpxIatXZkxCOPVZf6VJ+aBcjDTGm2NjhVaDongGDpJBKKU0 B9Gg== MIME-Version: 1.0 X-Received: by 10.60.36.133 with SMTP id q5mr1052701oej.63.1382668617261; Thu, 24 Oct 2013 19:36:57 -0700 (PDT) Received: by 10.76.19.115 with HTTP; Thu, 24 Oct 2013 19:36:57 -0700 (PDT) In-Reply-To: References: <82839443-30E1-4243-A82C-E0AA12E56607@gsoft.com.au> <1382664264.2498.10.camel@localhost> <1382665841.2498.14.camel@localhost> <1382667668.18382.1.camel@localhost> Date: Thu, 24 Oct 2013 22:36:57 -0400 Message-ID: Subject: Re: How to blink a sas drive light? From: Outback Dingo To: sbruno@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 02:36:58 -0000 On Thu, Oct 24, 2013 at 10:28 PM, Outback Dingo wrote: > > > > On Thu, Oct 24, 2013 at 10:21 PM, Sean Bruno wrote: > >> >> If you're up for Science, can you go upon you >> host's /usr/share/examples/ses >> > > been down that path...... its a wonder theres no "simple" way to just say > blink disk 18 from the cli > maybe i should be saying..... my disks are layed out in the chassis like left to right Row1 0-2, 0-1, 0-0, 1-2, 1-1, 1-0, 2-2, 2-1, 2-0, 3-2, 3-1, 3-0, 4-2, 4-1, 4-0 Row2 0-5, 0-4, 0-3, 1-5, 1-4, 1-3, 2-5, 2-4, 2-3, 3-5, 3-4, 3-3, 4-5, 4-4, 4-3 Row2 0-8, 0-7, 0-6, 1-8, 1-7, 1-6, 2-8, 2-7, 2-6, 3-8, 3-7, 3-6, 4-8, 4-7, 4-6 now ill assume 0- , 1- , 2- , 3- , and 4- are assigned an enclosure FreeBSD sees da19:mps0:0:35:0 SCSI sense ABORTED COMMAND asc:44,0 (Internal target failure) as dead, box scrolls forever at this and doesnt boot........ and its an SSD drive, there are a total of 4 in the chassis, and 40 SAS drives so.... how do i correlate what FreeBSD says in relation to its location in the chassis..... maybe this clears up the dilemma....... and pull the right drive to replace it. > > >> >> build the tools there, and post the output of getencstat /dev/ses0 ? >> >> sean >> >> p.s. if you've already done this, just send me the output privately. >> > > From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 03:05:14 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 83B7A5E5 for ; Fri, 25 Oct 2013 03:05:14 +0000 (UTC) (envelope-from sean_bruno@yahoo.com) Received: from nm33-vm2.bullet.mail.bf1.yahoo.com (nm33-vm2.bullet.mail.bf1.yahoo.com [72.30.239.202]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3434A2465 for ; Fri, 25 Oct 2013 03:05:13 +0000 (UTC) Received: from [98.139.215.143] by nm33.bullet.mail.bf1.yahoo.com with NNFMP; 25 Oct 2013 03:03:00 -0000 Received: from [98.139.211.192] by tm14.bullet.mail.bf1.yahoo.com with NNFMP; 25 Oct 2013 03:03:00 -0000 Received: from [127.0.0.1] by smtp201.mail.bf1.yahoo.com with NNFMP; 25 Oct 2013 03:03:00 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1382670180; bh=dvuEaGOQ3XeyMSSsmWFvCIUm/aglGLaBWZsetYUIJno=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Subject:From:Reply-To:To:Cc:In-Reply-To:References:Content-Type:Date:Message-ID:Mime-Version:X-Mailer; b=pgimXp9ztRZ0SjKqWdx5Ft/fCKa1ZosckPOwOOtYCCu73W552xQFuTtCI/choJDV3bYe3U4LaOkDOjIY+eQG9kx7CUxNidrOsjTZFmLM0CNvJoj56JMAujbjpmMonk2whJFfN7ibXml1sx5ETWnQNM0a5fFG+mIGOOTB5aQeetE= X-Yahoo-Newman-Id: 53965.32950.bm@smtp201.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: MscEHyAVM1lGVqvVJmYlauWl5mjpt6G0KrytfxwQhOFswvG YKQ9IL3x47vbHIiCinGdOljQ4zRMuaAQPYQfkqGVRt9bE9actVhF7oFW80L8 6CuEW1ZlIqiDJ6xjeIGi0wvlAAgsS0bst8QvqOLNZzFZey11v5WFS_tqVlTh nDQ1lnEc9VDVJLJP2BQO3P6bebKGKiHF2kd5eUvHSIsnGBRx_XWxUV7Olni9 qvn1RWrDYeZeQsFcEWA.O.9FqPwbc.MdqxSN0ZpK8poT40yDdS74DGWwAig2 Wtrxv9VrybNRVSyjLl8SkElR.0859e24CEPtU5jiJpTbAtt4w.yxh7z54HkI ZT9gX3AlLq4OkmNSGj3MYp.tiCiHkxoLIuvj8RVGP5plYgSYfSu7bJkABzmU E5d7AN2VFTc8wNtShTKGZPH8VC3_HX0DI2JrQ9pDoHcaExh8jdrwPPwFOLvh 7RjRCs7jWJe20.k0M8HDk58S.wuDpPW44FavApuIAAlTsV.WMXuuBSFDA.CL B1Ea30LtIWWT.6X_0BcV6D_zi2EXgqEHzimVRsa58Y5r9LdBMxYnlxw-- X-Yahoo-SMTP: u5BKR6OswBC_iZJVfGRoMkTIpc8pEA4- X-Rocket-Received: from [192.168.0.101] (sean_bruno@63.138.121.126 with ) by smtp201.mail.bf1.yahoo.com with SMTP; 24 Oct 2013 20:03:00 -0700 PDT Subject: Re: How to blink a sas drive light? From: Sean Bruno To: Outback Dingo In-Reply-To: References: <82839443-30E1-4243-A82C-E0AA12E56607@gsoft.com.au> <1382664264.2498.10.camel@localhost> <1382665841.2498.14.camel@localhost> <1382667668.18382.1.camel@localhost> Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-NeTbqeh90kjpaOwurmBU" Date: Thu, 24 Oct 2013 23:02:58 -0400 Message-ID: <1382670178.18382.8.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 03:05:14 -0000 --=-NeTbqeh90kjpaOwurmBU Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable On Thu, 2013-10-24 at 22:36 -0400, Outback Dingo wrote: >=20 >=20 >=20 > On Thu, Oct 24, 2013 at 10:28 PM, Outback Dingo > wrote: > =20 > =20 > =20 > On Thu, Oct 24, 2013 at 10:21 PM, Sean Bruno > wrote: > =20 > If you're up for Science, can you go upon you > host's /usr/share/examples/ses > =20 > =20 > been down that path...... its a wonder theres no "simple" way > to just say blink disk 18 from the cli > =20 >=20 >=20 > maybe i should be saying..... my disks are layed out in the chassis > like >=20 >=20 > left to right=20 >=20 >=20 > Row1 0-2, 0-1, 0-0, 1-2, 1-1, 1-0, 2-2, 2-1, 2-0, 3-2, 3-1, 3-0, 4-2, > 4-1, 4-0 > Row2 0-5, 0-4, 0-3, 1-5, 1-4, 1-3, 2-5, 2-4, 2-3, 3-5, 3-4, 3-3, 4-5, > 4-4, 4-3 >=20 > Row2 0-8, 0-7, 0-6, 1-8, 1-7, 1-6, 2-8, 2-7, 2-6, 3-8, 3-7, 3-6, 4-8, > 4-7, 4-6 >=20 >=20 >=20 > now ill assume 0- , 1- , 2- , 3- , and 4- are assigned an enclosure >=20 >=20 > FreeBSD sees da19:mps0:0:35:0 SCSI sense ABORTED COMMAND asc:44,0 > (Internal target failure) >=20 >=20 > as dead, box scrolls forever at this and doesnt boot........ and its > an SSD drive, there are a total of 4 in the chassis, and 40 SAS drives > so.... how do i correlate what FreeBSD says in relation to its > location in the chassis..... maybe this clears up the dilemma....... > and pull the right drive to replace it. =20 >=20 >=20 > =20 Yeah, its clear from this what you want. It *should* be simple, but I can't tell the wiring though. I *think* you want to use sysutils/smp_utils ... maybe. It sure looks like its for just this type of thing. I don't have anything available to give you more of a clue though. But, don't do that if you don't have a /dev/ses0 device to talk to in the first place. I looked back through the thread and I didn't see that you tried this way of doing it yet. =20 sean --=-NeTbqeh90kjpaOwurmBU Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (FreeBSD) iQEcBAABAgAGBQJSad9hAAoJEBkJRdwI6BaHbK4H/2MRn5FN41g6aXn6mkU0QidV 5AfHCAKpwVAgJiwJDt3edodqVt5QftsyqoIx38O6xu64Q9zrCHmqqvRr+ofTfyQI woRrEQe/9gsRsil0eHfneOieqpcEyH400NFlMva25qPGl2ESEXj8YChK5WEJRCb/ 8SJVP71YFEweUj3mxKB+dtnkNArQjvnQv6TuSAx6ZtAznfriCtnGUPB8PieQa/Z/ 4d6fwynwnqX7TpoKqqQ4KzRphgnEdHsTwGfc3EGYhNQIRaewC6DjXF//DmdYnbqh Zet54ewTQE+a8hkAerDZ+zD8LepyaEP8fNuPfJuCMWdn+VBMTy4S/EVadDX8tIA= =WYVS -----END PGP SIGNATURE----- --=-NeTbqeh90kjpaOwurmBU-- From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 03:05:51 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3A96F79B; Fri, 25 Oct 2013 03:05:51 +0000 (UTC) (envelope-from outbackdingo@gmail.com) Received: from mail-ob0-x22c.google.com (mail-ob0-x22c.google.com [IPv6:2607:f8b0:4003:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7434E2482; Fri, 25 Oct 2013 03:05:49 +0000 (UTC) Received: by mail-ob0-f172.google.com with SMTP id gq1so395976obb.17 for ; Thu, 24 Oct 2013 20:05:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=NWk2A6dXX3EVrfii3fjsj8z+AheRBClnMVOOOKtORtA=; b=yAcQJd9b4gy3Cue31loMfsR63GJOjYL2rB/HYTnRG1RkMOJjB5N6x/94zbiE34aeBj HM0fciG/5RsnHpeSN9jHNDZpvgRHa0EyqDIEYs2Egh20IPj6qHGt+WHK6yVYQhEG4udM xzFRrykKrhEk52BM/ZuBbSWB7LV/TMCnF9ISiYzQJS/pTK8sLo0Hx0y/NSv8BmlYf1YY /C8LmdQnVhsL2SmPuIy/AXY8DNDG2nrqLSxOYTDYdUqL2pg1SiOsE804qcua4irg89pg o0y//4FrF0MCTclSzX1J1UBMxfSEJHUaMUSGjfU+jspgWfb+0kaa8UUotittE6Zortr9 vWiw== MIME-Version: 1.0 X-Received: by 10.182.88.202 with SMTP id bi10mr1235998obb.52.1382670348524; Thu, 24 Oct 2013 20:05:48 -0700 (PDT) Received: by 10.76.19.115 with HTTP; Thu, 24 Oct 2013 20:05:48 -0700 (PDT) In-Reply-To: <1382670178.18382.8.camel@localhost> References: <82839443-30E1-4243-A82C-E0AA12E56607@gsoft.com.au> <1382664264.2498.10.camel@localhost> <1382665841.2498.14.camel@localhost> <1382667668.18382.1.camel@localhost> <1382670178.18382.8.camel@localhost> Date: Thu, 24 Oct 2013 23:05:48 -0400 Message-ID: Subject: Re: How to blink a sas drive light? From: Outback Dingo To: sbruno@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 03:05:51 -0000 On Thu, Oct 24, 2013 at 11:02 PM, Sean Bruno wrote: > On Thu, 2013-10-24 at 22:36 -0400, Outback Dingo wrote: > > > > > > > > On Thu, Oct 24, 2013 at 10:28 PM, Outback Dingo > > wrote: > > > > > > > > On Thu, Oct 24, 2013 at 10:21 PM, Sean Bruno > > wrote: > > > > If you're up for Science, can you go upon you > > host's /usr/share/examples/ses > > > > > > been down that path...... its a wonder theres no "simple" way > > to just say blink disk 18 from the cli > > > > > > > > maybe i should be saying..... my disks are layed out in the chassis > > like > > > > > > left to right > > > > > > Row1 0-2, 0-1, 0-0, 1-2, 1-1, 1-0, 2-2, 2-1, 2-0, 3-2, 3-1, 3-0, 4-2, > > 4-1, 4-0 > > Row2 0-5, 0-4, 0-3, 1-5, 1-4, 1-3, 2-5, 2-4, 2-3, 3-5, 3-4, 3-3, 4-5, > > 4-4, 4-3 > > > > Row2 0-8, 0-7, 0-6, 1-8, 1-7, 1-6, 2-8, 2-7, 2-6, 3-8, 3-7, 3-6, 4-8, > > 4-7, 4-6 > > > > > > > > now ill assume 0- , 1- , 2- , 3- , and 4- are assigned an enclosure > > > > > > FreeBSD sees da19:mps0:0:35:0 SCSI sense ABORTED COMMAND asc:44,0 > > (Internal target failure) > > > > > > as dead, box scrolls forever at this and doesnt boot........ and its > > an SSD drive, there are a total of 4 in the chassis, and 40 SAS drives > > so.... how do i correlate what FreeBSD says in relation to its > > location in the chassis..... maybe this clears up the dilemma....... > > and pull the right drive to replace it. > > > > > > > > Yeah, its clear from this what you want. It *should* be simple, but I > can't tell the wiring though. > > I *think* you want to use sysutils/smp_utils ... maybe. It sure looks > like its for just this type of thing. I don't have anything available > to give you more of a clue though. > > But, don't do that if you don't have a /dev/ses0 device to talk to in > the first place. I looked back through the thread and I didn't see that > you tried this way of doing it yet. > Thanks ill look into this, due to multipath theres actually 16 ses devices, ill get back on the research > > sean > > > From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 03:52:39 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A2552C75 for ; Fri, 25 Oct 2013 03:52:39 +0000 (UTC) (envelope-from sean_bruno@yahoo.com) Received: from nm5-vm0.bullet.mail.bf1.yahoo.com (nm5-vm0.bullet.mail.bf1.yahoo.com [98.139.213.150]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 411222775 for ; Fri, 25 Oct 2013 03:52:39 +0000 (UTC) Received: from [98.139.214.32] by nm5.bullet.mail.bf1.yahoo.com with NNFMP; 25 Oct 2013 03:49:19 -0000 Received: from [98.139.213.12] by tm15.bullet.mail.bf1.yahoo.com with NNFMP; 25 Oct 2013 03:49:19 -0000 Received: from [127.0.0.1] by smtp112.mail.bf1.yahoo.com with NNFMP; 25 Oct 2013 03:49:19 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1382672959; bh=iK7knhuabGlu4j+YsN0lgzMjuSO8uLwSqk4L+GqTYeU=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Subject:From:Reply-To:To:Content-Type:Date:Message-ID:Mime-Version:X-Mailer; b=Py04RT55iO5Fos2f4wO5Dt/df9kZXxZ6OWDsX7sQsjLsm2yDR0uE8gnCYTRnqrjh2VpnwI1uXR2f3Q7FheKoB2evaQw6+F0dzQSftk/6l2dGCECZ2Yy4lBjy38LDa8PXjf7iD0Qww/4K/gRVPZlWI6mNSVU3jhYRegrNBlBtOns= X-Yahoo-Newman-Id: 691694.56303.bm@smtp112.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: DDzECDgVM1k50aKgS6jSsPhnT7BGmYrF265yzwq.ejADzWa YHInIPgRPxI_tAcHzA1xeXadc8BiTXw2Hrqkf59Y6aj3OYPWP659xNB5xpOr 8JCw6Uo0wFnfxGshjrD9wiqM5kN1Rx0.TmvM_YN2XXTPsreZN_2Kl9c1Bjn3 j70tYdCqf74pNtzleD2a0oWKlXH5cEt9GZBvZ4zK3b5B4mwac0BCCvBlN_JU SExr5TUOAbIBSPitiiWKDcHL5f_yZAmPLfPyA.GfdLYgUrsNv38z6ZDOHIe7 qgq0KMPlE785RyX_JZWyScVa01TCt0LJcuAtpk3RKPfR3yNBPSg4FEZrrKZX yTgF7sGLNRvj.yZ9TCDJEjZdGkv23dJueEDPqe8mJdz82nDM3pXa9.SNy6i. IrcW_q9GFQeAcHlrMtD86i9IuOoKcPk7dj6B0rRFon4KoG9TuUH2zSGlZIbG fTosZiP75n.gTju36cH1gW7PhhY0LXGQDP3oRFZu1lgxCjbykemb.d.4uUv1 f3.oZ8GshpnTXkgRMlJiK8ElDedNCMG.HoSQwgQFYecRtxIw6r45hMA-- X-Yahoo-SMTP: u5BKR6OswBC_iZJVfGRoMkTIpc8pEA4- X-Rocket-Received: from [192.168.0.101] (sean_bruno@63.138.121.126 with ) by smtp112.mail.bf1.yahoo.com with SMTP; 24 Oct 2013 20:49:19 -0700 PDT Subject: warning spew from cddl libnvpair.c From: Sean Bruno To: hackers@freebsd.org Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-bW3ehibkUSKStlLpZweE" Date: Thu, 24 Oct 2013 23:49:17 -0400 Message-ID: <1382672957.18382.11.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 03:52:39 -0000 --=-bW3ehibkUSKStlLpZweE Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable libnvpair.c has some macros and preprocessor directives that make clang's -Wformat-security very unhappy. /home/sbruno/bsd/head/cddl/lib/libnvpair/../../../cddl/contrib/opensolaris/= lib/libnvpair/libnvpair.c:245:1: warning: format string is not a string lit= eral (po tentially insecure) [-Wformat-security] NVLIST_ARRPRTFUNC(byte_array, uchar_t, uchar_t, "0x%2.2x") ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/sbruno/bsd/head/cddl/lib/libnvpair/../../../cddl/contrib/opensolaris/= lib/libnvpair/libnvpair.c:238:23: note: expanded from macro 'NVLIST_ARRPRTF= UNC' (void) fprintf(fp, pctl->nvprt_btwnarrfmt); \ I don't see a real graceful way out of this. Also, this is totally "legit" C, so I don't see any reason to generate these warnings. Can someone educate me on either: 1. fixing these warnings the right way 2. how to disable the warning flags/makefile magic sean --=-bW3ehibkUSKStlLpZweE Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (FreeBSD) iQEcBAABAgAGBQJSaeo9AAoJEBkJRdwI6BaHtaEH/jQVI99HWBBB5g9dELNwT72r WXiLYrEpbwtWME3b0vOM92i/yh5hr8ei5xHzPpN+b3Y3DoKA0D3jNyYJi0uFvwvD ZbEmTB/VJpYssfDQL/a9uh+HTRe1gNS1YMZ7rNu63X3P8oLHN+EeMzEbHKLKVH62 DHoHPcYEhVvTHSNJ0Sk33oseL1DDxC+tb6laq37Zd6KQedqTnFt1P34BmbxbDZmN 0se2k3lZImaJQb1Isv/+zJPi4EE7LqvXEnxi6yW2tV6Ibl109RkzWvtR3gTUDqc1 xXJ93HmWDkvimdXTBzDN2qSKZs8ylM5GT5k4RAhvSa9XXBHZyptTQiObsMIwqnU= =EkFj -----END PGP SIGNATURE----- --=-bW3ehibkUSKStlLpZweE-- From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 06:38:43 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 93FE1F66 for ; Fri, 25 Oct 2013 06:38:43 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E66B12003 for ; Fri, 25 Oct 2013 06:38:42 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA11202; Fri, 25 Oct 2013 09:38:35 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VZb2Z-000A4H-15; Fri, 25 Oct 2013 09:38:35 +0300 Message-ID: <526A11B2.6090008@FreeBSD.org> Date: Fri, 25 Oct 2013 09:37:38 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Vitalij Satanivskij Subject: Re: FreeBSD 10.0-BETA1 #8 r256765M spend too much time in locks References: <20131024074826.GA50853@hell.ukr.net> <20131024075023.GA52443@hell.ukr.net> <20131024115519.GA72359@hell.ukr.net> <20131024165218.GA82686@hell.ukr.net> In-Reply-To: <20131024165218.GA82686@hell.ukr.net> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 06:38:43 -0000 When that high load happens again could you please run some profiling tool that is capable of capturing the whole stacks of hot code paths? I can suggest two alternatives: 1. hwpmc pmcstat -S instructions -O sample.out pmcstat -R sample.out -G summary.out 2. The following DTrace script: profile:::profile-1113 /!(curthread->td_flags & 0x20)/ { @stacks[stack()] = count(); } END { trunc(@stacks, 10); printa(@stacks); } -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 07:23:46 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 7214480; Fri, 25 Oct 2013 07:23:46 +0000 (UTC) (envelope-from satan@ukr.net) Received: from hell.ukr.net (hell.ukr.net [212.42.67.68]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2CCF22302; Fri, 25 Oct 2013 07:23:46 +0000 (UTC) Received: from satan by hell.ukr.net with local ID 1VZbkF-0008AU-M5 ; Fri, 25 Oct 2013 10:23:43 +0300 Date: Fri, 25 Oct 2013 10:23:43 +0300 From: Vitalij Satanivskij To: Andriy Gapon Subject: Re: FreeBSD 10.0-BETA1 #8 r256765M spend too much time in locks Message-ID: <20131025072343.GA31310@hell.ukr.net> References: <20131024074826.GA50853@hell.ukr.net> <20131024075023.GA52443@hell.ukr.net> <20131024115519.GA72359@hell.ukr.net> <20131024165218.GA82686@hell.ukr.net> <526A11B2.6090008@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <526A11B2.6090008@FreeBSD.org> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: Vitalij Satanivskij , freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 07:23:46 -0000 http://quad.org.ua/profiling.tgz results of both methods but for pmcstat to few buffers configured by default so not all statistics in summary ^( Andriy Gapon wrote: AG> AG> When that high load happens again could you please run some profiling tool that AG> is capable of capturing the whole stacks of hot code paths? AG> AG> I can suggest two alternatives: AG> AG> 1. hwpmc AG> pmcstat -S instructions -O sample.out AG> pmcstat -R sample.out -G summary.out AG> AG> 2. The following DTrace script: AG> AG> profile:::profile-1113 AG> /!(curthread->td_flags & 0x20)/ AG> { AG> AG> @stacks[stack()] = count(); AG> } AG> AG> END AG> { AG> trunc(@stacks, 10); AG> printa(@stacks); AG> } AG> -- AG> Andriy Gapon AG> _______________________________________________ AG> freebsd-hackers@freebsd.org mailing list AG> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers AG> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 09:31:19 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AA365F1A; Fri, 25 Oct 2013 09:31:19 +0000 (UTC) (envelope-from ap@bnc.net) Received: from mailomat.net (mailomat.net [81.20.89.254]) (using TLSv1 with cipher DES-CBC3-SHA (168/168 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 13EEC2927; Fri, 25 Oct 2013 09:31:18 +0000 (UTC) X-Junk-Score: 2 [X] X-SpamCatcher-Score: 2 [X] X-Junk-Score: 0 [] X-Cloudmark-Score: 0 [] X-Cloudmark-Analysis: v=2.1 cv=Ool2lUPt c=1 sm=1 tr=0 a=/nRDZZJyYxTE/j0HnTmKXw==:117 a=/nRDZZJyYxTE/j0HnTmKXw==:17 a=y6j5TWOJ03kA:10 a=Cln8rwbaxfUA:10 a=IkcTkHD0fZMA:10 a=ZDwDCB9QlRsA:10 a=JysbXFYnAAAA:8 a=dAPAsP0gAAAA:8 a=h2mDgNjbVZwA:10 a=pGLkceISAAAA:8 a=6u0sMF4ZwaLBgW30JmYA:9 a=QEXdDO2ut3YA:10 a=MSl-tDqOz04A:10 a=Jr6ESxdAV8kA:10 a=HX2QqK2MryAA:10 Received: from [194.39.192.125] (account bnc-mail@mailrelay.mailomat.net HELO bnc.net) by mailomat.net (CommuniGate Pro SMTP 6.0.5) with ESMTPSA id 63396200; Fri, 25 Oct 2013 11:31:08 +0200 X-Junk-Score: 2 [X] X-SpamCatcher-Score: 2 [X] Received: from [192.168.200.188] (account ap@bnc.net) by bnc.net (CommuniGate Pro XIMSS 6.0.5) with XIMSS id 6991550; Fri, 25 Oct 2013 11:31:08 +0200 X-Mailer: CommuniGate Pronto! 6.0.5 Subject: Re: How to blink a sas drive light? MIME-Version: 1.0 From: "Achim Patzner" In-Reply-To: References: <82839443-30E1-4243-A82C-E0AA12E56607@gsoft.com.au> <1382664264.2498.10.camel@localhost> To: "Outback Dingo" , Date: Fri, 25 Oct 2013 11:31:08 +0200 Message-ID: Content-Type: text/plain;charset="utf-8"; format="flowed" Content-Transfer-Encoding: 8bit Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 09:31:19 -0000 On Thu, 24 Oct 2013 21:30:42 -0400 Outback Dingo wrote: > utility am i looking for ? CmdTool2 (and don't forget the manual or you'll suffer from "if I had a 40" display at 400dpi" envy. If nothing can be found, take a look at Intel, HP or Dell. Achim From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 10:09:15 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3B6DC784 for ; Fri, 25 Oct 2013 10:09:15 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 5DC662AFA for ; Fri, 25 Oct 2013 10:09:13 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA13362; Fri, 25 Oct 2013 13:09:03 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1VZeKE-000AHi-Sq; Fri, 25 Oct 2013 13:09:02 +0300 Message-ID: <526A4306.2060500@FreeBSD.org> Date: Fri, 25 Oct 2013 13:08:06 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0.1 MIME-Version: 1.0 To: Vitalij Satanivskij Subject: Re: FreeBSD 10.0-BETA1 #8 r256765M spend too much time in locks References: <20131024074826.GA50853@hell.ukr.net> <20131024075023.GA52443@hell.ukr.net> <20131024115519.GA72359@hell.ukr.net> <20131024165218.GA82686@hell.ukr.net> <526A11B2.6090008@FreeBSD.org> <20131025072343.GA31310@hell.ukr.net> In-Reply-To: <20131025072343.GA31310@hell.ukr.net> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 10:09:15 -0000 on 25/10/2013 10:23 Vitalij Satanivskij said the following: > > > http://quad.org.ua/profiling.tgz > > results of both methods > > but for pmcstat to few buffers configured by default so not all statistics in summary ^( >From these profiling results alone I do not see pathologies. It looks like you have a lot of I/O going on[*]. My guess is that the I/O requests are sufficiently small and contiguous, so ZFS performs a lot for I/O aggregation. For that it allocates and then frees a lot of temporary buffers. And it seems that that's where the locks are greatly contended and CPU is burned. Specifically in KVA allocation in vmem_xalloc/vmem_xfree. You can try at least two approaches. 1. Disable I/O aggregation. See the following knobs: vfs.zfs.vdev.aggregation_limit: I/O requests are aggregated up to this size vfs.zfs.vdev.read_gap_limit: Acceptable gap between two reads being aggregated vfs.zfs.vdev.write_gap_limit: Acceptable gap between two writes being aggregated 2. Try to improve buffer allocation performance by using uma(9) for that. vfs.zfs.zio.use_uma=1 This is a boot time tunable. Footnotes: [*] But perhaps there is some pathology that causes all that I/O to happen. I can't tell that from the profiling data. So this could be another thing to try to check. > Andriy Gapon wrote: > AG> > AG> When that high load happens again could you please run some profiling tool that > AG> is capable of capturing the whole stacks of hot code paths? > AG> > AG> I can suggest two alternatives: > AG> > AG> 1. hwpmc > AG> pmcstat -S instructions -O sample.out > AG> pmcstat -R sample.out -G summary.out > AG> > AG> 2. The following DTrace script: > AG> > AG> profile:::profile-1113 > AG> /!(curthread->td_flags & 0x20)/ > AG> { > AG> > AG> @stacks[stack()] = count(); > AG> } > AG> > AG> END > AG> { > AG> trunc(@stacks, 10); > AG> printa(@stacks); > AG> } > AG> -- > AG> Andriy Gapon > AG> _______________________________________________ > AG> freebsd-hackers@freebsd.org mailing list > AG> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > AG> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 11:40:05 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4E1B8AF8; Fri, 25 Oct 2013 11:40:05 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from mo6-p00-ob.rzone.de (mo6-p00-ob.rzone.de [IPv6:2a01:238:20a:202:5300::1]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B23A82FDC; Fri, 25 Oct 2013 11:40:04 +0000 (UTC) X-RZG-AUTH: :JiIXek6mfvEEUpFQdo7Fj1/zg48CFjWjQv0cW+St/nW/auYssS93lfgJF/wLgTc= X-RZG-CLASS-ID: mo00 Received: from britannica.bec.de (ip-2-202-252-221.web.vodafone.de [2.202.252.221]) by smtp.strato.de (RZmta 32.11 DYNA|AUTH) with (TLSv1.0:DHE-RSA-AES256-SHA encrypted) ESMTPSA id 6057b9p9PB2KdQ ; Fri, 25 Oct 2013 13:39:47 +0200 (CEST) Received: by britannica.bec.de (sSMTP sendmail emulation); Fri, 25 Oct 2013 13:39:46 +0200 Date: Fri, 25 Oct 2013 13:39:46 +0200 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org, hackers@freebsd.org Subject: Re: warning spew from cddl libnvpair.c Message-ID: <20131025113946.GA15905@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org, hackers@freebsd.org References: <1382672957.18382.11.camel@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1382672957.18382.11.camel@localhost> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 11:40:05 -0000 On Thu, Oct 24, 2013 at 11:49:17PM -0400, Sean Bruno wrote: > libnvpair.c has some macros and preprocessor directives that make > clang's -Wformat-security very unhappy. > > /home/sbruno/bsd/head/cddl/lib/libnvpair/../../../cddl/contrib/opensolaris/lib/libnvpair/libnvpair.c:245:1: warning: format string is not a string literal (po > tentially insecure) [-Wformat-security] > NVLIST_ARRPRTFUNC(byte_array, uchar_t, uchar_t, "0x%2.2x") > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > /home/sbruno/bsd/head/cddl/lib/libnvpair/../../../cddl/contrib/opensolaris/lib/libnvpair/libnvpair.c:238:23: note: expanded from macro 'NVLIST_ARRPRTFUNC' > (void) fprintf(fp, pctl->nvprt_btwnarrfmt); \ > > > I don't see a real graceful way out of this. Also, this is totally > "legit" C, so I don't see any reason to generate these warnings. Can > someone educate me on either: > 1. fixing these warnings the right way > 2. how to disable the warning flags/makefile magic You can wrap the statement with #pragma GCC diagnostic push #pragma GCC diagnostic ignored "-Wformat-nonliteral" // Evil code here #pragma GCC diagnostic pop and/or use the corresponding _Pragma. Joerg From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 11:52:40 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 38258E36; Fri, 25 Oct 2013 11:52:40 +0000 (UTC) (envelope-from satan@ukr.net) Received: from hell.ukr.net (hell.ukr.net [212.42.67.68]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E6EB22095; Fri, 25 Oct 2013 11:52:39 +0000 (UTC) Received: from satan by hell.ukr.net with local ID 1VZfwN-00017k-5r ; Fri, 25 Oct 2013 14:52:31 +0300 Date: Fri, 25 Oct 2013 14:52:31 +0300 From: Vitalij Satanivskij To: Andriy Gapon Subject: Re: FreeBSD 10.0-BETA1 #8 r256765M spend too much time in locks Message-ID: <20131025115231.GA4274@hell.ukr.net> References: <20131024074826.GA50853@hell.ukr.net> <20131024075023.GA52443@hell.ukr.net> <20131024115519.GA72359@hell.ukr.net> <20131024165218.GA82686@hell.ukr.net> <526A11B2.6090008@FreeBSD.org> <20131025072343.GA31310@hell.ukr.net> <526A4306.2060500@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <526A4306.2060500@FreeBSD.org> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: Vitalij Satanivskij , freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 11:52:40 -0000 Thank you for help. I will try you recomendation next time when load grows. Just now some system was rebooted with some configuration changes, so to get high load we need to wait some time. Andriy Gapon wrote: AG> on 25/10/2013 10:23 Vitalij Satanivskij said the following: AG> > AG> > AG> > http://quad.org.ua/profiling.tgz AG> > AG> > results of both methods AG> > AG> > but for pmcstat to few buffers configured by default so not all statistics in summary ^( AG> AG> From these profiling results alone I do not see pathologies. AG> It looks like you have a lot of I/O going on[*]. AG> My guess is that the I/O requests are sufficiently small and contiguous, so ZFS AG> performs a lot for I/O aggregation. For that it allocates and then frees a lot AG> of temporary buffers. AG> And it seems that that's where the locks are greatly contended and CPU is AG> burned. Specifically in KVA allocation in vmem_xalloc/vmem_xfree. AG> AG> You can try at least two approaches. AG> AG> 1. Disable I/O aggregation. AG> See the following knobs: AG> vfs.zfs.vdev.aggregation_limit: I/O requests are aggregated up to this size AG> vfs.zfs.vdev.read_gap_limit: Acceptable gap between two reads being aggregated AG> vfs.zfs.vdev.write_gap_limit: Acceptable gap between two writes being aggregated AG> AG> 2. Try to improve buffer allocation performance by using uma(9) for that. AG> vfs.zfs.zio.use_uma=1 AG> This is a boot time tunable. AG> AG> Footnotes: AG> [*] But perhaps there is some pathology that causes all that I/O to happen. I AG> can't tell that from the profiling data. So this could be another thing to try AG> to check. AG> AG> > Andriy Gapon wrote: AG> > AG> AG> > AG> When that high load happens again could you please run some profiling tool that AG> > AG> is capable of capturing the whole stacks of hot code paths? AG> > AG> AG> > AG> I can suggest two alternatives: AG> > AG> AG> > AG> 1. hwpmc AG> > AG> pmcstat -S instructions -O sample.out AG> > AG> pmcstat -R sample.out -G summary.out AG> > AG> AG> > AG> 2. The following DTrace script: AG> > AG> AG> > AG> profile:::profile-1113 AG> > AG> /!(curthread->td_flags & 0x20)/ AG> > AG> { AG> > AG> AG> > AG> @stacks[stack()] = count(); AG> > AG> } AG> > AG> AG> > AG> END AG> > AG> { AG> > AG> trunc(@stacks, 10); AG> > AG> printa(@stacks); AG> > AG> } AG> > AG> -- AG> > AG> Andriy Gapon AG> > AG> _______________________________________________ AG> > AG> freebsd-hackers@freebsd.org mailing list AG> > AG> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers AG> > AG> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" AG> > AG> AG> AG> -- AG> Andriy Gapon AG> _______________________________________________ AG> freebsd-hackers@freebsd.org mailing list AG> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers AG> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 12:41:45 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id EC3E7D86 for ; Fri, 25 Oct 2013 12:41:45 +0000 (UTC) (envelope-from michael.schuh@gmail.com) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8B39B23A2 for ; Fri, 25 Oct 2013 12:41:45 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id h11so1000896wiv.16 for ; Fri, 25 Oct 2013 05:41:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=WDBz9ztUb9i9FHPLFvX8TRh31Doa/O38QOennaonmKY=; b=opz4SaPsTNJJ7+n48/lWlgYynuDEDZZuIyrQxvD3WvFg3ZI5XN9bipIqTz4rUsu3hr 4fIIbPa12FbpV2HEZxUm6xz6rU+rDx/Yf0Tnm8wf1LfdBDX6c4K3LTyg+VMyVx7s+3C5 tNX1IW7Ag0hXn5/cmJlhxBdF3ZMa+3ZoNymOSe1HrFB8EG+WHJ5+jM1/uADdUqZjzRXG 4dvxYYNrJeVKFj56QidJa4JC1wSlMHhpidPKZr4hDVPhvFXWPS+o/+IA5CQhml/UP+MV Ck8oJA4MSzmOuJjEFipuVj6gyXY9Rl20vBu/a7lgu7v8iAWswERddBnBxER1TqfD0ZP+ 4gXQ== MIME-Version: 1.0 X-Received: by 10.194.240.129 with SMTP id wa1mr7131797wjc.31.1382704903891; Fri, 25 Oct 2013 05:41:43 -0700 (PDT) Received: by 10.194.83.1 with HTTP; Fri, 25 Oct 2013 05:41:43 -0700 (PDT) Date: Fri, 25 Oct 2013 14:41:43 +0200 Message-ID: Subject: Re: How to blink a sas drive light? From: Michael Schuh To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 12:41:46 -0000 > > I need to replace a disk connected to a SAS expander also now, there are 44 > drives. > How do i use this ses to determine the drive, can i make its light blink ?? > > im getting > > da19:mps0:0:35:0 SCSI sense ABORTED COMMAND asc:44,0 (Internal target > failure) > > may be, one of http://www.freebsd.org/cgi/ports.cgi?query=LSI&stype=all&sektion=sysutils this results will help there From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 12:53:12 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 131CD296; Fri, 25 Oct 2013 12:53:12 +0000 (UTC) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (cust.static.213-3-30-106.swisscomdata.ch [213.3.30.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7CC0F243A; Fri, 25 Oct 2013 12:53:10 +0000 (UTC) Received: from insomnia.benzedrine.cx (localhost [127.0.0.1]) by insomnia.benzedrine.cx (8.14.6/8.14.5) with ESMTP id r9PCQhtq022198 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 25 Oct 2013 14:26:43 +0200 (MEST) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.14.6/8.14.5/Submit) id r9PCQhPH016909; Fri, 25 Oct 2013 14:26:43 +0200 (MEST) Date: Fri, 25 Oct 2013 14:26:43 +0200 From: Daniel Hartmeier To: freebsd-hackers@freebsd.org, hackers@freebsd.org Subject: Re: warning spew from cddl libnvpair.c Message-ID: <20131025122643.GC12556@insomnia.benzedrine.cx> References: <1382672957.18382.11.camel@localhost> <20131025113946.GA15905@britannica.bec.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131025113946.GA15905@britannica.bec.de> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 12:53:12 -0000 How is that valid? If nvprt_btwnarrfmt contains %, how would (void) fprintf(fp, pctl->nvprt_btwnarrfmt); \ not crash? And if it may not contain %, why not use (void) fprintf(fp, "%s", pctl->nvprt_btwnarrfmt); \ It can be set to arbitrary values with libnvpair.c nvlist_prtctl_setfmt() That's exactly why the compiler warns, no? If anything in the macro/preprocessor trickery is relevant, I'm not seeing it. Daniel From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 13:09:04 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4B94BA2E; Fri, 25 Oct 2013 13:09:04 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from mo6-p00-ob.rzone.de (mo6-p00-ob.rzone.de [IPv6:2a01:238:20a:202:5300::1]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id ADD572561; Fri, 25 Oct 2013 13:09:03 +0000 (UTC) X-RZG-AUTH: :JiIXek6mfvEEUpFQdo7Fj1/zg48CFjWjQv0cW+St/nW/avgusCdtw9+43oQGYIsZZxQQ0xcmsA== X-RZG-CLASS-ID: mo00 Received: from britannica.bec.de (cl-3506.cgn-01.de.sixxs.net [IPv6:2001:4dd0:ff00:db1::2]) by smtp.strato.de (RZmta 32.11 AUTH) with (TLSv1.0:DHE-RSA-AES256-SHA encrypted) ESMTPSA id u01a21p9PC7srU ; Fri, 25 Oct 2013 15:08:47 +0200 (CEST) Received: by britannica.bec.de (sSMTP sendmail emulation); Fri, 25 Oct 2013 15:08:46 +0200 Date: Fri, 25 Oct 2013 15:08:46 +0200 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org, hackers@freebsd.org Subject: Re: warning spew from cddl libnvpair.c Message-ID: <20131025130846.GA2388@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org, hackers@freebsd.org References: <1382672957.18382.11.camel@localhost> <20131025113946.GA15905@britannica.bec.de> <20131025122643.GC12556@insomnia.benzedrine.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131025122643.GC12556@insomnia.benzedrine.cx> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 13:09:04 -0000 On Fri, Oct 25, 2013 at 02:26:43PM +0200, Daniel Hartmeier wrote: > How is that valid? Different question. > If nvprt_btwnarrfmt contains %, how would > > (void) fprintf(fp, pctl->nvprt_btwnarrfmt); \ > > not crash? Correct. The version I see in NetBSD uses real format strings and arguments of fixed types, so it still has this kind of problems, but it is more clear that it is likely correct. Joerg From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 13:22:23 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3645CE44 for ; Fri, 25 Oct 2013 13:22:23 +0000 (UTC) (envelope-from sean_bruno@yahoo.com) Received: from nm48.bullet.mail.bf1.yahoo.com (nm48.bullet.mail.bf1.yahoo.com [216.109.114.64]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A70812673 for ; Fri, 25 Oct 2013 13:22:22 +0000 (UTC) Received: from [98.139.215.143] by nm48.bullet.mail.bf1.yahoo.com with NNFMP; 25 Oct 2013 13:16:35 -0000 Received: from [98.139.211.162] by tm14.bullet.mail.bf1.yahoo.com with NNFMP; 25 Oct 2013 13:16:35 -0000 Received: from [127.0.0.1] by smtp219.mail.bf1.yahoo.com with NNFMP; 25 Oct 2013 13:16:35 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1382706995; bh=xZWwp3PnqbtAUrM/SfPgtul8cB49bWz30tyBzphuGjc=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Subject:From:Reply-To:To:Cc:In-Reply-To:References:Content-Type:Date:Message-ID:Mime-Version:X-Mailer; b=Q14PjMMQd3xZi9vBGWJRbBpKwAo4XctUODjr2FplJvxtP/eXsf10m/JcHMcFp7QLzu6WhRW6M8Zfx7kHQ+xHynE6JF7JwHmlot+MnPFh4hLlSIIYQiaWs5oUZHqN2N/sjIQX+A/CwWaZcACUvnk6kEuiqSDgqO82HTYOgxWsEJA= X-Yahoo-Newman-Id: 268746.66920.bm@smtp219.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: eIEL0SYVM1kURxsC3D5vegHZKLKWt5PxXbbdXsbwVz1hahK XU6sq196uI0pJXEu9JOWB9uJsKy5smtWhGVhSmdi_.Tmuirk_X2X5entO5tf bLpbV7zqAE9BO4KWxQeON.rlQ0Bq2vMlm7cEV1Hxt5dn9Tyc.DkAkJel8Y6z mx5hjbth5F8o3S96Nq5rCiR9bRUSRitFdmJqrmNlm5N_uaXsRtYmbR3fmhkd SRQEt7ibsn.U.x2EKJrX.NTDtKLtPd3ienBQJL9EADX9Dpb.W3WjDZ6vCOBe yoY8lruSdEqymPlyfLWs3kka0NeDQnUEmClIEZoxV28VkXNR_h8vNHFIuEsN ayhCLQE2R4y1FR_aKObZChM3Wfm8E66_p1bbHG4N1aJ_KJ_6ypQqoIIcvz7h Gkp9dVK0qcaiOnAveXIuzr6agXw5GAV2dh_hSbExvbdUXsRu1juv71tQXtzD H_pFHAh4O6H65L1jtFYPxy6jNWT3.b.zI3HfwVnGSQevpyL2sAhtx9FVkBCP 2LHEUS7Mvpm.4wlx3NDOTsLmwFTuW1yIpcENhPPoVmDEKge9KROknA38- X-Yahoo-SMTP: u5BKR6OswBC_iZJVfGRoMkTIpc8pEA4- X-Rocket-Received: from [192.168.100.57] (sean_bruno@63.138.121.126 with ) by smtp219.mail.bf1.yahoo.com with SMTP; 25 Oct 2013 06:16:35 -0700 PDT Subject: Re: warning spew from cddl libnvpair.c From: Sean Bruno To: Daniel Hartmeier In-Reply-To: <20131025122643.GC12556@insomnia.benzedrine.cx> References: <1382672957.18382.11.camel@localhost> <20131025113946.GA15905@britannica.bec.de> <20131025122643.GC12556@insomnia.benzedrine.cx> Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-v4hO0we1ysCy4ncReLec" Date: Fri, 25 Oct 2013 09:16:33 -0400 Message-ID: <1382706993.2451.10.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Cc: freebsd-hackers@freebsd.org, hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: sbruno@freebsd.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 13:22:23 -0000 --=-v4hO0we1ysCy4ncReLec Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable On Fri, 2013-10-25 at 14:26 +0200, Daniel Hartmeier wrote: > How is that valid? >=20 > If nvprt_btwnarrfmt contains %, how would >=20 > (void) fprintf(fp, pctl->nvprt_btwnarrfmt); \ >=20 > not crash? >=20 If someone created a macro (e.g. NVLIST_PRTFUNC) that had % as an argument, bad things would happen, you would get a "incomplete format specifier warning on compile in that case. > And if it may not contain %, why not use >=20 > (void) fprintf(fp, "%s", pctl->nvprt_btwnarrfmt); \ >=20 That would mean writing a C function for each and every variable type to be= printed. right? > It can be set to arbitrary values with >=20 > libnvpair.c nvlist_prtctl_setfmt() >=20 > That's exactly why the compiler warns, no? I'm specifically looking at the macros that expand first (ARENDER for example). nvlist_prtctl_setfmt() doesn't seem to error check at all, but I'm not clear how its used outside of the library. >=20 > If anything in the macro/preprocessor trickery is relevant, I'm not > seeing it. sean --=-v4hO0we1ysCy4ncReLec Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (FreeBSD) iQEcBAABAgAGBQJSam8uAAoJEBkJRdwI6BaH7WoH/RnBGh1nouuzTZPu8e7kVmXK tk67Wrs4/QHXjnTAUmRQ0zpRGmMnRHdFhOlc97rwgWm4mAYbTFvR5Tdkt6668cLg quQlEUvGJk4HV3T48L7lNQeh5oQIo4YnpC40xNJXv4VQS1CTAXzzFkSn5CWqh10Q u2wMKmfY8+jhlD0Bbe1JWRUZlFL414jvZB1LXn1dkXYCtuNDvUB9rsHjOqAFfkrq xuvVDuk9Vhiqq8QOgKef6o95crGbxmcnYM0ckgzwFeGoph1qKlhVxCmqdU1p0yHQ gv18nJ7eFUj9RB4a94/0pNyUmR38JaFoUbuo3xZnOCrXIrYagIGNRfmmb9d38Ug= =qlTq -----END PGP SIGNATURE----- --=-v4hO0we1ysCy4ncReLec-- From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 14:08:17 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 99F11887; Fri, 25 Oct 2013 14:08:17 +0000 (UTC) (envelope-from ryao@gentoo.org) Received: from smtp.gentoo.org (dev.gentoo.org [IPv6:2001:470:ea4a:1:214:c2ff:fe64:b2d3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 74E3D292C; Fri, 25 Oct 2013 14:08:17 +0000 (UTC) Received: from [192.168.1.2] (pool-173-52-87-124.nycmny.fios.verizon.net [173.52.87.124]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: ryao) by smtp.gentoo.org (Postfix) with ESMTPSA id 7789733EE98; Fri, 25 Oct 2013 14:08:15 +0000 (UTC) Message-ID: <526A7B0A.4020800@gentoo.org> Date: Fri, 25 Oct 2013 10:07:06 -0400 From: Richard Yao User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130925 Thunderbird/17.0.9 MIME-Version: 1.0 To: sbruno@freebsd.org Subject: Re: warning spew from cddl libnvpair.c References: <1382672957.18382.11.camel@localhost> <20131025113946.GA15905@britannica.bec.de> <20131025122643.GC12556@insomnia.benzedrine.cx> <1382706993.2451.10.camel@localhost> In-Reply-To: <1382706993.2451.10.camel@localhost> X-Enigmail-Version: 1.5.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WiQN7B2oDh8uFILKme1ei2G9wdAPOnROH" Cc: Sean Bruno , Daniel Hartmeier , hackers@freebsd.org, freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 14:08:17 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --WiQN7B2oDh8uFILKme1ei2G9wdAPOnROH Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 10/25/2013 09:16 AM, Sean Bruno wrote: > I'm specifically looking at the macros that expand first (ARENDER for > example). nvlist_prtctl_setfmt() doesn't seem to error check at all, > but I'm not clear how its used outside of the library. This API appears to have no open source consumers on any Open ZFS platform. I do not know if there are any closed source consumers. --WiQN7B2oDh8uFILKme1ei2G9wdAPOnROH Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSansOAAoJECDuEZm+6ExkryMP/ih/yWsmTSNBwLbJDx10M5li XoYnsICFxKncWSKE2CUQ+QTXTePFe/uUqcbsK/sanvV7PUDSod3NNALu1aR0cxt5 4tI8L7yg/CbfwyVLaQ3n2z8pndw3Zc9qyGidVl8VYHQahk31ToKNMbOFhy+Q7MV6 t4UvXjTD5o/2MpjLoweQdvWvvOCAT+u0uSNMeIiWziMCDD4JviXnItKCyvi9E8My IYuwPv8V3YwxCGDZIQ7yz7UGLWROT+QRnuL3vfnrXyzJLBj0G121YsKCOpBq3R7B ZhSwfTPxWU9WLfMGEiTCeNyzNg1lmQ3w5P8V3VrRas1FzEy5bm4tc/6O2NznRHYj L1b98/u9n/gEFMGV08oO9BA6j2o0j1ZQgqQhSb47N67ZlAJalqHn8ZvCuYWjL7EE q65Gqfp2tIcYb44S6rb6W0D43x9TcDaE/GN/D6gaXo+XvU96ZYMe66IoRMyOexVZ xX7t0cynPjsm6k7UEdLBKj8GeU0FnrQsssclw0nhSnOHdhTaKjlIxPzI+vw9tldb B+/0fF+2PLIpw0Ndeyq1XPbFLG5BzbV3rqEButVbL4XqtwPMJPKvjUE4tikJVP13 J4SpGazhpfILR6ijG9rZHzEcV0DgSnrfHjCIV9moF2xOuDufBp4LRER74eyvKjKt ZPr2DiTo8E1wJ0PxTpkR =9cHO -----END PGP SIGNATURE----- --WiQN7B2oDh8uFILKme1ei2G9wdAPOnROH-- From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 25 16:01:55 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 35A3EDAC for ; Fri, 25 Oct 2013 16:01:55 +0000 (UTC) (envelope-from patrick_dkt@yahoo.com.hk) Received: from nm30-vm2.bullet.mail.sg3.yahoo.com (nm30-vm2.bullet.mail.sg3.yahoo.com [106.10.151.177]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9BC1D2011 for ; Fri, 25 Oct 2013 16:01:53 +0000 (UTC) Received: from [106.10.166.60] by nm30.bullet.mail.sg3.yahoo.com with NNFMP; 25 Oct 2013 15:59:05 -0000 Received: from [106.10.151.250] by tm17.bullet.mail.sg3.yahoo.com with NNFMP; 25 Oct 2013 15:59:05 -0000 Received: from [127.0.0.1] by omp1021.mail.sg3.yahoo.com with NNFMP; 25 Oct 2013 15:59:05 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 721773.24661.bm@omp1021.mail.sg3.yahoo.com Received: (qmail 90938 invoked by uid 60001); 25 Oct 2013 15:59:05 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com.hk; s=s1024; t=1382716745; bh=N/AD5FVO1XN+R5H24fVEPrcWU0KXtZp9zBTRNxk0lC4=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=IlqKCKl7qyOMJv0ALcGJMEwKFqkKX9p475q1Urmbwfbuhs3FCgIzEHtASwZQvxJkNf3ALUhUzRnL80qPHyEKEpwX0pk/dqRNtlBeGSAwlCJ4qKxSmGSrXFqj3ypIPPi4elf4/9L8nom354TeqE9UNV5puxwUisWbETNoQ/hnTnE= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com.hk; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=JtYk3b5XFziqGUHmnPjlOh/YnpraiQV/E3fsw0Xo4xkdvpm1rDFutST0C9tqr+CSQqbUlgMMc1C80LgNpeHJJNI0yU/ggB2udPFLmCWeWdusyPDeAE9w/1+u5AeeqAISWHBkMM7e4OLVtHkoFmNNLM6YAnFOCuwoPTN6mXipVu4=; X-YMail-OSG: h33wq0wVM1k0SfthnLZC7MiOKlNkR1D4RVwvX88GgRdw5t9 pFLQq.Fd4wHtAwJ2eP7v9lMGZWkpAALadiZNK4NLqsyDK0USbH4WeHee8TFW xMY.2HVOhDEt.EJNOURBJVe0fsnUOSsgxfysKU5ZYoccaIy2JL0Ny.keQTPe 5PpYhlPIVd.dO9gDsuUJzYU5zS3b8jJ_OSfHjk5AxeEFhImhWYVM8yls2PBV v3_5oU.rK9H2gBBs1m5l5Zbw38qufgPwHhufvCi6JmSBNLiGA2T2b4kutqaG 9kO99MMEC5MVe3A7tDAqUaX5TJzxNdCIqt_jV_1sPb65oMLliRnUxN4ENalv _GtY.qLOhQ312SPhoJw9XAMfHYd_khnH6yDoYthYqxByqAar5VI64GZF0wgC SXNqbrGR3Rq20uPonTX3Mbl7vkc4qPjT41ihc3D78qnx2b0CBwHotE8n0V1w XaIFkexodl0O.2R2hP34zz6dictS1rwUe97PF37.QWnvPojPzpsJ6AJ4Ns7N qNfw7TDXSTFj0cxAf7T7oaMnHIor9f1OZq93rcoJaBrfHoiy5hMEC_fY- Received: from [61.15.240.133] by web193506.mail.sg3.yahoo.com via HTTP; Fri, 25 Oct 2013 23:59:05 SGT X-Rocket-MIMEInfo: 002.001, SGkgLAoKSSBoYXZlIGFza2VkIHRoZSBxdWVzdGlvbiBpbiBmcmVlYnNkLXF1ZXN0aW9ucyBidXQgY2FuJ3QgZ2V0IHJlcGx5IHNvIEkgdHJpZWQgdG8gYXNrIGhlcmUgZmlyc3QgYmVmb3JlIHN1Ym1pdHRpbmcgYSBidWcgcmVwb3J0LgoKCkkgaGF2ZSBhIEZyZWVCU0QgOS4yIDY0IGJpdCBpbnN0YWxsZWQgaW4gYSBWTS4KVGhlIGhvc3QgaXMgV2luZG93cyA3IHdpdGggdm13YXJlIHdvcmtzdGF0aW9uLCB0aGUgaG9zdCBpcyB1c2luZyBhbiBhZHZhbmNlZCBmb3JtYXQgKDRrKSBoYXJkZGlzay4KCgpJIGhhdmUBMAEBAQE- X-Mailer: YahooMailWebService/0.8.160.587 References: Message-ID: <1382716745.90609.YahooMailNeo@web193506.mail.sg3.yahoo.com> Date: Fri, 25 Oct 2013 23:59:05 +0800 (SGT) From: Patrick Dung Subject: Strange write speed between /file and /tmp/file (may be related to 4k advanced format) To: freebsd hackers MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Patrick Dung List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2013 16:01:55 -0000 Hi ,=0A=0AI have asked the question in freebsd-questions but can't get repl= y so I tried to ask here first before submitting a bug report.=0A=0A=0AI ha= ve a FreeBSD 9.2 64 bit installed in a VM.=0AThe host is Windows 7 with vmw= are workstation, the host is using an advanced format (4k) harddisk.=0A=0A= =0AI have noticed there is strange write speed between /file and /tmp/file.= =0A/dev/da0p3 is UFS2 created by default installer.=0A=0AI am thinking if t= he problem is caused by disk alignment.=0Ada0p3 starting sector is 786594, = which seems not aligned by 4k (786594 *512/4096=3D98324.25)=0AThe dd result= is repeatable.=0A=0A=0A[root@testing2 /]# df -P=0AFilesystem=0A 512-blocks= =A0=A0=A0=A0=A0 Used=A0=A0=A0=A0 Avail Capacity=A0 Mounted=0A on=0A/dev/da0= p3=A0=A0=A0 6518712=A0=A0 4621000=A0=A0 1376216=A0=A0=A0 77%=A0=A0=A0 /=0Ad= evfs=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 2=A0=A0=A0=A0=A0=A0=A0=A0 2= =A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0 100%=A0=A0=A0 /dev=0Afdescfs=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0 2=A0=A0=A0=A0=A0=A0=A0=A0 2=A0=A0=A0=A0=A0=A0=A0= =A0 0=A0=A0 100%=A0=A0=A0 /dev/fd=0Aprocfs=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0 8=A0=A0=A0=A0=A0=A0=A0=A0 8=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0 100%=A0= =A0=A0 /proc=0Ahost:hgfs=A0=A0 591368064 405571456 185796608=A0=A0=A0 69%= =A0=A0=A0 /mnt-hgfs=0A[root@testing2 /]# gpart show=0A da0=0A=3D>=A0=A0=A0= =A0 34=A0 7548861=A0 da0=A0 GPT=A0 (3.6G)=0A=A0=A0=A0=A0=A0=A0 34=A0=A0=A0= =A0=A0 128=A0=A0=A0 1=A0 freebsd-boot=A0 (64k)=0A=A0=A0=A0=A0=A0 162=A0=A0 = 786432=A0=A0=A0 2=A0 freebsd-swap=A0 (384M)=0A=A0=A0 786594=A0 6760448=A0= =A0=A0 3=A0 freebsd-ufs=A0 (3.2G)=0A=A0 7547042=A0=A0=A0=A0 1853=A0=A0=A0= =A0=A0=A0 - free -=A0 (926k)=0A=0A[root@testing2 /]# dd if=3D/dev/zero of= =3D/tmp/file bs=3D50M count=3D4=0A4+0 records in=0A4+0 records out=0A209715= 200 bytes transferred in 10.303288 secs (20354202 bytes/sec)=0A[root@testin= g2 /]# rm /tmp/file=0A[root@testing2 /]# dd if=3D/dev/zero of=3D/tmp/file b= s=3D50M count=3D4=0A4+0 records in=0A4+0 records out=0A209715200 bytes tran= sferred in 9.789672 secs (21422087 bytes/sec)=0A[root@testing2 /]# rm /tmp/= file=0A[root@testing2 /]#=0A ~^C=0A[root@testing2 /]# dd if=3D/dev/zero of= =3D/file bs=3D50M count=3D4=0A4+0 records in=0A4+0 records out=0A209715200 = bytes transferred in 6.826694 secs (30719877 bytes/sec)=0A[root@testing2 /]= # rm /file=0A[root@testing2 /]# dd if=3D/dev/zero of=3D/file bs=3D50M count= =3D4=0A4+0 records in=0A4+0 records out=0A209715200 bytes transferred in 6.= 955417 secs (30151348 bytes/sec)=0A[root@testing2 /]# rm /file=0A=0AThanks = and regards,Patrick Dung