From owner-freebsd-isp@FreeBSD.ORG Mon Jul 6 08:05:11 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93394106564A for ; Mon, 6 Jul 2009 08:05:11 +0000 (UTC) (envelope-from tonix@interazioni.it) Received: from mx02.interazioni.net (mx02.interazioni.net [80.94.114.204]) by mx1.freebsd.org (Postfix) with ESMTP id DB91A8FC18 for ; Mon, 6 Jul 2009 08:05:10 +0000 (UTC) (envelope-from tonix@interazioni.it) Received: (qmail 8938 invoked by uid 88); 6 Jul 2009 08:05:06 -0000 Received: from unknown (HELO ?192.168.56.198?) (tonix@interazioni.it@85.18.206.139) by relay.interazioni.net with ESMTPA; 6 Jul 2009 08:05:06 -0000 Message-ID: <4A51B033.4070204@interazioni.it> Date: Mon, 06 Jul 2009 10:05:07 +0200 From: "Tonix (Antonio Nati)" User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) MIME-Version: 1.0 To: freebsd-isp@freebsd.org Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit Subject: CARP in standard kernel? X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jul 2009 08:05:11 -0000 Until now I used to compile my kernel, but I feel there should be no reasons, as of today, because most useable things are in standard kernel... except carp. Would be feasable to include carp in standard kernel? It would semplify a lot update/upgrade of standard FreeBSD. What do you think? Tonino -- ------------------------------------------------------------ Inter@zioni Interazioni di Antonio Nati http://www.interazioni.it tonix@interazioni.it ------------------------------------------------------------ From owner-freebsd-isp@FreeBSD.ORG Mon Jul 6 14:27:25 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 81A9F1065687 for ; Mon, 6 Jul 2009 14:27:25 +0000 (UTC) (envelope-from tonix@interazioni.it) Received: from mx02.interazioni.net (mx02.interazioni.net [80.94.114.204]) by mx1.freebsd.org (Postfix) with ESMTP id 1AAEE8FC25 for ; Mon, 6 Jul 2009 14:27:24 +0000 (UTC) (envelope-from tonix@interazioni.it) Received: (qmail 20415 invoked by uid 88); 6 Jul 2009 14:27:23 -0000 Received: from unknown (HELO ?192.168.56.198?) (tonix@interazioni.it@85.18.206.139) by relay.interazioni.net with ESMTPA; 6 Jul 2009 14:27:23 -0000 Message-ID: <4A5209CA.4030304@interazioni.it> Date: Mon, 06 Jul 2009 16:27:22 +0200 From: "Tonix (Antonio Nati)" User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) MIME-Version: 1.0 To: freebsd-isp@freebsd.org Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit Subject: ZFS in productions 64 bit X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jul 2009 14:27:25 -0000 Is anyone using in heavy production environment a ZFS FS with AMD 64 bit? Thanks, Tonino -- ------------------------------------------------------------ Inter@zioni Interazioni di Antonio Nati http://www.interazioni.it tonix@interazioni.it ------------------------------------------------------------ From owner-freebsd-isp@FreeBSD.ORG Mon Jul 6 21:24:33 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B90E51065689 for ; Mon, 6 Jul 2009 21:24:33 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from qw-out-2122.google.com (qw-out-2122.google.com [74.125.92.25]) by mx1.freebsd.org (Postfix) with ESMTP id 6F5FE8FC26 for ; Mon, 6 Jul 2009 21:24:33 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by qw-out-2122.google.com with SMTP id 5so1640225qwd.7 for ; Mon, 06 Jul 2009 14:24:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=S8iLc7EvF6I5yzH1GdRI2VkZrt+1KL8LFOEtJTBPKQM=; b=MGK+q4b1IbYU6EW6bbxRzBK+Y/vJAadHnO7cDVCCY1XRdZHw8NVLRVZJnIxHSAuJIu xDoM43um2va9PzHJGUlFLJ/SXVKl14r3AA3Uv9BRXRhllgqNA1+68I9sHtkYGVuElyTi 9+JGvFEP6bPeZq+XJOjQpVvspyJvZeNXaW4ZQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=TljGwBo5FKHa14Z2rj5BxequWzi69fuE9YwKpgjYJ2Ll3jsDPB4AWHRv4zhMYNz2LZ klGNR1kX074s8XHIyEUO8tI9woxxSrW4CsaicY6ds64jpL/ffrqbVYNPdDeqj0LtTG1r a7VX4Uuq6UQHAZYFp01Ny0AfMAO8iSz5C7o0E= MIME-Version: 1.0 Received: by 10.229.86.194 with SMTP id t2mr2733404qcl.49.1246913530161; Mon, 06 Jul 2009 13:52:10 -0700 (PDT) In-Reply-To: <4A5209CA.4030304@interazioni.it> References: <4A5209CA.4030304@interazioni.it> Date: Mon, 6 Jul 2009 13:52:10 -0700 Message-ID: From: Freddie Cash To: "Tonix (Antonio Nati)" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-isp@freebsd.org Subject: Re: ZFS in productions 64 bit X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jul 2009 21:24:34 -0000 On Mon, Jul 6, 2009 at 7:27 AM, Tonix (Antonio Nati) wrote: > Is anyone using in heavy production environment a ZFS FS with AMD 64 bit? > We're using FreeBSD 7.2 on our backup servers. The primary backup server does remote backups for over 105 servers, every night. And then pushes the changes to the secondary backup server, every day. Both servers are: 5U Chenbro case, with 24 hot-swappable SATA drive bays 1350 watt, 4-way redundant PSU (yes, it's overkill) Tyan h2000M motherboard 2x AMD Opteron 2220 CPUs @ 2.8 GHz (dual-core) 3Ware 9550SXU-12ML PCI-X RAID controller 3Ware 9650SE-12ML PCIe RAID controller Intel PRO/1000MT PCI-X quad-port gigabit NIC 24x 500 GB SATA harddrives 2x CompactFlash drives in CF-to-IDE or CF-to-SATA adapters The CompactFlash are configured using gmirror, and hold / and /usr. (/usr is there as we originally had some issues booting into single-user mode and getting the zpool up and running.) zpool is configured with 3x raidz2 vdevs, each vdev uses 8 harddrives. Gives us ~ 10 TB usable space in the pool. Everything other than / and /usr are ZFS, including /usr/src, /usr/obj, /usr/ports, /var, /tmp, /usr/local, /home, and so on. Over the course of a backup run, we average 80 MBytes/sec writes, which is limited by the horrible upload performance of the remote ADSL sites. We've benchmarked the system maxing out at 550 MBytes/sec write and 5.5 GBytes/sec read. We had to do a lot of manual tuning when we started out, to limit vm.kmem_size_max and vfs.zfs.arc_max, and to disable prefetch (vfs.zfs.prefetch_disable=1), as we started with 7-STABLE shortly after 7.0 was released. With FreeBSD 7.2, we've removed the tuning, but left prefetch disabled (with prefetch enabled, we'd lockup the system after about 5 hours of heavy rsync usage ... no swap space left). Our backups are done using rsync. We serialise the backups of the systems at each remote site, but run the backups for multiple sites in parallel. The only "non-standard" change we made was to switch to openssh-portable from ports, and enable the HPN patches. We saw our rsync throughput go up by 30% after tuning the network sysctls, using HPN. Other than trying to use USB sticks instead of CompactFlash originally, during the initial tuning phase, and when experimenting with prefetch, the system has been rock solid. Our next big ZFS project will be using similar hardware to create our own SAN setup, using iSCSI exports, for a virtualisation setup (Linux+KVM on the processing nodes, FreeBSD+ZFS on the storage nodes). -- Freddie Cash fjwcash@gmail.com From owner-freebsd-isp@FreeBSD.ORG Tue Jul 7 09:57:57 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0ED071065670 for ; Tue, 7 Jul 2009 09:57:57 +0000 (UTC) (envelope-from cardil.xavier@gmail.com) Received: from mail-fx0-f218.google.com (mail-fx0-f218.google.com [209.85.220.218]) by mx1.freebsd.org (Postfix) with ESMTP id 95A428FC0C for ; Tue, 7 Jul 2009 09:57:56 +0000 (UTC) (envelope-from cardil.xavier@gmail.com) Received: by fxm18 with SMTP id 18so4041391fxm.43 for ; Tue, 07 Jul 2009 02:57:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type:content-transfer-encoding; bh=z7UGuzRkgLu3Q/lsENRzDton+8PC00HfVVODhqpmgcc=; b=ZjOgzKv4KlDPlzwCILaNf0PWe3e0J28ExJ5AIfLtLUQT2rZ8asSipzfXYuvkBH5qEN HQb1FaVi/G62cVxUdOn/aosYQfdhSdWeoxQhb+BkKY6sr/Na6QDb7T8bxe6klWEtHxao x97uQjV8CgqQXdI/cdD8MdtHPQZnKtxJm08CU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; b=NWPkczRAQcsD7Q2W6bOwo35nw7Gw2AfxofNlY5ScXgxtnYbklvCGDtSkLeblFv/lTN OBJ33kmRurlgRHyJi/SV3UvN93FsC4gJf0iyZgWw+kokU9SFYxUngsSpyWsDd451FOzW KZKl4iOb+0qOSgO71Tac4v43POJO9fyv6wr1c= MIME-Version: 1.0 Received: by 10.103.172.7 with SMTP id z7mr3261034muo.87.1246959174797; Tue, 07 Jul 2009 02:32:54 -0700 (PDT) Date: Tue, 7 Jul 2009 11:32:54 +0200 Message-ID: <50c2d0150907070232g7642cec9j3a24511dda5f81c3@mail.gmail.com> From: Xavier Cardil To: freebsd-isp@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: mail treatment X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jul 2009 09:57:57 -0000 Hi all, another question: When I check some user inbox mail, the mail is stored as only 1 file. That's good because you don't get lots of files, one file per e-mail. But when it comes to mail treatment, I find it is harder to manage. I'm trying to find the way to get a message every time it come in, analyse the subject and the attachments and with that two results do something, like create another file to send it to a fax send application. Is there a way to cut the file by mail, or to receive the messages on different files ? Thank you. From owner-freebsd-isp@FreeBSD.ORG Tue Jul 7 10:19:53 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8105E106564A for ; Tue, 7 Jul 2009 10:19:53 +0000 (UTC) (envelope-from rblayzor.bulk@inoc.net) Received: from mail1.albyny.inoc.net (mail1.albyny.inoc.net [64.22.32.71]) by mx1.freebsd.org (Postfix) with ESMTP id 1182F8FC08 for ; Tue, 7 Jul 2009 10:19:52 +0000 (UTC) (envelope-from rblayzor.bulk@inoc.net) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=inoc.net; h=Received:From:To:Subject:Date; b=MYN5+XdKm5JjWhs1J0LweKow+L9s/6r79ektagg7aSrhsiWpFKZ/872S/y2q5ZZUAmpoxDVBHajbHKsw+IT7dICU8EV64yQ1DTtKf/JQ7a+ZfSaxEXtOIe+Zd9bdMnkIuQmeVuYR2/j05viBFGAi8/fhjBeKN5lI9SdZMHAqxpc=; X-Default-Received-SPF: pass (skip=loggedin (res=PASS)) x-ip-name=cpe-67-240-119-200.nycap.res.rr.com; Received: from [172.16.0.5] (cpe-67-240-119-200.nycap.res.rr.com [67.240.119.200]) by mail1.albyny.inoc.net (build v9.5.29) with ESMTP id 1863889-1941382 for multiple; Tue, 07 Jul 2009 10:03:42 +0000 Message-Id: <1289A8B6-0414-4F14-B9C0-30E43E454ED9@inoc.net> From: Robert Blayzor To: Xavier Cardil In-Reply-To: <50c2d0150907070232g7642cec9j3a24511dda5f81c3@mail.gmail.com> Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v935.3) Date: Tue, 7 Jul 2009 06:03:40 -0400 References: <50c2d0150907070232g7642cec9j3a24511dda5f81c3@mail.gmail.com> X-Mailer: Apple Mail (2.935.3) Cc: freebsd-isp@freebsd.org Subject: Re: mail treatment X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jul 2009 10:19:53 -0000 On Jul 7, 2009, at 5:32 AM, Xavier Cardil wrote: > I'm trying to find the way to get a message every time it come in, > analyse the subject and the attachments and with that two results do > something, procmail ? -- Robert Blayzor, BOFH INOC, LLC rblayzor@inoc.net http://www.inoc.net/~rblayzor/ From owner-freebsd-isp@FreeBSD.ORG Tue Jul 7 10:40:18 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97D59106566C for ; Tue, 7 Jul 2009 10:40:18 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from flat.berklix.org (flat.berklix.org [83.236.223.115]) by mx1.freebsd.org (Postfix) with ESMTP id 717078FC08 for ; Tue, 7 Jul 2009 10:40:16 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from park.js.berklix.net (p549A7462.dip.t-dialin.net [84.154.116.98]) (authenticated bits=0) by flat.berklix.org (8.13.8/8.13.8) with ESMTP id n67AGvtB021141; Tue, 7 Jul 2009 12:17:01 +0200 (CEST) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id n67AGlBw016255; Tue, 7 Jul 2009 12:16:47 +0200 (CEST) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id n67AH1kc050687; Tue, 7 Jul 2009 12:17:06 +0200 (CEST) (envelope-from jhs@fire.js.berklix.net) Message-Id: <200907071017.n67AH1kc050687@fire.js.berklix.net> To: Xavier Cardil From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Tue, 07 Jul 2009 11:32:54 +0200." <50c2d0150907070232g7642cec9j3a24511dda5f81c3@mail.gmail.com> Date: Tue, 07 Jul 2009 12:17:01 +0200 Sender: jhs@berklix.com Cc: freebsd-isp@freebsd.org Subject: Re: mail treatment X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jul 2009 10:40:18 -0000 Hi, Reference: > From: Xavier Cardil > Date: Tue, 7 Jul 2009 11:32:54 +0200 > Message-id: <50c2d0150907070232g7642cec9j3a24511dda5f81c3@mail.gmail.com> Xavier Cardil wrote: > Hi all, another question: > > When I check some user inbox mail, the mail is stored as only 1 file. > That's good because you don't get lots of files, one file per e-mail. > But when it comes to mail treatment, I find it is harder to manage. > I'm trying to find the way to get a message every time it come in, > analyse the subject and the attachments and with that two results do > something, like create another file to send it to a fax send > application. Is there a way to cut the file by mail, or to receive the > messages on different files ? > > Thank you. You are asking on wrong list. Better ask on eg hackers@ or questions@ next time. See list remit on http://freebsd.org/ However, Answer/ what I use: /usr/ports/mail/procmail Which is nmh compatible /usr/ports/mail/nmh Which is compatible with most MUA Mail User Agents. Procmail has lots of docu. & some example syntax, but here's my personal filter syntax for procmail http://www.berklix.com/~jhs/dots/.procmailrc http://www.berklix.com/~jhs/dots/.procmailrc.lists # for this list Cheers, Julian -- Julian Stacey: BSD Unix Linux C Sys. Eng. Consultant Munich http://berklix.com Mail in plain ASCII text, HTML & Base64 are spam. http://asciiribbon.org From owner-freebsd-isp@FreeBSD.ORG Tue Jul 7 12:46:56 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5B6E106564A for ; Tue, 7 Jul 2009 12:46:56 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: from smtp.ibctech.ca (v6.ibctech.ca [IPv6:2607:f118::b6]) by mx1.freebsd.org (Postfix) with SMTP id 6C0898FC15 for ; Tue, 7 Jul 2009 12:46:56 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: (qmail 79376 invoked by uid 89); 7 Jul 2009 12:48:23 -0000 Received: from unknown (HELO ?IPv6:2607:f118::5?) (steve@ibctech.ca@2607:f118::5) by 2607:f118::b6 with ESMTPA; 7 Jul 2009 12:48:23 -0000 Message-ID: <4A5343B6.2090608@ibctech.ca> Date: Tue, 07 Jul 2009 08:46:46 -0400 From: Steve Bertrand User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: Freddie Cash References: <4A5209CA.4030304@interazioni.it> In-Reply-To: X-Enigmail-Version: 0.95.7 Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms010503080506020200090305" Cc: freebsd-isp@freebsd.org, "Tonix \(Antonio Nati\)" Subject: Re: ZFS in productions 64 bit X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jul 2009 12:46:57 -0000 This is a cryptographically signed message in MIME format. --------------ms010503080506020200090305 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Freddie Cash wrote: > On Mon, Jul 6, 2009 at 7:27 AM, Tonix (Antonio Nati) > wrote: > >> Is anyone using in heavy production environment a ZFS FS with AMD 64 bit? [...snip...] > We've > benchmarked the system maxing out at 550 MBytes/sec write and 5.5 GBytes/sec > read. Holy crap! Would you be so kind as to share some of your relevant pieces of config on how you have the OS and ZFS set up? > Our backups are done using rsync. We serialise the backups of the systems > at each remote site, but run the backups for multiple sites in parallel. Although not nearly as impressive (nor 64-bit), I've been using ZFS on my AMANDA backup server since 7.0, and it too (after a bit of tweaking) has been rock solid. > Our next big ZFS project will be using similar hardware to create our own > SAN setup, using iSCSI exports, for a virtualisation setup (Linux+KVM on the > processing nodes, FreeBSD+ZFS on the storage nodes). Please do make notes ;) Steve --------------ms010503080506020200090305 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIII/zCC AtowggJDoAMCAQICEEs5xg/J3t77QWJ4SatV1HcwDQYJKoZIhvcNAQEFBQAwYjELMAkGA1UE BhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMT I1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBMB4XDTA5MDUwNzIzMTYxMFoX DTEwMDUwNzIzMTYxMFowQjEfMB0GA1UEAxMWVGhhd3RlIEZyZWVtYWlsIE1lbWJlcjEfMB0G CSqGSIb3DQEJARYQc3RldmVAaWJjdGVjaC5jYTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC AQoCggEBAJSTRAjP1RVa87/mnZn+PBTbENgyhhBJ4rWApmaNcthzRdk2DB/49KrXx3EQP60w Lj4KU0DFkiGNVj9BnVxRAx/WDXKxGC3uGGEG6gjyWv8KFMWMsH9mL7y7uNow1HueT6pZUf9o yY8Ewd+01QpGi7FfXOae7lGHhbEwnEJGwz08ytRfLmH0KtEzlZanZZhwDGX5s1kIHnyxdACh 3byXY6Z2bOrx0rcrQHCnHJppxddR60F7igjaMuBFstE51h9XTgXDNKJbglqTug5ghGihNuP6 VsBN7ue62y96UGIE22TvKEcAQ665vQGjHqZeSzZYy+hWNOa27pWFmhlqFjx0x8MCAwEAAaMt MCswGwYDVR0RBBQwEoEQc3RldmVAaWJjdGVjaC5jYTAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3 DQEBBQUAA4GBAMOmjxjp2Xzk6ZHLwTgFDzVhm98RjRT3UXotKjNIR7SgwfWF5wkJrx4I+dXu ui5ztMEq4bTTRgJ344MqE6uZiZlg+tBIFHZGCJfKdzsX4QuV2jmw0sR5dMaYxG6tlDB0YUMv gTqzV7ZDpiusTMOZe9pP1PdxFhOcIJXtMQDj5LhuMIIC2jCCAkOgAwIBAgIQSznGD8ne3vtB YnhJq1XUdzANBgkqhkiG9w0BAQUFADBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3Rl IENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVt YWlsIElzc3VpbmcgQ0EwHhcNMDkwNTA3MjMxNjEwWhcNMTAwNTA3MjMxNjEwWjBCMR8wHQYD VQQDExZUaGF3dGUgRnJlZW1haWwgTWVtYmVyMR8wHQYJKoZIhvcNAQkBFhBzdGV2ZUBpYmN0 ZWNoLmNhMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAlJNECM/VFVrzv+admf48 FNsQ2DKGEEnitYCmZo1y2HNF2TYMH/j0qtfHcRA/rTAuPgpTQMWSIY1WP0GdXFEDH9YNcrEY Le4YYQbqCPJa/woUxYywf2YvvLu42jDUe55PqllR/2jJjwTB37TVCkaLsV9c5p7uUYeFsTCc QkbDPTzK1F8uYfQq0TOVlqdlmHAMZfmzWQgefLF0AKHdvJdjpnZs6vHStytAcKccmmnF11Hr QXuKCNoy4EWy0TnWH1dOBcM0oluCWpO6DmCEaKE24/pWwE3u57rbL3pQYgTbZO8oRwBDrrm9 AaMepl5LNljL6FY05rbulYWaGWoWPHTHwwIDAQABoy0wKzAbBgNVHREEFDASgRBzdGV2ZUBp YmN0ZWNoLmNhMAwGA1UdEwEB/wQCMAAwDQYJKoZIhvcNAQEFBQADgYEAw6aPGOnZfOTpkcvB OAUPNWGb3xGNFPdRei0qM0hHtKDB9YXnCQmvHgj51e66LnO0wSrhtNNGAnfjgyoTq5mJmWD6 0EgUdkYIl8p3OxfhC5XaObDSxHl0xpjEbq2UMHRhQy+BOrNXtkOmK6xMw5l72k/U93EWE5wg le0xAOPkuG4wggM/MIICqKADAgECAgENMA0GCSqGSIb3DQEBBQUAMIHRMQswCQYDVQQGEwJa QTEVMBMGA1UECBMMV2VzdGVybiBDYXBlMRIwEAYDVQQHEwlDYXBlIFRvd24xGjAYBgNVBAoT EVRoYXd0ZSBDb25zdWx0aW5nMSgwJgYDVQQLEx9DZXJ0aWZpY2F0aW9uIFNlcnZpY2VzIERp dmlzaW9uMSQwIgYDVQQDExtUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgQ0ExKzApBgkqhkiG 9w0BCQEWHHBlcnNvbmFsLWZyZWVtYWlsQHRoYXd0ZS5jb20wHhcNMDMwNzE3MDAwMDAwWhcN MTMwNzE2MjM1OTU5WjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRp bmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3Vp bmcgQ0EwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMSmPFVzVftOucqZWh5owHUEcJ3f 6f+jHuy9zfVb8hp2vX8MOmHyv1HOAdTlUAow1wJjWiyJFXCO3cnwK4Vaqj9xVsuvPAsH5/Ef kTYkKhPPK9Xzgnc9A74r/rsYPge/QIACZNenprufZdHFKlSFD0gEf6e20TxhBEAeZBlyYLf7 AgMBAAGjgZQwgZEwEgYDVR0TAQH/BAgwBgEB/wIBADBDBgNVHR8EPDA6MDigNqA0hjJodHRw Oi8vY3JsLnRoYXd0ZS5jb20vVGhhd3RlUGVyc29uYWxGcmVlbWFpbENBLmNybDALBgNVHQ8E BAMCAQYwKQYDVR0RBCIwIKQeMBwxGjAYBgNVBAMTEVByaXZhdGVMYWJlbDItMTM4MA0GCSqG SIb3DQEBBQUAA4GBAEiM0VCD6gsuzA2jZqxnD3+vrL7CF6FDlpSdf0whuPg2H6otnzYvwPQc UCCTcDz9reFhYsPZOhl+hLGZGwDFGguCdJ4lUJRix9sncVcljd2pnDmOjCBPZV+V2vf3h9bG CE6u9uo05RAaWzVNd+NWIXiC3CEZNd4ksdMdRv9dX2VPMYIDZDCCA2ACAQEwdjBiMQswCQYD VQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UE AxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECEEs5xg/J3t77QWJ4SatV 1HcwCQYFKw4DAhoFAKCCAcMwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0B CQUxDxcNMDkwNzA3MTI0NjQ2WjAjBgkqhkiG9w0BCQQxFgQU17ok+ObgPZfga7wd6c1Gddg4 koEwUgYJKoZIhvcNAQkPMUUwQzAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZI hvcNAwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwgYUGCSsGAQQBgjcQBDF4MHYwYjEL MAkGA1UEBhMCWkExJTAjBgNVBAoTHFRoYXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAq BgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBGcmVlbWFpbCBJc3N1aW5nIENBAhBLOcYPyd7e+0Fi eEmrVdR3MIGHBgsqhkiG9w0BCRACCzF4oHYwYjELMAkGA1UEBhMCWkExJTAjBgNVBAoTHFRo YXd0ZSBDb25zdWx0aW5nIChQdHkpIEx0ZC4xLDAqBgNVBAMTI1RoYXd0ZSBQZXJzb25hbCBG cmVlbWFpbCBJc3N1aW5nIENBAhBLOcYPyd7e+0FieEmrVdR3MA0GCSqGSIb3DQEBAQUABIIB ADsw38RowTrMDQ+sxaNtwmAWPm1n8Lv5JIJ2r/Oo2PeBS5CdzDEdMy+Ox/F9nt+lrk3ExDvn Az0TEhnAS2BDTsFocwXJ9/3049aYq1XcHE+PZmMnAz3HPxlFDogcD+5uvLHgdqSPWfVV9/qD TArJPKxelA9wFi+BtXW4+NdOsZIx/h560apMognhHtEp0sVr3tobX3icVLJIRMAkWAntGkd0 41l2iya0Fkqg7U5rZb/ACuq+P6honkscVaoYQY7D2YudEjVJYUJDwkBEWOAB2WRO3I4QUirT 3zglComx81bnfeQ/Z5+g+vVZEl5WHZsuCzdiyBQClq53o/bkGq8LLyIAAAAAAAA= --------------ms010503080506020200090305-- From owner-freebsd-isp@FreeBSD.ORG Tue Jul 7 15:51:07 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0341610657C4 for ; Tue, 7 Jul 2009 15:51:07 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-pz0-f193.google.com (mail-pz0-f193.google.com [209.85.222.193]) by mx1.freebsd.org (Postfix) with ESMTP id CB5AC8FC2D for ; Tue, 7 Jul 2009 15:51:06 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by pzk31 with SMTP id 31so2861029pzk.3 for ; Tue, 07 Jul 2009 08:51:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=wU2Dxgij2xDwXv9cDa4rufF13mjhVmgQB60Rqqozmuk=; b=TZQZ9c8q99M2dGNmU2lwljuNOQK2MGE2Aom0OxzCPqh8sv0VMIYHMbJdRd1zdotkq0 YKWd4YcTL/Zd/9YI3a6nCBA/npYy1tb5320rvOYwvC2u1lhgkgqrVGpYZoLbZNzxO0wT G62qhvulS1US52hE0iuf1MTcywIp29sbgYLFw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=kwx917F2Z+HFM4sktMhTSyV+D8vZ70Jw0voRrF4xOlWqjr6jZA9y8jAjP3hgjcH0A1 50SesNOmtc15tl6iJU/usrEFuNhOmqbgIQKk7BIk3WMKe6h/Qv1kBa/kwYiVMpYH4g2N K9eU8PF4b19r6Fo7MtYx1kkuxH2ZVJy8uRjUM= MIME-Version: 1.0 Received: by 10.141.41.12 with SMTP id t12mr2517266rvj.53.1246981866216; Tue, 07 Jul 2009 08:51:06 -0700 (PDT) In-Reply-To: <4A5343B6.2090608@ibctech.ca> References: <4A5209CA.4030304@interazioni.it> <4A5343B6.2090608@ibctech.ca> Date: Tue, 7 Jul 2009 08:51:06 -0700 Message-ID: From: Freddie Cash To: freebsd-isp@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: ZFS in productions 64 bit X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jul 2009 15:51:07 -0000 On Tue, Jul 7, 2009 at 5:46 AM, Steve Bertrand wrote: > Freddie Cash wrote: > > On Mon, Jul 6, 2009 at 7:27 AM, Tonix (Antonio Nati) > > wrote: > > > >> Is anyone using in heavy production environment a ZFS FS with AMD 64 > bit? > > [...snip...] > > > We've > > benchmarked the system maxing out at 550 MBytes/sec write and 5.5 > GBytes/sec > > read. > > Holy crap! > > Would you be so kind as to share some of your relevant pieces of config > on how you have the OS and ZFS set up? > http://forums.freebsd.org/showthread.php?t=3689 All the gory details of how we do our backups for over 100 remote servers every night, in under 5 hours, with daily snapshots going back just shy of 6 months (average size of the daily snapshots is 10 GB). If we had 10 Mbps symmetrical connections to our remote sites instead of 2.0/0.5 Mbps ADSL, we could probably drop the backup time to an hour. :) If there's anything missing from there that you would like to know, just ask. :) -- Freddie Cash fjwcash@gmail.com From owner-freebsd-isp@FreeBSD.ORG Tue Jul 7 15:54:03 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 53A651065674 for ; Tue, 7 Jul 2009 15:54:03 +0000 (UTC) (envelope-from ericx@ericx.net) Received: from QMTA06.emeryville.ca.mail.comcast.net (qmta06.emeryville.ca.mail.comcast.net [76.96.30.56]) by mx1.freebsd.org (Postfix) with ESMTP id 2C97D8FC0A for ; Tue, 7 Jul 2009 15:54:03 +0000 (UTC) (envelope-from ericx@ericx.net) Received: from OMTA12.emeryville.ca.mail.comcast.net ([76.96.30.44]) by QMTA06.emeryville.ca.mail.comcast.net with comcast id D2W21c0010x6nqcA63gu0E; Tue, 07 Jul 2009 15:40:54 +0000 Received: from smtp.ericx.net ([76.24.209.147]) by OMTA12.emeryville.ca.mail.comcast.net with comcast id D3gs1c00G3BMG6c8Y3gtes; Tue, 07 Jul 2009 15:40:54 +0000 Received: from smtp.ericx.net (localhost.ericx.net [127.0.0.1]) by smtp.ericx.net (Postfix) with ESMTP id B5923129F7D8 for ; Tue, 7 Jul 2009 15:42:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=ericx.net; h=message-id :date:from:mime-version:to:subject:content-type :content-transfer-encoding; s=selector1; bh=6PmwnrQ+lZUuP1AjtXt0 AKyKxUU=; b=n7bwA8umINBfusi+1pfU9TqwLb9seN7ZkZO+Ls3A0Xz6ukx4Sssq 9+lH6fBMNkcwMBUIT7/ZpEgnFkYczcgzIMge85WjpJ94MyIFmahOV/L5znj4NiRW GCoNG9Silnu6v0HJ6dAayQ/LrC/N438ZRXtxPXZZWSVwPh9rbMQ9nnA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=ericx.net; h=message-id:date :from:mime-version:to:subject:content-type :content-transfer-encoding; q=dns; s=selector1; b=Ov2kjp+koWsvHF Z/gamnBXKRfZBMu3Sw+NNvl2w0WyhnfAmX2iBUFedjVMubkp5TTBLu5naWzJJeRx x3FZCz7bfzrpTWVr/2avW/usgT8MMA2paBnimyXlP139RsZmkK98kYuTner/fkYO u9jnecj7NrOHC0fkKdMlDbb4KpM0Q= Received: from cheesenip.vineyard.net (static-141-149-190-252.bos.east.verizon.net [141.149.190.252]) by smtp.ericx.net (Postfix) with ESMTPSA id 54504129EEAA for ; Tue, 7 Jul 2009 15:42:18 +0000 (UTC) Message-ID: <4A536C7B.7030902@ericx.net> Date: Tue, 07 Jul 2009 11:40:43 -0400 From: "Eric W. Bates" Organization: Vineyard.NET, Inc. User-Agent: Thunderbird 2.0.0.16 (X11/20080814) MIME-Version: 1.0 To: freebsd-isp@freebsd.org X-Enigmail-Version: 0.95.6 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: zfs in production X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jul 2009 15:54:03 -0000 Sorry to ask an obvious question... But what's the advantage to zfs? From owner-freebsd-isp@FreeBSD.ORG Tue Jul 7 16:10:17 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 495E6106566C for ; Tue, 7 Jul 2009 16:10:17 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.224]) by mx1.freebsd.org (Postfix) with ESMTP id 1E8088FC25 for ; Tue, 7 Jul 2009 16:10:16 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by rv-out-0506.google.com with SMTP id f9so1623373rvb.43 for ; Tue, 07 Jul 2009 09:10:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=uG7GWuM2Rowyb/0SfFafSmtpKdfAFYTywKdb8uFpymA=; b=K7igJVGiOlBOKZSW8fiO7WX8gA80DFFBYWT2j/CDAUPqPNjxwlWp6lmGW9okB95Bxh iL9PR9BeU77QK/rsRDuO529Dgqj3/XyjBa+VbgT6ZzMa/J7FgiDmTnLsKa087L62oxx+ HAit8VGbOJeONWnqQRTQoIHU0cFM5Xby8nP5E= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=mG9nQWBH63Ow3nAvKhwt591PfDgOG9IqeIjdn2gq9gkWrRLDEr/lMJkxsMVjwG/HwY bWNFYEYnT1KUN1PuB1rNtUyCppiQbnsbU8t0HWY3yR5uu1j6CimZcFuchb0Beo+3zF8j q42dCEcfu+xUjMtlIvuSGEdA9fKHJvctJ38RI= MIME-Version: 1.0 Received: by 10.141.4.20 with SMTP id g20mr3718362rvi.121.1246983016579; Tue, 07 Jul 2009 09:10:16 -0700 (PDT) In-Reply-To: <4A536C7B.7030902@ericx.net> References: <4A536C7B.7030902@ericx.net> Date: Tue, 7 Jul 2009 09:10:16 -0700 Message-ID: From: Freddie Cash To: freebsd-isp@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: zfs in production X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jul 2009 16:10:17 -0000 On Tue, Jul 7, 2009 at 8:40 AM, Eric W. Bates wrote: > Sorry to ask an obvious question... But what's the advantage to zfs? > There's plenty of information regarding ZFS available online. :) A very quick synopsis of the main features would be: - pooled storage setup, which can be easily expanded as needed, and is available to all filesystems/volumes - built-in striping (RAID0), n-way mirroring (RAID1), raidz1 (modified RAID5), raidz2 (modified RAID6) - copy-on-write, and transactional filesystem - posix-compliant filesystem layer, and block storage / volume manager - infinite, constant-time, in-filesystem snapshots (read-only) - snapshots can be promoted to clones (read/write) - clones can be promoted to separate filesystems - built-in compression - end-to-end data integrity using checksums There's a whole bunch more to it than the above. Once you start using pooled storage for servers, it's really hard to go back to the old way of slicing and partitioning and allocating specific amounts of storage to each filesytem. :) -- Freddie Cash fjwcash@gmail.com From owner-freebsd-isp@FreeBSD.ORG Tue Jul 7 16:26:57 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14C781065678 for ; Tue, 7 Jul 2009 16:26:57 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-px0-f172.google.com (mail-px0-f172.google.com [209.85.216.172]) by mx1.freebsd.org (Postfix) with ESMTP id DC9408FC0A for ; Tue, 7 Jul 2009 16:26:56 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by pxi2 with SMTP id 2so699877pxi.3 for ; Tue, 07 Jul 2009 09:26:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=efdFNoaDQXzkvp2DNhREzbCO21ES/OMHKJ2nLY4SYXE=; b=LutSaXs3zSOpTLuWriysSMDwuNe7h/Tg8J51xIzuYgXvStjrRa3BZR/3C+wjZh5DVe jOh04bOue9KgV7IcsETi4IgdTwpTfD+H6n6Rzmd0suQ7sD+HMphXuLf9ETQ6c6bqBJ2o PIn/+VH5UswgV9yCe6WvNdcKQsmjTxMsMC1qQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=dHXM6FmVEsKXH3zL43F27LRL9v6pY8k0z5oD+aYPu5t06SjBewFTJKxZEXWsb4mLvY 9Dflt4tDsgADYWl2bbiPJA20U8wEzQM/bSnnl185PXahcYWhwFzm6FwaY5BQdkzqblCd ryDDgkXdvPNplWpgR23XBxiluN7ihx69E9V7s= MIME-Version: 1.0 Received: by 10.140.164.1 with SMTP id m1mr3735993rve.77.1246984016339; Tue, 07 Jul 2009 09:26:56 -0700 (PDT) In-Reply-To: <4A5376D8.5080402@interazioni.it> References: <4A5209CA.4030304@interazioni.it> <4A5376D8.5080402@interazioni.it> Date: Tue, 7 Jul 2009 09:26:56 -0700 Message-ID: From: Freddie Cash To: freebsd-isp@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: ZFS in productions 64 bit X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jul 2009 16:26:57 -0000 On Tue, Jul 7, 2009 at 9:24 AM, Tonix (Antonio Nati) wrote: > Freddie Cash ha scritto: > > On Mon, Jul 6, 2009 at 7:27 AM, Tonix (Antonio Nati) wrote: > > > > Is anyone using in heavy production environment a ZFS FS with AMD 64 bit? > > > > We're using FreeBSD 7.2 on our backup servers. The primary backup server > does remote backups for over 105 servers, every night. And then pushes the > changes to the secondary backup server, every day. > > > > > Are you using ZFS only on backup servers, or also on remote servers to make > a snaphost of data to be backed up? > Only on the backup servers. The remote servers are running either Debian Linux 4.0, FreeBSD 6.x/7.x, or RHEL 4.x. And we do a couple of manual backups of Windows XP stations using rsync for Windows. -- Freddie Cash fjwcash@gmail.com From owner-freebsd-isp@FreeBSD.ORG Tue Jul 7 16:33:20 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 59EC4106564A for ; Tue, 7 Jul 2009 16:33:20 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from ey-out-2122.google.com (ey-out-2122.google.com [74.125.78.26]) by mx1.freebsd.org (Postfix) with ESMTP id D8D888FC08 for ; Tue, 7 Jul 2009 16:33:19 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: by ey-out-2122.google.com with SMTP id 9so1293308eyd.3 for ; Tue, 07 Jul 2009 09:33:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:subject:from:to:cc :in-reply-to:references:content-type:date:message-id:mime-version :x-mailer:content-transfer-encoding; bh=dij1QYT2rpqxCzlsjm+qpQ6N6r4kz4RAg7+m2jy3jb0=; b=D4MLFOTufGanWdD1NNdSLTPlUqXi9BnAY8LfyUX84EcJum5Htug0QbY7yjnSY2Vodn dvKvPjZneGwubk3Pf/fam6LZI8fMjxlfPwwPYpLxn6ClcHvm3UE1MVWNAzbTaDNiAaCo 4kTGJo4cGcqAs3JFBXBR5Q78WLON6Yclc1lIg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=subject:from:to:cc:in-reply-to:references:content-type:date :message-id:mime-version:x-mailer:content-transfer-encoding; b=ZRJjUjg+K3jJLxpSUI2+DcYTm4eRuRSKEb4ICU5mp3GQ4B/qp1KI11OOWJjMLNH6UH +vTi7uHVcPxLqeFNJ1cHSQ/WB6TRdaf1KlM/cow1XjiermohzmUdEaXPev6lW0TQivqs RSEljIrfjVnClUvApjvB3y5JTzZkoSl/mFPko= Received: by 10.210.16.11 with SMTP id 11mr5242590ebp.37.1246983148351; Tue, 07 Jul 2009 09:12:28 -0700 (PDT) Received: from ?127.0.0.1? (87-194-39-182.bethere.co.uk [87.194.39.182]) by mx.google.com with ESMTPS id 23sm3782322eya.9.2009.07.07.09.12.27 (version=SSLv3 cipher=RC4-MD5); Tue, 07 Jul 2009 09:12:28 -0700 (PDT) From: Tom Evans To: "Eric W. Bates" In-Reply-To: <4A536C7B.7030902@ericx.net> References: <4A536C7B.7030902@ericx.net> Content-Type: text/plain Date: Tue, 07 Jul 2009 17:12:26 +0100 Message-Id: <1246983146.2437.299.camel@strangepork.london.mintel.ad> Mime-Version: 1.0 X-Mailer: Evolution 2.26.1.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-isp@freebsd.org Subject: Re: zfs in production X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jul 2009 16:33:20 -0000 On Tue, 2009-07-07 at 11:40 -0400, Eric W. Bates wrote: > Sorry to ask an obvious question... But what's the advantage to zfs? http://www.google.com/search?q=advantages+of+zfs From owner-freebsd-isp@FreeBSD.ORG Tue Jul 7 16:49:49 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 395CA1065670 for ; Tue, 7 Jul 2009 16:49:49 +0000 (UTC) (envelope-from tonix@interazioni.it) Received: from mx02.interazioni.net (mx02.interazioni.net [80.94.114.204]) by mx1.freebsd.org (Postfix) with ESMTP id A49E38FC16 for ; Tue, 7 Jul 2009 16:49:48 +0000 (UTC) (envelope-from tonix@interazioni.it) Received: (qmail 73145 invoked by uid 88); 7 Jul 2009 16:49:47 -0000 Received: from unknown (HELO ?192.168.56.198?) (tonix@interazioni.it@85.18.206.139) by relay.interazioni.net with ESMTPA; 7 Jul 2009 16:49:47 -0000 Message-ID: <4A537CAC.7090608@interazioni.it> Date: Tue, 07 Jul 2009 18:49:48 +0200 From: "Tonix (Antonio Nati)" User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) MIME-Version: 1.0 To: freebsd-isp@freebsd.org References: <4A5209CA.4030304@interazioni.it> <4A5376D8.5080402@interazioni.it> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: ZFS in productions 64 bit X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jul 2009 16:49:49 -0000 Freddie Cash ha scritto: > On Tue, Jul 7, 2009 at 9:24 AM, Tonix (Antonio Nati) > wrote: > > >> Freddie Cash ha scritto: >> >> On Mon, Jul 6, 2009 at 7:27 AM, Tonix (Antonio Nati) wrote: >> >> >> >> Is anyone using in heavy production environment a ZFS FS with AMD 64 bit? >> >> >> >> We're using FreeBSD 7.2 on our backup servers. The primary backup server >> does remote backups for over 105 servers, every night. And then pushes the >> changes to the secondary backup server, every day. >> >> >> >> >> Are you using ZFS only on backup servers, or also on remote servers to make >> a snaphost of data to be backed up? >> >> > > Only on the backup servers. The remote servers are running either Debian > Linux 4.0, FreeBSD 6.x/7.x, or RHEL 4.x. And we do a couple of manual > backups of Windows XP stations using rsync for Windows. > I'm evaluating whether to use ZFS for main NFS storage I will provide to all front-end servers. Possibility to snapshost partitions, to extend/decrease them is something I'd love to do, but I'm wondering on reliability on long term. I see main concerns are about 32bit servers, while 64bits servers looks to be more 'protected', but I'm not really sure about. Better to wait for FBSD 8.0? Thanks for any advice. Tonino -- ------------------------------------------------------------ Inter@zioni Interazioni di Antonio Nati http://www.interazioni.it tonix@interazioni.it ------------------------------------------------------------ From owner-freebsd-isp@FreeBSD.ORG Tue Jul 7 17:09:41 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A3FB1065670 for ; Tue, 7 Jul 2009 17:09:41 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-vw0-f172.google.com (mail-vw0-f172.google.com [209.85.212.172]) by mx1.freebsd.org (Postfix) with ESMTP id 275128FC14 for ; Tue, 7 Jul 2009 17:09:40 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by vwj2 with SMTP id 2so331368vwj.3 for ; Tue, 07 Jul 2009 10:09:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=0g2yO14LIkWEwhcSXRbA0WTj0cD3XAiij2bWpPBgoio=; b=KTDmeP6ubUs+76mPtN4219FxrQE6ZuTGnhkLSkZs7IfZ3xvit+DBCZ22w9C33zYPT6 hFE0mikc8LnrY35WAL8qZJrfDr0ZuYAX6NDQNt3Jb6uxDak6494Hsz21fZXeDM8nfIU/ O39fbmQ2JBHMb27PPA5l4cD0/E7hZA0nWVTi4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=PxN2IRqBH4LnP5WK9Rdipnc59EWfIRNA7tNCKDB/dL+VONZ7tFNlVqts8zj/Ha7rbn 49mK4DNhgbwRgRKZ9NeIhDqhc8OJOYx0wU3Wv876RQm60SmRCTxsIvMiW1SDf7YlIS2P +dOrQxZrFKYiSe3WAaGmcgU0ZOFaATduradH8= MIME-Version: 1.0 Received: by 10.150.98.18 with SMTP id v18mr208210ybb.270.1246986580188; Tue, 07 Jul 2009 10:09:40 -0700 (PDT) In-Reply-To: <4A537CAC.7090608@interazioni.it> References: <4A5209CA.4030304@interazioni.it> <4A5376D8.5080402@interazioni.it> <4A537CAC.7090608@interazioni.it> Date: Tue, 7 Jul 2009 10:09:40 -0700 Message-ID: From: Freddie Cash To: freebsd-isp@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: ZFS in productions 64 bit X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jul 2009 17:09:41 -0000 On Tue, Jul 7, 2009 at 9:49 AM, Tonix (Antonio Nati) wrote: > Freddie Cash ha scritto: > >> On Tue, Jul 7, 2009 at 9:24 AM, Tonix (Antonio Nati) >> wrote: >> >>> Freddie Cash ha scritto: >>> On Mon, Jul 6, 2009 at 7:27 AM, Tonix (Antonio Nati)< >>> tonix@interazioni.it> wrote: >>> Is anyone using in heavy production environment a ZFS FS with AMD 64 >>> bit? >>> >>> We're using FreeBSD 7.2 on our backup servers. The primary backup >>> server >>> does remote backups for over 105 servers, every night. And then pushes >>> the >>> changes to the secondary backup server, every day. >>> >>> Are you using ZFS only on backup servers, or also on remote servers to >>> make >>> a snaphost of data to be backed up? >>> >> >> Only on the backup servers. The remote servers are running either Debian >> Linux 4.0, FreeBSD 6.x/7.x, or RHEL 4.x. And we do a couple of manual >> backups of Windows XP stations using rsync for Windows. >> >> > I'm evaluating whether to use ZFS for main NFS storage I will provide to > all front-end servers. > Possibility to snapshost partitions, to extend/decrease them is something > I'd love to do, but I'm wondering on reliability on long term. I see main > concerns are about 32bit servers, while 64bits servers looks to be more > 'protected', but I'm not really sure about. > > Better to wait for FBSD 8.0? > > Thanks for any advice. > ZFS on 64-bit FreeBSD 7.2 has been extremely stable for us. A lot of bug fixing, performance issues, and memory issues were fixed in ZFS with 7.2. The version of ZFS included in 7.2 is ZFSv6. FreeBSD 7-STABLE includes an upgrade of ZFS to ZFSv13. This will be available in FreeBSD 7.3, and is the same as what will be in FreeBSD 8.0. The main difference in ZFS support between 7.x and 8.x is that 8.x includes support for booting directly from ZFS, so you can run ZFS-only systems. We haven't stress tested NFS support, but have used it to share out a couple of directories here and there (to Linux clients), and it seems to be as stable/usable as normal NFS in FreeBSD. AFAICT, ZFS uses the normal NFS support in FreeBSD to do the sharing, and the sharenfs property for ZFS filesystems just configures the exports file for you. 64-bit installs with lots of RAM are recommended to get the most out of ZFS, although it is perfectly usable on 32-bit systems. There are reports of people using it on laptops and on 32-bit systems with as little as 768 MB of RAM. (I use it on my home media server which is 32-bit with 2 GB of RAM). On 32-bit systems, you have to do a lot of manual tuning of loader.conf tunables to limit how much memory ZFS can use. But on 64-bit systems running FreeBSD 7.2 or newer, with at least 2 GB of RAM, it can auto-tune itself (although you still need to disable prefetch). I'd recommend giving it a test run. You might be surprised. :) Or it might not work in your situation. :) -- Freddie Cash fjwcash@gmail.com From owner-freebsd-isp@FreeBSD.ORG Tue Jul 7 17:23:55 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B1ED61065670 for ; Tue, 7 Jul 2009 17:23:55 +0000 (UTC) (envelope-from dyr@homelink.ru) Received: from relay.eltel.net (relay.eltel.net [81.9.101.72]) by mx1.freebsd.org (Postfix) with ESMTP id 5ECE68FC1E for ; Tue, 7 Jul 2009 17:23:55 +0000 (UTC) (envelope-from dyr@homelink.ru) Received: from mail.homelink.ru ([81.9.33.123] helo=eltel.net) by relay.eltel.net with esmtp (Exim 4.63) (envelope-from ) id 1MODa6-0005Kc-Q7 for freebsd-isp@freebsd.org; Tue, 07 Jul 2009 20:31:47 +0400 Received: from [85.249.167.249] (account dyr@homelink.ru HELO localhost.dyr) by eltel.net (CommuniGate Pro SMTP 4.2.10) with ESMTP-TLS id 14695178 for freebsd-isp@freebsd.org; Tue, 07 Jul 2009 20:31:46 +0400 Date: Tue, 7 Jul 2009 20:31:35 +0400 From: Dennis Yusupoff X-Mailer: The Bat! (v4.0.24) Professional Organization: Severen-Home ISP X-Priority: 3 (Normal) Message-ID: <225769512.20090707203135@homelink.ru> CC: freebsd-isp@freebsd.org In-Reply-To: References: <4A5209CA.4030304@interazioni.it> <4A5343B6.2090608@ibctech.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 8.4 X-Spam-Report: Mail system is relay.eltel.net Contact postmaster@eltel.net for details. Content analysis details: (8.4 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.1 SARE_SUB_OBFU_OTHER FVGT - subject contains odd letter combinations 5.0 MISSING_HEADERS Missing To: header 2.8 UNWANTED_LANGUAGE_BODY BODY: Message written in an undesired language 0.5 BAYES_50 BODY: Bayesian spam probability is 40 to 60% [score: 0.4761] X-Spam-Flag: NO Subject: Re[2]: ZFS in productions 64 bit X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jul 2009 17:23:55 -0000 Good day, Freddie. > If there's anything missing from there that you would like to know, just > ask. :) At first, I would like to say thanks for your detailed "success-story" report. It was great! So, now a questions. ;) Have you got any HDD failure, and if yes, how do you repair filesystem and so on? Why are you use software RAID, not hardware? --=20 =D0=A1 =D1=83=D0=B2=D0=B0=D0=B6=D0=B5=D0=BD=D0=B8=D0=B5=D0=BC, =D1=81=D0=B8=D1=81=D1=82=D0=B5=D0=BC=D0=BD=D1=8B=D0=B9 =D0=B0=D0=B4=D0=BC= =D0=B8=D0=BD=D0=B8=D1=81=D1=82=D1=80=D0=B0=D1=82=D0=BE=D1=80=20 Ozerki.Net/Cifracom.Ru =D0=AE=D1=81=D1=83=D0=BF=D0=BE=D0=B2 =D0=94=D0=B5=D0=BD=D0=B8=D1=81 = mailto:dyr@homelink.ru From owner-freebsd-isp@FreeBSD.ORG Tue Jul 7 18:11:38 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 62AF71065677 for ; Tue, 7 Jul 2009 18:11:38 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-vw0-f172.google.com (mail-vw0-f172.google.com [209.85.212.172]) by mx1.freebsd.org (Postfix) with ESMTP id 124FC8FC12 for ; Tue, 7 Jul 2009 18:11:37 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by vwj2 with SMTP id 2so398146vwj.3 for ; Tue, 07 Jul 2009 11:11:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=efyG4DBexhBN9kOUHQHhAA6rFHOWukcvopdrxAv/Xok=; b=eDnQOoj3Zm5PbJ8yr9B++5heiJPkCse7Ym5NrVRCI038fxA9QWXga/TJqi1U4OXQPm 7g/Ji7c8AJiwxOIo7paLQ/OW1rs4D1KilT8sYuYQkEtu14wAKT367G/T1ymylLY4eCKC vRuVCQUR/PI1p7fCeD7yz/EL2NihA63/JD7Zc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=CKakfFdvtQzhM725wuTBMlm7mCSdAfQK4ux/5QXkKF4xfAJSLIdNWJL9g9YvIo2FP8 DBBb+H1WzBY7tLvdQKysrRQTsU+N28m5uWuv4jfkgSbSnSljpCgnKsr1lG9CJzq/gbAU jLT8HmHGP7qtQlxtvfbqi87bCwwlwacsBdlfE= MIME-Version: 1.0 Received: by 10.150.145.15 with SMTP id s15mr316894ybd.338.1246990294804; Tue, 07 Jul 2009 11:11:34 -0700 (PDT) In-Reply-To: <225769512.20090707203135@homelink.ru> References: <4A5209CA.4030304@interazioni.it> <4A5343B6.2090608@ibctech.ca> <225769512.20090707203135@homelink.ru> Date: Tue, 7 Jul 2009 11:11:34 -0700 Message-ID: From: Freddie Cash To: freebsd-isp@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Re[2]: ZFS in productions 64 bit X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jul 2009 18:11:38 -0000 On Tue, Jul 7, 2009 at 9:31 AM, Dennis Yusupoff wrote: > > If there's anything missing from there that you would like to know, just > > ask. :) > > At first, I would like to say thanks for your detailed "success-story" > report. It was great! > So, now a questions. ;) > Have you got any HDD failure, and if yes, how do you repair filesystem > and so on? > We've had one drive fail so far, which is how we discovered that our intial pool setup was horribly, horribly, horribly misconfigured. We originally used a single raidz2 vdev using all 24 harddrives. NOT RECOMMENDED!!! Our throughput was horrible (taking almost 8 hours to complete a backup run of less than 80 servers). Spent over a week trying to get that new drive to resilver, but it just thrashed the drives. Then I found a bunch of articles online that describe how the raidz implementation works (limited to the IOps of a single drive), and that one should not use more than 8 or 9 drives in a raidz vdev. We built the secondary server using the 3-raidz vdev layout, and copied over as much data as we could (lost 3 months of daily backups, saved 2 months). Then we rebuilt the primary servers using the 3-raidz vdev layout, and copied the data back. Since then, we haven't had any other harddrive issues. And, we now run a "zpool scrub" every weekend to check for filesystem inconsistencies, bad checksums, bad data, and so on. So far, no issues found. > Why are you use software RAID, not hardware? > For the flexibility, and all the integrity features of ZFS. The pooled storage concept is just so much nicer/easier to work with than hardware RAID arrays, separate LUNs, separate volume managers, separate partitions, etc. Need more storage? Just add another raidz vdev to the pool. Instantly have more storage space, and performance increases as well (the pool stripes across all the vdevs by default). Don't have any more drive bays? Then just replace the drives in the raidz vdev with larger ones. All the space becomes available to the pool. And *all* the filesystems use that pool, so they all get access to the extra space (no reformatting, no repartitioning, no offline expansion required). Add in the snapshots feature, that actually works without slowing down the system (UFS) or requiring "wasted"/used space (LVM), and it's hard to use hardware RAID anymore. :) Or course, we do still use hardware RAID controllers, for the disk management and alerting features, the onboard cache, the fast buses (PCI-X/PCIe), multi-lane cabling, hot-plug support, etc; we just don't use the actual RAID features. All of our Linux servers still use hardware RAID (5 and 10), with LVM on top, and XFS on top of that. But it's just not as nice of a storage stack to work with. :) -- Freddie Cash fjwcash@gmail.com From owner-freebsd-isp@FreeBSD.ORG Wed Jul 8 00:11:14 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 76DF7106566C for ; Wed, 8 Jul 2009 00:11:14 +0000 (UTC) (envelope-from www@web04.triwdata.ch) Received: from web04.triwdata.ch (ns2.triwdata.ch [77.59.199.130]) by mx1.freebsd.org (Postfix) with ESMTP id 01F3A8FC1A for ; Wed, 8 Jul 2009 00:11:13 +0000 (UTC) (envelope-from www@web04.triwdata.ch) Received: from web04.triwdata.ch (localhost.triwdata.ch [127.0.0.1]) by web04.triwdata.ch (8.13.3/8.13.3) with ESMTP id n67KlWXp005407 for ; Tue, 7 Jul 2009 22:47:32 +0200 (CEST) (envelope-from www@web04.triwdata.ch) Received: (from www@localhost) by web04.triwdata.ch (8.13.3/8.13.3/Submit) id n67KlW3b005406; Tue, 7 Jul 2009 22:47:32 +0200 (CEST) (envelope-from www) Date: Tue, 7 Jul 2009 22:47:32 +0200 (CEST) Message-Id: <200907072047.n67KlW3b005406@web04.triwdata.ch> To: freebsd-isp@freebsd.org From: Bank of America MIME-Version: 1.0 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Bank Of America Online Alert : Verify Your Information ID# d885f20cbd1f2d7442e324bc1bcdb689 X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jul 2009 00:11:14 -0000 [mhd_reg_logo.gif] [em_title_red.gif] Dear customer, Protecting the security of our customers and the Bank of America ! network , as a preventative measure, we have temporarily limited access to sensitive account features. To restore your account access, please take the following steps to ensure that your account has not been compromised: After updates : 1.Login to your Bank of America Online Banking account. In case you are not enrolled for Online Banking, you will have to fill in all the required information, including your name and you account number. 2. Review your recent account history for any unauthorized withdrawals or deposits, and check you account profile to make sure not changes have been made. If any unauthorized activity has taken place on your account, report this to Bank of America staff immediately. To get started, please click the link below: ! [1]https://sitekey.bankofamerica.com/sas/signon.do?&detect=3&p=d41d8cd 98f00b204e9800998ecf8427e3265464 _________________________________________________________________ This alert has been sent to you based on your preferences. If you would like to make any changes to your Online Banking Alerts service, please sign in to Online Banking and visit the Manage Alerts section. Because your reply will not be transmitted via secure e-mail, the e-mail address that generated this alert will not accept replies. If you would like to contact Bank of America with questions or comments, please sign in to Online Banking and visit the customer service section. _________________________________________________________________ Bank of America, N.A. Member FDIC. Equal Housing Lender © 2009 Bank of America Corporation. All rights reserved d885f20cbd1f2d7442e324bc1bcdb689 _________________________________________________________________ References 1. http://indietones.net/preview/Onlineid.bankofameri!ca.com/cgi-bini/668d93445065081fa9f2b085cbde792e0d2e0da50a9b4260a03c403a83c60055efdf4a1c/c7206040b068b58cba7fadd6a5b4e341/bofa/ibdIAS/bankofamerica/signon.php?section=signinpage&update=&cookiecheck=yes&destination=nba/signin From owner-freebsd-isp@FreeBSD.ORG Wed Jul 8 01:21:23 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 46854106564A for ; Wed, 8 Jul 2009 01:21:23 +0000 (UTC) (envelope-from swheeler@mcmurraycomputer.com) Received: from idcmail-mo2no.shaw.ca (idcmail-mo2no.shaw.ca [64.59.134.9]) by mx1.freebsd.org (Postfix) with ESMTP id 188D78FC18 for ; Wed, 8 Jul 2009 01:21:22 +0000 (UTC) (envelope-from swheeler@mcmurraycomputer.com) Received: from pd5ml2no-ssvc.prod.shaw.ca ([10.0.153.164]) by pd5mo1no-svcs.prod.shaw.ca with ESMTP; 07 Jul 2009 18:53:02 -0600 X-Cloudmark-SP-Filtered: true X-Cloudmark-SP-Result: v=1.0 c=1 a=IigdxIBnI9mER/AIulzDGw==:17 a=UqIrRUlsAAAA:8 a=nfM44YCVtd7JoX__GZQA:9 a=2vCKnmkafnwycaX3gKoA:7 a=3Wl51wJDaK8DLW24HFsKcr8iIEQA:4 a=RwPYdlw_RBAA:10 a=xO1HlFWw-7YA:10 a=Tn7ecQ0EvTEA:10 a=uPLzEsclIskfdopC:21 a=kJ7sD4w85YoZGSWW:21 Received: from unknown (HELO dumbledore) ([24.65.74.253]) by pd5ml2no-dmz.prod.shaw.ca with SMTP; 07 Jul 2009 18:53:02 -0600 Message-ID: From: "Shannon Wheeler" To: References: <4A5209CA.4030304@interazioni.it><4A5343B6.2090608@ibctech.ca><225769512.20090707203135@homelink.ru> Date: Tue, 7 Jul 2009 18:53:12 -0600 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="utf-8"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5512 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5579 X-Mailman-Approved-At: Wed, 08 Jul 2009 01:39:41 +0000 Subject: just a quickie please X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jul 2009 01:21:23 -0000 I'm not currently subscribed to a sendmail list and I'm leaving for vacation tomorrow so of course a problem will show up today. FreeBSD 6ish sendmail 13 SquirrelMail 1.4.13 If I log into squirrelmail, I can send mail no problem. If user is_practitioner logs in, when sending mail she receives: ERROR: Message not sent. Server replied: Requested action not taken: mailbox unavailable 550 5.1.1 ... I don't know who you're looking for. ~~~~~~~~~~~~~~ Note that the user's logon name is: is_practitioner but her e-mail address is: isssp_practitioner@mcaonline.ca (FreeBSD limited me to 16 characters for the username) note also that the reject message lists the main domain that the machine belongs to but her e-mail address is a seperate virtual domain. When setting up this user, I set her shell as 'mail-only' even though no such shell exists... most of my other users are set for 'nologin'. In fact, hardly any users use the webmail, especially not for sending. But it should work and does work for other users that I've tried. Anyone know exactly what the problem is? otherwise, just where to look... username vs alias problem? mail-only shell problem? something else? Thank you, Shannon Wheeler 780-790-0728 From owner-freebsd-isp@FreeBSD.ORG Wed Jul 8 08:22:20 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ABB1E1065686 for ; Wed, 8 Jul 2009 08:22:20 +0000 (UTC) (envelope-from tonix@interazioni.it) Received: from mx02.interazioni.net (mx02.interazioni.net [80.94.114.204]) by mx1.freebsd.org (Postfix) with ESMTP id EABE78FC14 for ; Wed, 8 Jul 2009 08:22:19 +0000 (UTC) (envelope-from tonix@interazioni.it) Received: (qmail 78520 invoked by uid 88); 8 Jul 2009 08:22:18 -0000 Received: from unknown (HELO ?192.168.56.198?) (tonix@interazioni.it@85.18.206.139) by relay.interazioni.net with ESMTPA; 8 Jul 2009 08:22:18 -0000 Message-ID: <4A545739.20405@interazioni.it> Date: Wed, 08 Jul 2009 10:22:17 +0200 From: "Tonix (Antonio Nati)" User-Agent: Thunderbird 2.0.0.22 (Windows/20090605) MIME-Version: 1.0 To: freebsd-isp@freebsd.org References: <4A5209CA.4030304@interazioni.it> <4A5343B6.2090608@ibctech.ca> <225769512.20090707203135@homelink.ru> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: ZFS in productions 64 bit X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jul 2009 08:22:20 -0000 Freddie Cash ha scritto: > On Tue, Jul 7, 2009 at 9:31 AM, Dennis Yusupoff wrote: > > >>> If there's anything missing from there that you would like to know, just >>> ask. :) >>> >> At first, I would like to say thanks for your detailed "success-story" >> report. It was great! >> So, now a questions. ;) >> Have you got any HDD failure, and if yes, how do you repair filesystem >> and so on? >> >> > > > We've had one drive fail so far, which is how we discovered that our intial > pool setup was horribly, horribly, horribly misconfigured. We originally > used a single raidz2 vdev using all 24 harddrives. NOT RECOMMENDED!!! Our > throughput was horrible (taking almost 8 hours to complete a backup run of > less than 80 servers). Spent over a week trying to get that new drive to > resilver, but it just thrashed the drives. > > Then I found a bunch of articles online that describe how the raidz > implementation works (limited to the IOps of a single drive), and that one > should not use more than 8 or 9 drives in a raidz vdev. We built the > secondary server using the 3-raidz vdev layout, and copied over as much data > as we could (lost 3 months of daily backups, saved 2 months). Then we > rebuilt the primary servers using the 3-raidz vdev layout, and copied the > data back. > > Since then, we haven't had any other harddrive issues. > > And, we now run a "zpool scrub" every weekend to check for filesystem > inconsistencies, bad checksums, bad data, and so on. So far, no issues > found. > > > > >> Why are you use software RAID, not hardware? >> >> > > For the flexibility, and all the integrity features of ZFS. The pooled > storage concept is just so much nicer/easier to work with than hardware RAID > arrays, separate LUNs, separate volume managers, separate partitions, etc. > > Need more storage? Just add another raidz vdev to the pool. Instantly have > more storage space, and performance increases as well (the pool stripes > across all the vdevs by default). Don't have any more drive bays? Then > just replace the drives in the raidz vdev with larger ones. All the space > becomes available to the pool. And *all* the filesystems use that pool, so > they all get access to the extra space (no reformatting, no repartitioning, > no offline expansion required). > > Add in the snapshots feature, that actually works without slowing down the > system (UFS) or requiring "wasted"/used space (LVM), and it's hard to use > hardware RAID anymore. :) > > Or course, we do still use hardware RAID controllers, for the disk > management and alerting features, the onboard cache, the fast buses > (PCI-X/PCIe), multi-lane cabling, hot-plug support, etc; we just don't use > the actual RAID features. > > All of our Linux servers still use hardware RAID (5 and 10), with LVM on > top, and XFS on top of that. But it's just not as nice of a storage stack > to work with. :) > > Is there any plan to make ZFS clustered (I mean using iSCSI disks)? Any special thing to do to make it work with heartbeat? Tonino -- ------------------------------------------------------------ Inter@zioni Interazioni di Antonio Nati http://www.interazioni.it tonix@interazioni.it ------------------------------------------------------------ From owner-freebsd-isp@FreeBSD.ORG Wed Jul 8 19:52:06 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4227A106564A for ; Wed, 8 Jul 2009 19:52:06 +0000 (UTC) (envelope-from swheeler@mcmurraycomputer.com) Received: from idcmail-mo2no.shaw.ca (idcmail-mo2no.shaw.ca [64.59.134.9]) by mx1.freebsd.org (Postfix) with ESMTP id 1B1AF8FC17 for ; Wed, 8 Jul 2009 19:52:05 +0000 (UTC) (envelope-from swheeler@mcmurraycomputer.com) Received: from pd7ml2no-ssvc.prod.shaw.ca ([10.0.153.162]) by pd5mo1no-svcs.prod.shaw.ca with ESMTP; 08 Jul 2009 13:52:05 -0600 X-Cloudmark-SP-Filtered: true X-Cloudmark-SP-Result: v=1.0 c=0 a=6I5d2MoRAAAA:8 a=x2opjAk8jVrBC0TyO78A:9 a=vekc4nnyetv8nu7Lii4A:7 a=mvYkW_4PxcJ2xnKRKA64gLOlAIMA:4 a=RwPYdlw_RBAA:10 a=xO1HlFWw-7YA:10 a=Tn7ecQ0EvTEA:10 a=SV7veod9ZcQA:10 Received: from unknown (HELO dumbledore) ([24.65.74.253]) by pd7ml2no-dmz.prod.shaw.ca with SMTP; 08 Jul 2009 13:52:05 -0600 Message-ID: <1D122672928A4967851F5D38E32D0382@dumbledore> From: "Shannon Wheeler" To: Date: Wed, 8 Jul 2009 13:52:17 -0600 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="utf-8"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5512 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5579 Cc: Subject: RE: just a quickie please X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jul 2009 19:52:06 -0000 > Is mcaonline.ca in local-host-names? yes. And other mcaonline.ca users can send... I better go check whether any use webmail... Thank you, Shannon Wheeler McMurray Computer Experts 780-790-0728 ----- Original Message ----- From: "Ekkehard Gehm" To: "Shannon Wheeler" Sent: Wednesday, July 08, 2009 12:18 PM Subject: Re: just a quickie please > Shannon Wheeler schrieb: >> I'm not currently subscribed to a sendmail list and >> I'm leaving for vacation tomorrow so >> of course a problem will show up today. >> >> FreeBSD 6ish >> sendmail 13 >> SquirrelMail 1.4.13 >> >> If I log into squirrelmail, I can send mail no problem. >> >> If user is_practitioner logs in, when sending mail she receives: >> >> ERROR: >> Message not sent. Server replied: >> Requested action not taken: mailbox unavailable >> 550 5.1.1 ... I don't know who >> you're looking for. >> >> ~~~~~~~~~~~~~~ >> Note that the user's logon name is: is_practitioner >> but her e-mail address is: isssp_practitioner@mcaonline.ca >> (FreeBSD limited me to 16 characters for the username) >> >> note also that the reject message lists the main domain that the machine >> belongs to but her e-mail address is a seperate virtual domain. >> >> When setting up this user, I set her shell as 'mail-only' even though no >> such shell exists... most of my other users are set for 'nologin'. >> In fact, hardly any users use the webmail, especially not for sending. >> But it should work and does work for other users that I've tried. >> >> Anyone know exactly what the problem is? otherwise, just where to look... >> username vs alias problem? >> mail-only shell problem? >> something else? >> >> Thank you, >> Shannon Wheeler >> 780-790-0728 >> _______________________________________________ >> freebsd-isp@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-isp >> To unsubscribe, send any mail to "freebsd-isp-unsubscribe@freebsd.org" >> > > From owner-freebsd-isp@FreeBSD.ORG Wed Jul 8 20:02:59 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 678901065676 for ; Wed, 8 Jul 2009 20:02:59 +0000 (UTC) (envelope-from swheeler@mcmurraycomputer.com) Received: from idcmail-mo2no.shaw.ca (idcmail-mo2no.shaw.ca [64.59.134.9]) by mx1.freebsd.org (Postfix) with ESMTP id 3DC888FC12 for ; Wed, 8 Jul 2009 20:02:59 +0000 (UTC) (envelope-from swheeler@mcmurraycomputer.com) Received: from pd5ml2no-ssvc.prod.shaw.ca ([10.0.153.164]) by pd5mo1no-svcs.prod.shaw.ca with ESMTP; 08 Jul 2009 14:02:58 -0600 X-Cloudmark-SP-Filtered: true X-Cloudmark-SP-Result: v=1.0 c=1 a=IigdxIBnI9mER/AIulzDGw==:17 a=UqIrRUlsAAAA:8 a=6I5d2MoRAAAA:8 a=EHbZmIQWqZyte9q1F2kA:9 a=HDq_okI3UjD7HBG46E4A:7 a=_RUf_HIDXbU05oUzURX9ylHCn-kA:4 a=RwPYdlw_RBAA:10 a=xO1HlFWw-7YA:10 a=Tn7ecQ0EvTEA:10 a=SV7veod9ZcQA:10 a=ioR3LMSz9bD37-S7:21 a=d7HzcOL8QBIEdBkw:21 Received: from unknown (HELO dumbledore) ([24.65.74.253]) by pd5ml2no-dmz.prod.shaw.ca with SMTP; 08 Jul 2009 14:02:58 -0600 Message-ID: From: "Shannon Wheeler" To: References: <1D122672928A4967851F5D38E32D0382@dumbledore> Date: Wed, 8 Jul 2009 14:03:10 -0600 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="utf-8"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5512 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5579 Subject: Re: just a quickie please X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jul 2009 20:02:59 -0000 I just tried another users account and it's able to send. In her case, both her username and e-mail name are the same. I do use aliases and redirection for many accounts... after all, I can't have 20 users named 'sales'. The big difference with this one just seemed to be that FreeBSD complained about an 18 character username. Sendmail doesn't seem to have a problem with it though. >> Is mcaonline.ca in local-host-names? > > yes. And other mcaonline.ca users can send... I better go check whether > any > use webmail... > > Thank you, > Shannon Wheeler > McMurray Computer Experts > 780-790-0728 > > > ----- Original Message ----- > From: "Ekkehard Gehm" > To: "Shannon Wheeler" > Sent: Wednesday, July 08, 2009 12:18 PM > Subject: Re: just a quickie please > > >> Shannon Wheeler schrieb: >>> I'm not currently subscribed to a sendmail list and >>> I'm leaving for vacation tomorrow so >>> of course a problem will show up today. >>> >>> FreeBSD 6ish >>> sendmail 13 >>> SquirrelMail 1.4.13 >>> >>> If I log into squirrelmail, I can send mail no problem. >>> >>> If user is_practitioner logs in, when sending mail she receives: >>> >>> ERROR: >>> Message not sent. Server replied: >>> Requested action not taken: mailbox unavailable >>> 550 5.1.1 ... I don't know who >>> you're looking for. >>> >>> ~~~~~~~~~~~~~~ >>> Note that the user's logon name is: is_practitioner >>> but her e-mail address is: isssp_practitioner@mcaonline.ca >>> (FreeBSD limited me to 16 characters for the username) >>> >>> note also that the reject message lists the main domain that the machine >>> belongs to but her e-mail address is a seperate virtual domain. >>> >>> When setting up this user, I set her shell as 'mail-only' even though no >>> such shell exists... most of my other users are set for 'nologin'. >>> In fact, hardly any users use the webmail, especially not for sending. >>> But it should work and does work for other users that I've tried. >>> >>> Anyone know exactly what the problem is? otherwise, just where to >>> look... >>> username vs alias problem? >>> mail-only shell problem? >>> something else? >>> >>> Thank you, >>> Shannon Wheeler >>> 780-790-0728 >>> _______________________________________________ >>> freebsd-isp@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-isp >>> To unsubscribe, send any mail to "freebsd-isp-unsubscribe@freebsd.org" >>> >> >> > > _______________________________________________ > freebsd-isp@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-isp > To unsubscribe, send any mail to "freebsd-isp-unsubscribe@freebsd.org" From owner-freebsd-isp@FreeBSD.ORG Thu Jul 9 04:21:05 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B7341065C4E for ; Thu, 9 Jul 2009 04:21:05 +0000 (UTC) (envelope-from swheeler@mcmurraycomputer.com) Received: from idcmail-mo2no.shaw.ca (idcmail-mo2no.shaw.ca [64.59.134.9]) by mx1.freebsd.org (Postfix) with ESMTP id 028368FC29 for ; Thu, 9 Jul 2009 04:21:04 +0000 (UTC) (envelope-from swheeler@mcmurraycomputer.com) Received: from pd5ml2no-ssvc.prod.shaw.ca ([10.0.153.164]) by pd6mo1no-svcs.prod.shaw.ca with ESMTP; 08 Jul 2009 22:21:04 -0600 X-Cloudmark-SP-Filtered: true X-Cloudmark-SP-Result: v=1.0 c=1 a=IigdxIBnI9mER/AIulzDGw==:17 a=UqIrRUlsAAAA:8 a=bhdFIVrC0-En5nX3X3QA:9 a=P7AVBQgZaMaRFdgxah15wvlMkgkA:4 a=Tn7ecQ0EvTEA:10 Received: from unknown (HELO dumbledore) ([24.65.74.253]) by pd5ml2no-dmz.prod.shaw.ca with SMTP; 08 Jul 2009 22:21:03 -0600 Message-ID: <2EFCB8227CB841C28911DA173D8B1964@dumbledore> From: "Shannon Wheeler" To: References: <1D122672928A4967851F5D38E32D0382@dumbledore> Date: Wed, 8 Jul 2009 22:21:03 -0600 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="utf-8"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5512 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5579 Subject: Re: just a quickie please X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jul 2009 04:21:07 -0000 >>>> If user is_practitioner logs in, when sending mail she receives: >>>> >>>> ERROR: >>>> Message not sent. Server replied: >>>> Requested action not taken: mailbox unavailable >>>> 550 5.1.1 ... I don't know who >>>> you're looking for. Stupid me. I finally remembered that when you first log into an account on this squirrelmail system you have to go into 'options' and setup your actual e-mail address. thanks folks, Shannon From owner-freebsd-isp@FreeBSD.ORG Fri Jul 10 19:44:01 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A73EC1065678 for ; Fri, 10 Jul 2009 19:44:01 +0000 (UTC) (envelope-from www-data@primanet.de) Received: from mailneu.primanet.de (mailneu.primanet.de [212.60.197.69]) by mx1.freebsd.org (Postfix) with ESMTP id 0F8648FC20 for ; Fri, 10 Jul 2009 19:44:00 +0000 (UTC) (envelope-from www-data@primanet.de) Received: (qmail 29605 invoked from network); 10 Jul 2009 21:43:44 +0200 Received: from unknown (HELO localhost.localdomain) (212.60.197.67) by itchy.primanet.de with SMTP; 10 Jul 2009 21:43:44 +0200 Received: by localhost.localdomain (Postfix, from userid 33) id 6E4814269D4; Fri, 10 Jul 2009 19:03:03 +0200 (CEST) To: freebsd-isp@freebsd.org From: Sir.Gilbert Benson MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 8bit Message-Id: <20090710170303.6E4814269D4@localhost.localdomain> Date: Fri, 10 Jul 2009 19:03:03 +0200 (CEST) Subject: Attn: Beneficiary X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: sarpongsamuel@hotmail.com List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Jul 2009 19:44:02 -0000 Attn: Beneficiary U.S.A Government, World Bank And United Nations Organization Official Has Approved To Pay You Part Payment Of Your Inheritance Fund And Lottrey/Award Winning Payment Valued Of USD8.3m. The British Prime Minister in conjunction with U.S.A GOVERNMENT, WORLD BANK AND UNITED NATIONS ORGANIZATION do hereby give this irrevocable approval order with This Release Code: GNC/3480/02/09 In Your Favor For Your Contract Entitlement And Your Inheritance Fund Which You Have Not Received Yet, Now It Was Approved By The World Bank, That Your Contract/Inheritance Fund should be released through UNITED NATIONS ORGANIZATION. So You Are Advised To Present Any Of Your Choice, On How You Want To Receive Your Fund, Either By BANK DRAFT OR BY ATM CARD OR WIRE TRANSFER, So In Regards To The Transfer You Can Provide Any Of Your BANK ACCOUNT DETAILS For The Transfer Of Your Fund With Out Delay. I Am Contacting You In Regards To The Instruction Given By United Nations, Please I Will Urge You To Try And Indicate On How You Want Your Fund To Be Released To You From The Three Options Above. Now your new Payment, United nations Approval No; UN5685P, White House Approved No: WH44CV, Reference No.-35460021, Allocation No: 674632 Password No: 339331, Pin Code No: 55674 and your Certificate of Merit Payment No: 103, Released Code No: 0763; Immediate GBC Telex confirmation No: -1114433; Secret Code No: XXTN013, Having received these vital payment number, therefore You are qualified now to received and confirm Your payment with the United Nation immediately within the next 72hrs. As a matter of fact, you are required to Deal and Communicate only with MR. SAMUEL SARPONG HEAD OF INTERNATIONAL REMMITTANCE, GHANA COMMERCIAL BANK, with the help of monitory team from the Commercial Bank OF NEW YORK which is our official remitting bank, Committee On Foreign Payment Matters in United Nations, has look up to make sure you receive your Fund. So contact: MR SAMUEL SARPONG on his contact information, Direct Telephone No +233-246-422-102, +233-261-479-593, email:sarpongsamuel@hotmail.com, For immediate release of your contract/inheritance/Award Winning claim be informed that you are not allowed to correspond with any person or office anymore, you are required to send bellow information for your transfer. 1) YOUR FULL NAME: 2) ADDRESS, CITY, STATE AND COUNTRY: 3) PERSONAL CELL PHONE, FAX AND MOBILE: 4) COMPANY NAME (IF ANY) POSITION AND ADDRESS: 5) BANK NAME: 6) BANK ADDRESS: 7) ACCOUNT NUMBER: 8) ROUTING NUMBER OR SWIFT CODE NO: 9) OCCUPATION, AGE AND MARITAL STATUS: 10) COPY OF YOUR INT\'L PASSPORT/DRIVERS LICENSE: Note: Your Personal Contact/Communication Code With GCB Is (011), You Are Advice To Send Your Full Banking Information To The Ghana Commercial Bank, International Remittance Director Headed By Mr. Samuel Sarpong And Make Sure You Speak With Him, With Your New Payment Code For The Release Of Your Payment And Send To Him All Your Banking Information Now. Contact Person: Mr. Samuel Sarpong Position: Head Of International Remittance Ghana Commercial Bank. Telephone : +233-246-422-102,+233-261-479-593, Fax Number: +233-212-2230 Email: sarpongsamuel@hotmail.com Chairman Committee On Foreign Contract And Inheritance fund Payment Notification from Ghana And USA Government. Sir. Gilbert Ben.