From owner-freebsd-jail@freebsd.org Sun Feb 19 10:57:12 2017 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BFFAACDBDA8 for ; Sun, 19 Feb 2017 10:57:12 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D16E1F9B for ; Sun, 19 Feb 2017 10:57:11 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from iris.berlin.strato.de ([87.169.120.214]) by mail.gmx.com (mrgmx103 [212.227.17.174]) with ESMTPSA (Nemesis) id 0MHokD-1ce1eD3okv-003aGo for ; Sun, 19 Feb 2017 11:51:57 +0100 From: Nikos Vassiliadis Subject: unionfs and nullfs combination To: freebsd-jail@freebsd.org Message-ID: <72a56f7e-8e71-2b98-0978-6de863013ce5@gmx.com> Date: Sun, 19 Feb 2017 11:51:56 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:kAkbPIuM+SHlUSF+GSFCGi8q0MtpXHCR1NOXBrC5lv3c6jXaXx6 OlIMWTiZuHn6QuEyOq30vMdUsqtCKoJ/rlNjTsaDuHWsG8JH5iTyC4eyo48Rd8cPxWNp+5V xx01edaN/XtourDfJ8Y5BMomr4GbFGBfBe7IrVU228vPub54bbAa6qPCBp1Ug67R4dTQ01c KcjYIjohjtaq+LLvP3N8Q== X-UI-Out-Filterresults: notjunk:1;V01:K0:k9UxH0Gp8T0=:d/s8S9ZMQvSj6e9PVK2Kdk EZCrl49FwqGdMoEt0+FcT/Tp3IuGSHslCm+HNdNI/r450YEikIihcqRpGdqvrkrCozNHZD01S RUBeKzW7iRbBeECZTPVvQywEzJZ0DPVyPkbyV9KRcM9Uz9pTxlJcGLSpsG3jrRRMoVI5Wv3hn uFOml+19Q+DB+kvMjEJnM45VcIu5wstBRQ69ULdYaySm6MxNhKQyMj8Ppfu/zoz2aYhaMdloB zVtQbouu3jWu79WHgpAUXcBfVk5MItKzf7cr9NEK0u/iUere+ljD9StUzNUbIL28AjYFaZ/vu aQpm6VMAyLWo9wqLJNHgypvXim9Afo/TXEvdZKPI8t+x72MF6wGBg+bbIGlSBy/8x870O3gLr Rd7on7TJASKgEcQEC5VUW/Tfrfsmw6dml53YtJqycpIbJgnrL5sjCCH+lel/vg8oMPPT6YjOU RIgxR4p6Pxe0j9bDSEVwQzKi1TFiW3bXNRjZPwPKGTWTXIkCYpH1jcAjT1jVx1lqofFuFroMm qL2nXAWIroTnvQ9kkZHv8IBdkU44m0vqE+AmLEoPThLNKg7Bpfa5cTTBLrq8jzsHhSNytoYIQ tRtrAUn18rJDVphPV5+N/E14jZQm16myp5SXqG3RT6Z0fhsIHRIzXY15xpJjZcD9QXTgAgOYm CDAnEBhWdXO3dZSORZAwX5IecCSc81DoDa0CKDOfwOJYrCCJ5tVbfUAaNxkaa6l/oGG7vLpGo FmBzZAy7JDkj44TZgEtHW20tIoT5ShYpdn7FrJF1xA9kcdpT2aOZCj21vxv9TZfWfl1O9Ayeq WvgImoD X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2017 10:57:12 -0000 Hi, One relatively cheap way to create thin jails in the pre-ZFS era, was to combine nullfs and unionfs (1). This seem to work only in 10 and previous branches. Do you use such a combination? It seems like a very relevant feature nowadays, when people use all these cloud-based systems, which oftentimes have little resources to run ZFS and UFS is most likely a better choice... 1: > https://rsmith.home.xs4all.nl/freebsd/using-nullfs-and-unionfs-for-the-ports-tree-in-a-jail.html Thanks in advance for your thoughts, Nikos From owner-freebsd-jail@freebsd.org Sun Feb 19 11:56:35 2017 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46AF8CE5C50 for ; Sun, 19 Feb 2017 11:56:35 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0D40E1B3B for ; Sun, 19 Feb 2017 11:56:35 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from pi by home.opsec.eu with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1cfQ6T-000FFj-NY; Sun, 19 Feb 2017 12:56:33 +0100 Date: Sun, 19 Feb 2017 12:56:33 +0100 From: Kurt Jaeger To: Nikos Vassiliadis Cc: freebsd-jail@freebsd.org Subject: Re: unionfs and nullfs combination Message-ID: <20170219115633.GW13006@home.opsec.eu> References: <72a56f7e-8e71-2b98-0978-6de863013ce5@gmx.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <72a56f7e-8e71-2b98-0978-6de863013ce5@gmx.com> X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2017 11:56:35 -0000 Hi! > One relatively cheap way to create thin jails in the pre-ZFS era, > was to combine nullfs and unionfs (1). This seem to work only in > 10 and previous branches. Do you use such a combination? We had this running with FreeBSD 6.x, but unionfs had issues, among them the whiteout problem. If you have a directory where many small files with random names are created in the upper layer, and deleted afterwards, the directory in the upper layer grows with each file because of the way whiteout files are handled. There's a mount option whiteout=whenneeded that should fix this, I no longer remember what stopped us from using it. > It seems like a very relevant feature nowadays, when people > use all these cloud-based systems, which oftentimes have little > resources to run ZFS and UFS is most likely a better choice... Funny, I have the impression that disk space, RAM and CPU are plenty compared to the past, so I would prefer ZFS anytime now. Our next jail box will probably use ZFS dedup with lots of RAM. -- pi@opsec.eu +49 171 3101372 3 years to go ! From owner-freebsd-jail@freebsd.org Sun Feb 19 12:38:43 2017 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C2ACCE5F92 for ; Sun, 19 Feb 2017 12:38:43 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E82DDFAE for ; Sun, 19 Feb 2017 12:38:42 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from pi by home.opsec.eu with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1cfQ9K-000FGH-Tl; Sun, 19 Feb 2017 12:59:30 +0100 Date: Sun, 19 Feb 2017 12:59:30 +0100 From: Kurt Jaeger To: Nikos Vassiliadis Cc: freebsd-jail@freebsd.org Subject: Re: unionfs and nullfs combination Message-ID: <20170219115930.GX13006@home.opsec.eu> References: <72a56f7e-8e71-2b98-0978-6de863013ce5@gmx.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <72a56f7e-8e71-2b98-0978-6de863013ce5@gmx.com> X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2017 12:38:43 -0000 Hi! > One relatively cheap way to create thin jails in the pre-ZFS era, > was to combine nullfs and unionfs (1). This seem to work only in > 10 and previous branches. Do you use such a combination? Ah, to correct myself here: We only used unionfs, not in combination with nullfs. Can you describe why nullfs with unionfs does not work in 11 ? -- pi@opsec.eu +49 171 3101372 3 years to go ! From owner-freebsd-jail@freebsd.org Sun Feb 19 14:22:18 2017 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 652E0CE5D59 for ; Sun, 19 Feb 2017 14:22:18 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B446698F for ; Sun, 19 Feb 2017 14:22:17 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from iris.berlin.strato.de ([192.166.200.216]) by mail.gmx.com (mrgmx001 [212.227.17.184]) with ESMTPSA (Nemesis) id 0MKHMk-1ch73R30TO-001kkf; Sun, 19 Feb 2017 15:21:19 +0100 Subject: Re: unionfs and nullfs combination To: Kurt Jaeger References: <72a56f7e-8e71-2b98-0978-6de863013ce5@gmx.com> <20170219115930.GX13006@home.opsec.eu> Cc: freebsd-jail@freebsd.org From: Nikos Vassiliadis Message-ID: Date: Sun, 19 Feb 2017 15:21:19 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: <20170219115930.GX13006@home.opsec.eu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:WD019ql4TXbw0ltkRN0bI54i864Qtpo/VvkpWT/WOzB/JDZRS32 6DEDDhpShVawUF/XQ9rie6OTO7NRvrJbxcbRk7ETkyoarS2/f+Q1OMuRvX7I1cJQW3lX864 /PnYfTzhoeHiSkMXtnuXuNI/79ppr6uYTTMszVtWACzCYZdNFFjdyYzIR5oLr5EPuU8LvIh o9dNvicldg++aQ5RmXT8Q== X-UI-Out-Filterresults: notjunk:1;V01:K0:M+nSS1HRZ/Y=:29auel9J8Vp+i4e0vWuw+G Vu+QlvwUnuROQh4q9K8y1scxnyOZYMyNEOhW+2Cr/yF75j+GsyqvAHhqWr3tZUUNfgEKstyV5 TiYCO9ACBQ01hs/KPuO/1E/yx9jKTlAKbnW7UUVVkZsQ3laNNv16PbjTGLa426BjUV2l6azyu Rs9OZXnE8IXHzO+DQdTWD7f1PrcxK0DVfGGg82sEaQikVAtpMQmEPQV6da1AQFFC2BH/E0R8r Tk0Offoi2PzufxEiBCAQ0WNjVXoGy+rY4evNGYnVRAW6zdImWDK643+5pFiVwtbNBNIZ08/vh YDK2YVPYE3+YJQxPzEZmz1OiKVH6uxVPUJ63K3fou7lpEg4/VaZss1U6wQUeT1LR0oIvAXUEH JZ1k1NMBVnk2wHht0rCqxTIMIdf2JHRCY+jewbdUathuZHXtFRWunhcCnoukqPgY1O1q8zvfj 4/A7LK24cKJTvDmYDcS3sejdoVUbE5F5whPxEBdcKxUXbgr2yItfjvD3hn1rw6WK5Ho7aS9lk bMcafR/Zboaq00BEXZY8yE/XlTrLJNTjwpj0eFidkSAVuHuMtaSprUl5WK91112+YDA6G69kl Gdo9ljsD2Ef1KygMpxhLz9g+iFkA/KBDA9o5WQtrHkuH04S72IOcceEqDF1epg/IWR7Kx0TGV 9lzK6KPX0RUVY/7GYSykrkco51v5y0wlmm5jgNS23IrXfd3hv81WTTPJQ2Oxzu51rO3RweirE yAgmtFotPIeG85gHwciOvHGvR9l1HJmw5XIFw8LU+Z8rD/aYyWqHSSBNaYOXQc9BCaB4WUpAB 60Ksqt4 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2017 14:22:18 -0000 Hi Kurt, On 02/19/2017 12:59 PM, Kurt Jaeger wrote: > Ah, to correct myself here: We only used unionfs, not in combination > with nullfs. Can you describe why nullfs with unionfs does not > work in 11 ? > It panics easily. I use the following shell script to create a working environment: > PREFIX=/jails > BASEJAIL=${PREFIX}/base-jail > JAILS="mongo-1 mongo-2 mongo-3 mongo-4 mongo-5 mongo-6" > > mkdir -p $BASEJAIL > for jail in $JAILS > do > mkdir -p ${PREFIX}/$jail > mkdir -p ${PREFIX}/upper/$jail > mount -t nullfs -o ro $BASEJAIL ${PREFIX}/$jail > mount -t unionfs -o noatime ${PREFIX}/upper/$jail ${PREFIX}/$jail > # mount -t devfs none ${PREFIX}/$jail/dev > # cp /etc/resolv.conf ${PREFIX}/$jail/etc/resolv.conf > done > > #chroot $PREFIX/mongo-1 rm -rv /var > #chroot $PREFIX/mongo-2 rm -rv /var Then I can trigger a panic if I run this: rm -rf /jails/mongo-*/* And the panic: > Unread portion of the kernel message buffer: > panic: Memory modified after free 0xfffff80004d0e000(1024) val=adc0dede @ 0xfffff80004d0e000 > > cpuid = 0 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe012058ce90 > vpanic() at vpanic+0x186/frame 0xfffffe012058cf10 > panic() at panic+0x43/frame 0xfffffe012058cf70 > trash_ctor() at trash_ctor+0x4b/frame 0xfffffe012058cf80 > uma_zalloc_arg() at uma_zalloc_arg+0x514/frame 0xfffffe012058cfe0 > unionfs_relookup() at unionfs_relookup+0x41/frame 0xfffffe012058d040 > unionfs_mkshadowdir() at unionfs_mkshadowdir+0x120/frame 0xfffffe012058d270 > unionfs_lookup() at unionfs_lookup+0x883/frame 0xfffffe012058d3c0 > VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xda/frame 0xfffffe012058d3f0 > vfs_cache_lookup() at vfs_cache_lookup+0xd6/frame 0xfffffe012058d450 > VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xda/frame 0xfffffe012058d480 > lookup() at lookup+0x6d2/frame 0xfffffe012058d520 > namei() at namei+0x504/frame 0xfffffe012058d5e0 > kern_statat() at kern_statat+0x98/frame 0xfffffe012058d790 > sys_fstatat() at sys_fstatat+0x2c/frame 0xfffffe012058d830 > amd64_syscall() at amd64_syscall+0x2f9/frame 0xfffffe012058d9b0 > Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe012058d9b0 > --- syscall (493, FreeBSD ELF64, sys_fstatat), rip = 0x8008ba62a, rsp = 0x7fffffffe728, rbp = 0x7fffffffe7e0 --- > KDB: enter: panic Thanks in advance for any ideas, Nikos From owner-freebsd-jail@freebsd.org Sun Feb 19 14:27:42 2017 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8FCF4CE5EBF for ; Sun, 19 Feb 2017 14:27:42 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E4AA9AF8 for ; Sun, 19 Feb 2017 14:27:41 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from iris.berlin.strato.de ([192.166.200.216]) by mail.gmx.com (mrgmx002 [212.227.17.184]) with ESMTPSA (Nemesis) id 0MKIEQ-1ch7332UAo-001gnG; Sun, 19 Feb 2017 15:20:55 +0100 Subject: Re: unionfs and nullfs combination To: Kurt Jaeger References: <72a56f7e-8e71-2b98-0978-6de863013ce5@gmx.com> <20170219115633.GW13006@home.opsec.eu> Cc: freebsd-jail@freebsd.org From: Nikos Vassiliadis Message-ID: <6a3bc22c-b203-2bda-c822-cb80b9368037@gmx.com> Date: Sun, 19 Feb 2017 15:20:49 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: <20170219115633.GW13006@home.opsec.eu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:RWvhmQS/1H60WYwOPostUhF71EC55SswmDPy9S5IFeWV6zKmm4E Z/w+/tAZsNI7ndjsEKagEFMKKEm5/YhkYAz/SZQpb2jGGeFDTaddvwDAgJW4isz9n1u5rH2 H1Ej8I8wkcvoHYYRmgINDaAQ+zpP+YYqVOCHnlDVyrvaHQ0b3cYyUkKo9fkxDSpn5iXptKI xnFH3uMpVZ9e/KXeirXBg== X-UI-Out-Filterresults: notjunk:1;V01:K0:XXtJ6l1U/Vo=:DDRa6D+jWjBW2Hv6Ctz1ez +Ix3fF0bYlqPALPmnUN6pUpG3DGHdNCNYrLSGt3uBSItD/yKscJFVTBL3fazcxEao3iPLxwr2 bQr+aGPfmfG5q35qa0Mem1sIDsXVrDRmiQgG5GxxzFcZtbxwu+0keWUujH1HaDgsjofO5YnUw TZ+fgatSBsIZ2BBMRSxECvx8yGc2Xkts7FAsr4kkOfPXyC2FMrIO4neNS/Fk4RqeSr0OGhfWY e+UgJMvluJ4UBVB/SHDsrNWnmgiSc52tTjybnJ21mtl78NfXFPgGVrr4HE6vFLudExdEp24tb 0Coab8PsWRJuY8DMkjh1gjJVTx6zjI4TxZTsJWUk/VYuduFzKo/21LuVwBZ83vZULOoI+CTPE 9FSDXnmNmAGFBTmEgp1jgRkKBqbkI9eTP0t9S7yG3DiFGjYCAvnR4y9QTkdZRPCHk2CBux1OQ WagOFQx375kQeWW3XTVFIdJpXiUABSKHcOIgH1S2EsZ8kaNcPjjZrG9QVrrxSVhK3fg5Bh6RJ LjYlqHlMHtpY9Sgd8SFwDhufFZdxY0MzncyzphQ3Sdtw5vkuJTFCkW7iR/UXafiRzVfC3eER5 XDuVN7cXaKM9Lz69n8lK7w548ysNjLX/GrfT5psuWbNUkmnmjXz03WVHkbVcA/rCynhuic9zX /9dwz51tonAqSz5L1erryoODGFdYkpZlN6zKsP2hz8e95v3etKnuapHjUJvSzSg0cT3fvj33l TGDTWqKPwEjgbfUHTuYOma8MG8Lr8TCXQoJATt8V0W/GEQXk2qk2QF1rZ3IwFD1SQBYiEIoGe zqJcCMB X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2017 14:27:42 -0000 Hi Kurt, On 02/19/2017 12:56 PM, Kurt Jaeger wrote: > We had this running with FreeBSD 6.x, but unionfs had issues, > among them the whiteout problem. I am not sure exactly when unionfs was re-written... Sometime during 7-CURRENT but I am not sure if everything was backported to 6. >> It seems like a very relevant feature nowadays, when people >> use all these cloud-based systems, which oftentimes have little >> resources to run ZFS and UFS is most likely a better choice... > > Funny, I have the impression that disk space, RAM and CPU are > plenty compared to the past, so I would prefer ZFS anytime now. > Our next jail box will probably use ZFS dedup with lots of RAM. > Very true. And I love ZFS. I am talking about cloud installations where VM are created and destroyed in a fast pace and are mostly small like 1GB of RAM. UFS is very relevant for such installations. And being able to be cloud-friendly is a good thing:) Nikos From owner-freebsd-jail@freebsd.org Thu Feb 23 22:57:31 2017 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D7B29CEB5FA for ; Thu, 23 Feb 2017 22:57:31 +0000 (UTC) (envelope-from lily.james@b2bsalesus.com) Received: from mail-pg0-x234.google.com (mail-pg0-x234.google.com [IPv6:2607:f8b0:400e:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A5711B36 for ; Thu, 23 Feb 2017 22:57:31 +0000 (UTC) (envelope-from lily.james@b2bsalesus.com) Received: by mail-pg0-x234.google.com with SMTP id 1so2224640pgi.1 for ; Thu, 23 Feb 2017 14:57:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=b2bsalesus-com.20150623.gappssmtp.com; s=20150623; h=return-receipt-to:from:to:subject:date:message-id:mime-version :thread-index:content-language:disposition-notification-to; bh=w6pcrk7g9kEngHKC/S8zhTABBMIbLjCwfTxalPkWsd0=; b=EGInWBuX76e40NqI+b41Vlvid73OAvx9QaNVuPOv/t9Zi8IdwgQBwria69/1CoHxlK 4iSNL6gRL1xEr21NITekeO44VmKi8vnh1esmER0jooDTRgcbhWY45rWdaRaBRwMnCvXd ryTgx/J12b9azz99Ndx4z69K9xkl5REfoi2sHaoNk15NkigNMD1qrw7qDs5EosD8p1RY dgpwt8SiKzPNuaQNgFG3kkaKlYVK0mKlcEfJ969G2jbJxj0TF29nE+hGP+ho9NCXFS/M AcxfpiHYdScTU2Rzmh3JNzIgRJ0fwa1EyS+MAwcgl7XDcmwMtch5c8qhaTfQfHFZGRwl SAYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:return-receipt-to:from:to:subject:date :message-id:mime-version:thread-index:content-language :disposition-notification-to; bh=w6pcrk7g9kEngHKC/S8zhTABBMIbLjCwfTxalPkWsd0=; b=OvAq9rw5HnfXflv1wIsFzHoJpGAcZYDFCgMO57vwM/FOUI0Ur2VAs7v5//f8gn4pcs FhcUGRL2D0eijM2nOOTC34cL2z1Lnn8aebkNKoFkRzsmaoWmSby7ylAZ898zjUCCcyUt vjkLFqtVS8ufNoSXn8iuRYscFKdmjCe7NSiRLC+YsNK8CCJvvqqYBahmbrTcXGldiZky ODsN0mOI7Ac5MA2+hV8ssBrBSXXUTevGRHA32EXAJbFJAoRXTHfpJMszp5Am+mD5xlWN ko4utIVEz1LcAljoot/jEAmwVYRkv8QwLrTD966hoY1yq+7rKIxvsquagxBrpQ4gKAhv U82A== X-Gm-Message-State: AMke39npOTU91EipW3BFavM4m8g6hL0I8yr8tuwCcDaxZg29P6oQFPW70kr3mALHy5n/MA== X-Received: by 10.98.158.219 with SMTP id f88mr9305203pfk.50.1487890650937; Thu, 23 Feb 2017 14:57:30 -0800 (PST) Received: from SBSBLRDT09 ([106.51.26.228]) by smtp.gmail.com with ESMTPSA id e2sm11606386pga.61.2017.02.23.14.57.29 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 23 Feb 2017 14:57:30 -0800 (PST) From: "Lily James" To: Subject: Wireless Communications Industry Sales Leads Date: Thu, 23 Feb 2017 17:55:57 -0500 Message-ID: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AdKOJqZTbWmVHo2ISNOPH39p9fHvnw== Content-Language: en-us Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Feb 2017 22:57:31 -0000 Hi, Hope you are doing well ! For your upcoming Marketing promotions, would you be interested in owning an "Updated and Tele Verified B2B Database / Potential Customer Opt-in List" of "Wireless Communication Industry Professionals" who might be interested in your products or services . We maintain over 48 Million B2B Contacts across North America, UK, Europe and Australia. Every contact includes Verified/Opt-In Email Addresses, Phone Number, Physical Address, Name, Website, Company, Title, FAX Number, Industry and SIC Codes. Industry Sectors: * Telecom / Mobile / Wireless Infrastructure. * Owners / Operators / Carriers / Investment Community Representatives. * Equipment Manufacturers / Service Providers / Tower and Professional services * Product Distribution (Dealers, Manufacturers, Reps, Distributors, Systems Integrators. * Enterprises / Resellers / Manufacturers / Developers * Internet Telephony * Network Technologies / Infrastructure Solutions * ICT / IOT * Wi-Fi / Wireless Broadband / FTTH / FTTX * SDN / NFV / Optical Networking * 2G / 3G / 4G / 5G / Next-Gen * M2M * Microwave * Test / Measurement Equipment. * Antennas, Cable Television Transmission * Cellular Phones / GPS * Radio and Television Broadcasting Equipment / Receivers and Transceivers * Satellite Communication Equipment. Job Titles: * C-Level / Decision Makers / Founder / Principal / President / Partner * CXO - CEO/CMO/COO/CFO/CTO/ VP / DIRECTOR / Sr. MANAGERIAL * Business Development / Sales / Marketing / Marcom / Corporate Communications / PR * Pre-Sales / Sales Support * Legal / Contracts / Projects * Investor / Financial Analyst * SCM / Logistics / Purchase / Materials / Sourcing / Procurement. * Engineering / Technical Operations * Consultant / Research & Development Specialists * Installation / Maintenance If you would like to reach any other Industries/Titles. You can just fill in below and revert back on the same email. Target Industries: ______? (We maintain Data across All B2B Industries across North America, UK and Europe). Job Titles: ______? Geography: ______? Please let me know your thoughts. So that I can send you few samples to check our database accuracy along with counts and pricing. Look forward to hear from you. Best Regards, Lily James| Marketing Analyst _____ To remove from this mailing: reply with subject line as "Leave out" and indicate your email address to be removed from our database.