From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 12 03:14:05 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D2A1DD75; Mon, 12 Jan 2015 03:14:05 +0000 (UTC) Received: from mail-la0-x22b.google.com (mail-la0-x22b.google.com [IPv6:2a00:1450:4010:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6C9568CB; Mon, 12 Jan 2015 03:14:05 +0000 (UTC) Received: by mail-la0-f43.google.com with SMTP id s18so22038897lam.2; Sun, 11 Jan 2015 19:14:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:reply-to:to:cc:subject:date:mime-version :content-type:content-transfer-encoding:thread-index :content-language; bh=zoqZylkcTprr558XBgRdwHrQPcQ6WnXkjaotW0mzm8o=; b=dmb+QxcIFQin1TXfPJq3ksZV4iHjT5PzImcpviP0I5/yD5PV+Sm6yOcnvSPnycAev0 6t6PEk3EZxdeXGytwiwyB5XfBCit4SSR21fn6wi+ktTiAaOsUwbK4RLSq2N17Qb0Yw1g WtRcAiZYEajo76PE1hkHMjYQVa5D+bsLJnekMXJW1yI1ZgUVDGc789fTyH5pn0GuXIgE 1PB086LUhWewny+JgHvCHKYTQuFfpcWEzoJ40DYpvnUtIF9A2z1OTWKoh/HMOyQhCe8X lLWGP2FAPLGnwcNiMazfyjD4jQS69IRNa7JhFTivh52V7+MFk00dvkKoYnUivpSbJxe2 7vJQ== X-Received: by 10.152.115.146 with SMTP id jo18mr34634340lab.9.1421032443159; Sun, 11 Jan 2015 19:14:03 -0800 (PST) Received: from rimwks1w7x64 ([2001:470:1f15:8e:b007:2759:7397:9491]) by mx.google.com with ESMTPSA id w3sm3718872lag.35.2015.01.11.19.14.01 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 11 Jan 2015 19:14:02 -0800 (PST) Message-ID: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> X-Google-Original-Message-ID: <010101d02e15$d0862eb0$71928c10$@IM@gmail.com> From: rozhuk.im@gmail.com X-Google-Original-From: Reply-To: To: , Subject: ChaCha8/12/20 and GEOM ELI tests Date: Mon, 12 Jan 2015 06:13:59 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1251" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdAuFc83D4H1X8qAQ5a13xwwqTkDpw== Content-Language: ru Cc: rozhuk.im@gmail.com X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jan 2015 03:14:06 -0000 FreeBSD firewall 11.0-CURRENT FreeBSD 11.0-CURRENT #3 r276867M: Fri Jan = 9 09:34:39 MSK 2015 root@firewall:/usr/obj/usr/src/sys/RIMx64 amd64 Cha=D1ha patch: http://netlab.linkpc.net/download/software/FreeBSD/patches/chacha.patch HW: Core Duo E8500, 8Gb DDR2-800. dd if=3D/dev/zero of=3D/dev/md0 bs=3D1m 2148489421 bytes/sec # sector =3D 512b 3DES-CBC-192 =3D 20773120 bytes/sec AES-CBC-128 =3D 85276853 bytes/sec AES-CBC-256 =3D 68893016 bytes/sec AES-XTS-128 =3D 68194868 bytes/sec AES-XTS-256 =3D 56611573 bytes/sec Blowfish-CBC-128 =3D 11169657 bytes/sec Blowfish-CBC-256 =3D 11185891 bytes/sec Camellia-CBC-128 =3D 78077243 bytes/sec Camellia-CBC-256 =3D 65732219 bytes/sec ChaCha8-XTS-256 =3D 258042765 bytes/sec ChaCha12-XTS-256 =3D 223616967 bytes/sec ChaCha20-XTS-256 =3D 176005366 bytes/sec XChaCha8-XTS-256 =3D 228292624 bytes/sec XChaCha12-XTS-256 =3D 195577624 bytes/sec XChaCha20-XTS-256 =3D 152247267 bytes/sec XChaCha20-XTS-128 =3D 152717737 bytes/sec ! 128 bit key have same speed = as 256 # sector =3D 4kb 3DES-CBC-192 =3D 22018189 bytes/sec AES-CBC-128 =3D 104097143 bytes/sec AES-CBC-256 =3D 81983833 bytes/sec AES-XTS-128 =3D 78559346 bytes/sec AES-XTS-256 =3D 66047200 bytes/sec Blowfish-CBC-128 =3D 38635464 bytes/sec Blowfish-CBC-256 =3D 38810555 bytes/sec Camellia-CBC-128 =3D 92814510 bytes/sec Camellia-CBC-256 =3D 75949489 bytes/sec ChaCha8-XTS-256 =3D 337336982 bytes/sec ChaCha12-XTS-256 =3D 284740187 bytes/sec ChaCha20-XTS-256 =3D 217326865 bytes/sec XChaCha8-XTS-256 =3D 328424551 bytes/sec XChaCha12-XTS-256 =3D 278579692 bytes/sec XChaCha20-XTS-256 =3D 211660225 bytes/sec Optimized AES-XTS - speed like AES-CBC: AES-XTS-128 =3D 102841051 bytes/sec AES-XTS-256 =3D 80813644 bytes/sec Prepare env: mdmfs -S -o async -s 4g md /media Per test: geli init -v -e ALGO_NAME -i 8 -l KEY_LEN -s SECTOR_SIZE /dev/md0 geli attach /dev/md0 dd if=3D/dev/zero of=3D/dev/md0.eli bs=3D1m geli detach /dev/md0.eli top -aSCHIP CPU 0: 0.0% user, 0.0% nice, 45.8% system, 0.0% interrupt, 54.2% idle CPU 1: 0.0% user, 0.0% nice, 54.2% system, 0.0% interrupt, 45.8% idle Mem: 4104M Active, 364M Inact, 558M Wired, 828M Buf, 2927M Free Swap: PID USERNAME PRI NICE SIZE RES STATE C TIME CPU COMMAND 10 root 155 ki31 0K 32K RUN 0 842:15 54.04% = [idle{idle: cpu0}] 5319 root 43 - 0K 16K CPU1 1 0:30 51.55% = [g_eli[1] md0] 10 root 155 ki31 0K 32K RUN 1 842:36 45.69% = [idle{idle: cpu1}] 5318 root 43 - 0K 16K RUN 0 0:32 43.47% = [g_eli[0] md0] 3490 root -8 - 0K 16K mdwait 1 2:11 2.79% [md0] 12 root -8 - 0K 48K - 1 0:48 1.25% [geom{g_up}] 5399 root -8 0 12188K 3904K physwr 1 0:00 0.81% dd if=3D/dev/zero of=3D/dev/md0.eli bs=3D1m 3506 root 40 0 21668K 3688K CPU0 0 0:11 0.16% top = -aSCHIP 12 root -8 - 0K 48K - 1 0:06 0.14% [geom{g_down}] From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 12 05:24:41 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 40161219 for ; Mon, 12 Jan 2015 05:24:41 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1on0053.outbound.protection.outlook.com [157.56.110.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EC37E61A for ; Mon, 12 Jan 2015 05:24:40 +0000 (UTC) Received: from BN3PR0801MB0931.namprd08.prod.outlook.com (25.160.184.25) by BN3PR0801MB0929.namprd08.prod.outlook.com (25.160.184.23) with Microsoft SMTP Server (TLS) id 15.1.53.17; Mon, 12 Jan 2015 05:24:31 +0000 Received: from BN3PR0801MB0931.namprd08.prod.outlook.com ([25.160.184.25]) by BN3PR0801MB0931.namprd08.prod.outlook.com ([25.160.184.25]) with mapi id 15.01.0053.000; Mon, 12 Jan 2015 05:24:31 +0000 From: "Pokala, Ravi" To: "freebsd-hackers@freebsd.org" Subject: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged Thread-Topic: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged Thread-Index: AQHQLigK82/0oK+YUEaY/TDDejfaPg== Date: Mon, 12 Jan 2015 05:24:31 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.7.141117 x-originating-ip: [24.6.178.251] authentication-results: spf=none (sender IP is ) smtp.mailfrom=rpokala@panasas.com; x-dmarcaction-test: None x-microsoft-antispam: BCL:0;PCL:0;RULEID:(3005003);SRVR:BN3PR0801MB0929; x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:; SRVR:BN3PR0801MB0929; x-forefront-prvs: 0454444834 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(979002)(6009001)(164054003)(199003)(189002)(107886001)(102836002)(40100003)(122556002)(106356001)(68736005)(229853001)(92566002)(2351001)(450100001)(62966003)(77156002)(99286002)(66066001)(110136001)(54356999)(97736003)(105586002)(2656002)(106116001)(2900100001)(46102003)(83506001)(50986999)(36756003)(64706001)(86362001)(87936001)(101416001)(969003)(989001)(999001)(1009001)(1019001); DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR0801MB0929; H:BN3PR0801MB0931.namprd08.prod.outlook.com; FPR:; SPF:None; MLV:ovrnspm; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: panasas.com does not designate permitted sender hosts) Content-Type: text/plain; charset="us-ascii" Content-ID: <6C49B7CD126D414E97FF45E1427CC3EC@namprd08.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: panasas.com X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Jan 2015 05:24:31.2380 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: acf01c9d-c699-42af-bdbb-44bf582e60b0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0801MB0929 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jan 2015 05:24:41 -0000 Hi folks, Several of us have noticed that there's a long pause at the start of booting the kernel on amd64 systems with large amounts of RAM. During this pause, the kernel is mapping in the memory ranges, but does not emit any progress indicators. Because this can take quite a while, it can look like the kernel has hung. I filed PR 196650 about this issue; this patch just prints out a dot for every GB that's mapped in. We've been using this patch for years at Panasas, and I'm hoping someone can submit it for me. Thanks, Ravi Index: sys/amd64/amd64/machdep.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/amd64/amd64/machdep.c (revision 276903) +++ sys/amd64/amd64/machdep.c (working copy) @@ -1575,6 +1575,9 @@ u_long physmem_start, physmem_tunable, memtest; pt_entry_t *pte; quad_t dcons_addr, dcons_size; + u_int32_t page_counter; + int PAGES_PER_MB =3D ((1024 * 1024) / PAGE_SIZE); + int PAGES_PER_GB =3D (PAGES_PER_MB * 1024); =20 bzero(physmap, sizeof(physmap)); basemem =3D 0; @@ -1678,6 +1681,8 @@ * physmap is in bytes, so when converting to page boundaries, * round up the start address and round down the end address. */ + printf("Mapping system memory"); + page_counter =3D 0; for (i =3D 0; i <=3D physmap_idx; i +=3D 2) { vm_paddr_t end; =20 @@ -1688,6 +1693,14 @@ int tmp, page_bad, full; int *ptr =3D (int *)CADDR1; =20 + /* + * Print a "." every GB, to show we're making progress + */ + page_counter++; + if ((page_counter % PAGES_PER_GB) =3D=3D 0) { + printf("."); + } + full =3D FALSE; /* * block out kernel memory as not available. @@ -1792,6 +1805,9 @@ break; } } + printf("\nMapped %d GB + %d MB total\n", + (page_counter / PAGES_PER_GB), + ((page_counter % PAGES_PER_GB) / PAGES_PER_MB)); *pte =3D 0; invltlb(); =20 From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 12 05:34:12 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 11FBF3BF for ; Mon, 12 Jan 2015 05:34:12 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id C62856F1 for ; Mon, 12 Jan 2015 05:34:11 +0000 (UTC) Received: from [192.168.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 302CD8D4AE for ; Mon, 12 Jan 2015 05:27:19 +0000 (UTC) Message-ID: <54B35B36.4040504@freebsd.org> Date: Mon, 12 Jan 2015 00:27:18 -0500 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged References: In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ULFWqxnx12mCOCA5xt46Ojt8mPgQE16oE" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jan 2015 05:34:12 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --ULFWqxnx12mCOCA5xt46Ojt8mPgQE16oE Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2015-01-12 00:24, Pokala, Ravi wrote: > Hi folks, >=20 > Several of us have noticed that there's a long pause at the start of > booting the kernel on amd64 systems with large amounts of RAM. During t= his > pause, the kernel is mapping in the memory ranges, but does not emit an= y > progress indicators. Because this can take quite a while, it can look l= ike > the kernel has hung. I filed PR 196650 about this issue; this patch jus= t > prints out a dot for every GB that's mapped in. We've been using this > patch for years at Panasas, and I'm hoping someone can submit it for me= =2E >=20 > Thanks, >=20 > Ravi >=20 >=20 > Index: sys/amd64/amd64/machdep.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/amd64/amd64/machdep.c (revision 276903) > +++ sys/amd64/amd64/machdep.c (working copy) > @@ -1575,6 +1575,9 @@ > u_long physmem_start, physmem_tunable, memtest; > pt_entry_t *pte; > quad_t dcons_addr, dcons_size; > + u_int32_t page_counter; > + int PAGES_PER_MB =3D ((1024 * 1024) / PAGE_SIZE); > + int PAGES_PER_GB =3D (PAGES_PER_MB * 1024); > =20 > bzero(physmap, sizeof(physmap)); > basemem =3D 0; > @@ -1678,6 +1681,8 @@ > * physmap is in bytes, so when converting to page boundaries, > * round up the start address and round down the end address. > */ > + printf("Mapping system memory"); > + page_counter =3D 0; > for (i =3D 0; i <=3D physmap_idx; i +=3D 2) { > vm_paddr_t end; > =20 > @@ -1688,6 +1693,14 @@ > int tmp, page_bad, full; > int *ptr =3D (int *)CADDR1; > =20 > + /* > + * Print a "." every GB, to show we're making progress > + */ > + page_counter++; > + if ((page_counter % PAGES_PER_GB) =3D=3D 0) { > + printf("."); > + } > + > full =3D FALSE; > /* > * block out kernel memory as not available. > @@ -1792,6 +1805,9 @@ > break; > } > } > + printf("\nMapped %d GB + %d MB total\n", > + (page_counter / PAGES_PER_GB), > + ((page_counter % PAGES_PER_GB) / PAGES_PER_MB)); > *pte =3D 0; > invltlb(); > =20 >=20 >=20 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" >=20 Which version(s) of FreeBSD were you seeing this delay on? I know older versions had it, quite badly. setting hw.memtest.tests=3D0 in /boot/loader.conf makes most of the delay= go away, and the default on -CURRENT is now 0 instead of 1. IIRC, in 10, it defaults to 1, but is switched to 0 in the case of virtual machines, because touching every page of memory ruined memory over-commit type features. Is this feature still useful with memtest.tests=3D0? --=20 Allan Jude --ULFWqxnx12mCOCA5xt46Ojt8mPgQE16oE Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJUs1s7AAoJEJrBFpNRJZKfXCAP/jzLD28j26Af1RnPN8UdyZHg BOoB2CEhuq6h7VLUl+8Ec6mJ6+JgCJYeBiuVPj6h5VGjec/jVwH2S2CVuGzOYJy5 FfzjBZnRShTEVFqN9ySZRgg1q1R7HxHEm/e27Z8EDLLvqPUfqp2rg1s3QW48lDcS /psZj/LiIMplltpXw4GBSIiaJeDamHLJAfy1FN2edcMl1vh2qZkhSqO9tvwVPP3S UcjUNwftDw03cKrA6+76M50RCqfI0nwjg6+M3inahbWQH/dqzi1gNFm1N3F69paY +2M5QYqUSD7r68yzhEFASZR4D/uuv95mdHNGCiINUBTtQYzLZmsRgIs7E1bN0r6D kYNSuAbNTdc/gLOieaSjPevEs0R0aLEqM14Yb//76mljEytvi8NUYYjuxtnpwm3o TAL2cxiPAJ/z8wob5VzPL8sva+2aoWPqeHfpH6/LDrCBw1xRkeNFIUKoYmxlBjyG 2mg4+C2TdkWbBrTVaK7v5+W8hDY7gOMSPP+EuBi3F/ez6OodNpwZHGRIKhIFeqsb V0g40P6dL5Zg228wIBpBK+p3wcSzeTSh44VKQaRlt1ju6YeGgeNB1qFagL7kRj3K qfforsPsp0NZoKTHMQC1No5OEBRenXjpIJS56c6CL2DLH7qCa1zwre6En3yof90n DzlleLrOezbK1gmWSoy8 =teXD -----END PGP SIGNATURE----- --ULFWqxnx12mCOCA5xt46Ojt8mPgQE16oE-- From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 12 07:22:57 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F563262; Mon, 12 Jan 2015 07:22:57 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 59459FE; Mon, 12 Jan 2015 07:22:57 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t0C7MoSR009962 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 11 Jan 2015 23:22:50 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t0C7Mnvp009961; Sun, 11 Jan 2015 23:22:49 -0800 (PST) (envelope-from jmg) Date: Sun, 11 Jan 2015 23:22:49 -0800 From: John-Mark Gurney To: rozhuk.im@gmail.com Subject: Re: ChaCha8/12/20 and GEOM ELI tests Message-ID: <20150112072249.GM1949@funkthat.com> References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Sun, 11 Jan 2015 23:22:50 -0800 (PST) Cc: freebsd-hackers@freebsd.org, freebsd-geom@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jan 2015 07:22:57 -0000 rozhuk.im@gmail.com wrote this message on Mon, Jan 12, 2015 at 06:13 +0300: > FreeBSD firewall 11.0-CURRENT FreeBSD 11.0-CURRENT #3 r276867M: Fri Jan 9 > 09:34:39 MSK 2015 root@firewall:/usr/obj/usr/src/sys/RIMx64 amd64 > > Cha?ha patch: > http://netlab.linkpc.net/download/software/FreeBSD/patches/chacha.patch What's the difference between CHACHA and XCHACHA? Also, where are the man page diffs? They might have explained the difference between the two, and explained why two versions of chacha are needed... Is there a reason you decided to write your own ChaCha implementation instead of using one of the standard ones? Did you run performance tests between your implementation and others? > HW: Core Duo E8500, 8Gb DDR2-800. > dd if=/dev/zero of=/dev/md0 bs=1m > 2148489421 bytes/sec > > > # sector = 512b > 3DES-CBC-192 = 20773120 bytes/sec > AES-CBC-128 = 85276853 bytes/sec > AES-CBC-256 = 68893016 bytes/sec > AES-XTS-128 = 68194868 bytes/sec > AES-XTS-256 = 56611573 bytes/sec > Blowfish-CBC-128 = 11169657 bytes/sec > Blowfish-CBC-256 = 11185891 bytes/sec > Camellia-CBC-128 = 78077243 bytes/sec > Camellia-CBC-256 = 65732219 bytes/sec > ChaCha8-XTS-256 = 258042765 bytes/sec > ChaCha12-XTS-256 = 223616967 bytes/sec > ChaCha20-XTS-256 = 176005366 bytes/sec > XChaCha8-XTS-256 = 228292624 bytes/sec > XChaCha12-XTS-256 = 195577624 bytes/sec > XChaCha20-XTS-256 = 152247267 bytes/sec > XChaCha20-XTS-128 = 152717737 bytes/sec ! 128 bit key have same speed as 256 > > > # sector = 4kb > 3DES-CBC-192 = 22018189 bytes/sec > AES-CBC-128 = 104097143 bytes/sec > AES-CBC-256 = 81983833 bytes/sec > AES-XTS-128 = 78559346 bytes/sec > AES-XTS-256 = 66047200 bytes/sec > Blowfish-CBC-128 = 38635464 bytes/sec > Blowfish-CBC-256 = 38810555 bytes/sec > Camellia-CBC-128 = 92814510 bytes/sec > Camellia-CBC-256 = 75949489 bytes/sec > ChaCha8-XTS-256 = 337336982 bytes/sec > ChaCha12-XTS-256 = 284740187 bytes/sec > ChaCha20-XTS-256 = 217326865 bytes/sec > XChaCha8-XTS-256 = 328424551 bytes/sec > XChaCha12-XTS-256 = 278579692 bytes/sec > XChaCha20-XTS-256 = 211660225 bytes/sec > > Optimized AES-XTS - speed like AES-CBC: > AES-XTS-128 = 102841051 bytes/sec > AES-XTS-256 = 80813644 bytes/sec Is this from a different patch or what? Can you talk more about this? Thanks. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 12 09:25:01 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 84DDFE49; Mon, 12 Jan 2015 09:25:01 +0000 (UTC) Received: from mail.highsecure.ru (mail6.highsecure.ru [IPv6:2a01:4f8:190:43b5::99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 40A4AE57; Mon, 12 Jan 2015 09:25:01 +0000 (UTC) Received: from medway.cl.cam.ac.uk (medway.cl.cam.ac.uk [IPv6:2001:630:212:238:21c:c0ff:fe4b:2b85]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: vsevolod@highsecure.ru) by mail.highsecure.ru (Postfix) with ESMTPSA id 1A2A7300198; Mon, 12 Jan 2015 10:24:49 +0100 (CET) Message-ID: <54B392DB.4020304@highsecure.ru> Date: Mon, 12 Jan 2015 09:24:43 +0000 From: Vsevolod Stakhov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: John-Mark Gurney , rozhuk.im@gmail.com Subject: Re: ChaCha8/12/20 and GEOM ELI tests References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <20150112072249.GM1949@funkthat.com> In-Reply-To: <20150112072249.GM1949@funkthat.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=highsecure.ru; s=dkim; t=1421054689; bh=UOghiykLJPlmhG8MNwDQU/Qa1l4GP4xpse8r/eD32LE=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=Q5qfrceP+IJvQMFlvZdf6xYU6d769v2aoz0CAqszN0umCcOrceIsBe8EHzSWW5L0Yg34d+QL/RfVQUlq/QhI9ImdBq0+eX9/iRnErJaMNEQiMyH5RJ1cRx2BwBIgKlVQkPdBL10W6Hzsx/F5DfKqFwDVCnQEeOqG6+IaMEF0HjA= Cc: freebsd-hackers@freebsd.org, freebsd-geom@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jan 2015 09:25:01 -0000 On 12/01/15 07:22, John-Mark Gurney wrote: > rozhuk.im@gmail.com wrote this message on Mon, Jan 12, 2015 at 06:13 +0300: >> FreeBSD firewall 11.0-CURRENT FreeBSD 11.0-CURRENT #3 r276867M: Fri Jan 9 >> 09:34:39 MSK 2015 root@firewall:/usr/obj/usr/src/sys/RIMx64 amd64 >> >> Cha?ha patch: >> http://netlab.linkpc.net/download/software/FreeBSD/patches/chacha.patch > > What's the difference between CHACHA and XCHACHA? XChaha is the version of chacha that accepts longer nonce (namely 20 bytes instead of 8). This mode could be used in random nonces model, thought it is certainly slightly slower as it requires 1 more round per operation. But the difference is negligible on 4K blocks. -- Vsevolod Stakhov From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 12 16:42:43 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 328D0D10; Mon, 12 Jan 2015 16:42:43 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 08ACF6C4; Mon, 12 Jan 2015 16:42:43 +0000 (UTC) Received: from new-host-4.home (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id A8813B915; Mon, 12 Jan 2015 11:42:41 -0500 (EST) Message-ID: <54B3F980.6080402@FreeBSD.org> Date: Mon, 12 Jan 2015 11:42:40 -0500 From: John Baldwin User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Gavin Mu , Adrian Chadd Subject: Re: status of projects/numa? References: <35AF9553-9536-41B0-8C2C-D907ED8E8531@gmail.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 12 Jan 2015 11:42:41 -0500 (EST) Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jan 2015 16:42:43 -0000 On 12/23/14 8:11 PM, Gavin Mu wrote: > Hi, Adrian, > > Our impl is based on FreeBSD 7.x, and full of tricky, and we are moving to newer FreeBSD version. This is why I am asking for this, we would like to re-write and keep sync with FreeBSD community. We want to know the latest status and where we can start from. > > I can see that projects/numa and also Jeff's repo have more functions implemented, and one userland API proposal on github. There is no update for a long time for all. > > There is also the wiki page at https://wiki.freebsd.org/NUMA. I am not sure if it is outdated or not. The wiki page is a very recent thing (much more recent than projects/numa) and I hope it will serve as a task list for the current round of work. However, even if it is a bit of a simplistic overview. In my most recent talks with Jeff, some of the VM changes are quite a bit more involved than that page implies (e.g. the need to use multiple vmem's (one per domain) in the kmem layer). I've seen the rest of this thread, but I am curious what you have internally? I wrote the very simple first-touch bits that were committed to 9 that I used in production on 7.x and 8.x (and have since been replaced in 10.x and later), but it was very simple and did not expose anything API-wise. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 12 16:45:18 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9FA64F22 for ; Mon, 12 Jan 2015 16:45:18 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7892875B for ; Mon, 12 Jan 2015 16:45:18 +0000 (UTC) Received: from new-host-4.home (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 6BE67B915; Mon, 12 Jan 2015 11:45:17 -0500 (EST) Message-ID: <54B3FA1D.6020605@FreeBSD.org> Date: Mon, 12 Jan 2015 11:45:17 -0500 From: John Baldwin User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Robert Bonomi , freebsd-hackers@freebsd.org Subject: Re: passing 'config' file data to a kernel loadable module ?? References: <201412302113.sBULDhan082070@host203.r-bonomi.com> In-Reply-To: <201412302113.sBULDhan082070@host203.r-bonomi.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 12 Jan 2015 11:45:17 -0500 (EST) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jan 2015 16:45:18 -0000 On 12/30/14 4:13 PM, Robert Bonomi wrote: > > I'm hacking on an existing screen-saver module, and would like to set options > in the config file that are passed to the compiler when the module is built. > > I can't find a decent description of the gory details of HOW 'config' works. > Chapter 2 of the SMM, "Building 4.4BSD Kernels with Config" has falsehoods -- > to wit: the example on page 10, under the 'options' keyword, given as > 'options -DFUNNY,-DHAHA' > causes 'config' to abort with a fatal error. > Apparently an 'option' must be defined 'somewhere else', before config > recognizes it. Yes, in sys/conf/options*. > I've got an 'ugly' method -- "makeoptions CONF_CLFAGS+=-D{{NAME}}" -- that > works for compiling the '.o' that gets linked into the kernel executable, > BUT that make variable is not passed to the (separate!!) compile stage for > building a.ko module. Yes, the source-code is compiled _twice_, with the > resulting .o file in two separate locations, and the options passed to the > compiler are *NOT* identical in both instances -- they're 'close', but not > 'exactly the same'. Is there a good reason why they're different? You can use DEBUG with makeoptions as a similar hack. I think that might also work for modules? > Is there an 'approved' way for getting a supplemental 'definition' from the > kernel config file passed to the compile of a loadable module? > > Is there a 'less ugly' way than what I described above, to get such a > definition passed to compilation of a module that will be linked into the > kernel? > > Ideally, is there a single means/method that will accomplish both tasks? > > Wish-list: something that does _not_ require modifying other files in the > 'magic data' that config uses? That is not currently doable. :( > Peripherally related thereunto, is there a cookbook or other example of > making a 'port' for a kernel module, and what all needs to be in it? Do you need to have these settings be compile options vs loader tunables / sysctls? The latter are preferred if at all possible. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 12 19:57:48 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 565309BA for ; Mon, 12 Jan 2015 19:57:48 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1on0059.outbound.protection.outlook.com [157.56.110.59]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E9E93347 for ; Mon, 12 Jan 2015 19:57:47 +0000 (UTC) Received: from DM2PR0801MB0944.namprd08.prod.outlook.com (25.160.131.27) by DM2PR0801MB0942.namprd08.prod.outlook.com (25.160.131.25) with Microsoft SMTP Server (TLS) id 15.1.53.17; Mon, 12 Jan 2015 19:43:23 +0000 Received: from DM2PR0801MB0944.namprd08.prod.outlook.com ([25.160.131.27]) by DM2PR0801MB0944.namprd08.prod.outlook.com ([25.160.131.27]) with mapi id 15.01.0053.000; Mon, 12 Jan 2015 19:43:23 +0000 From: "Pokala, Ravi" To: "freebsd-hackers@freebsd.org" Subject: Re: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged Thread-Topic: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged Thread-Index: AQHQLqAF6jIVY6e0Rka/enbEqt4SMA== Date: Mon, 12 Jan 2015 19:43:22 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.7.141117 x-originating-ip: [64.80.217.3] authentication-results: spf=none (sender IP is ) smtp.mailfrom=rpokala@panasas.com; x-dmarcaction-test: None x-microsoft-antispam: BCL:0;PCL:0;RULEID:(3005003);SRVR:DM2PR0801MB0942; x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:; SRVR:DM2PR0801MB0942; x-forefront-prvs: 0454444834 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(51704005)(13464003)(377424004)(24454002)(164054003)(199003)(189002)(68736005)(2900100001)(122556002)(64706001)(40100003)(102836002)(62966003)(77156002)(450100001)(99286002)(110136001)(101416001)(66066001)(107886001)(2351001)(106116001)(105586002)(83506001)(86362001)(19580395003)(19580405001)(106356001)(54356999)(50986999)(92566002)(46102003)(2656002)(512944002)(87936001)(97736003)(36756003); DIR:OUT; SFP:1101; SCL:1; SRVR:DM2PR0801MB0942; H:DM2PR0801MB0944.namprd08.prod.outlook.com; FPR:; SPF:None; MLV:sfv; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: panasas.com does not designate permitted sender hosts) Content-Type: text/plain; charset="us-ascii" Content-ID: <039C518E66EF3A42B2C12380A5170966@namprd08.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: panasas.com X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Jan 2015 19:43:22.2759 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: acf01c9d-c699-42af-bdbb-44bf582e60b0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM2PR0801MB0942 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jan 2015 19:57:48 -0000 -----Original Message----- >Date: Mon, 12 Jan 2015 00:27:18 -0500 >From: Allan Jude >To: freebsd-hackers@freebsd.org >Subject: Re: [PATCH] Display progress during getmemsize() so the > kernel doesn't look like it hanged >Message-ID: <54B35B36.4040504@freebsd.org> >Content-Type: text/plain; charset=3D"windows-1252" > >On 2015-01-12 00:24, Pokala, Ravi wrote: >> Hi folks, >>=20 >> Several of us have noticed that there's a long pause at the start of >>booting the kernel on amd64 systems with large amounts of RAM. During >>This pause, the kernel is mapping in the memory ranges, but does not >>emit any progress indicators. Because this can take quite a while, it >>can look like the kernel has hung. I filed PR 196650 about this issue; >>this patch just prints out a dot for every GB that's mapped in. We've >>been using this patch for years at Panasas, and I'm hoping someone can >>submit it for me. >>=20 >> Thanks, >>=20 >> Ravi >>=20 >>=20 >> >>... =20 >>=20 >>=20 >> >Which version(s) of FreeBSD were you seeing this delay on? We first noted it with 7.2, because that's what we used on our first amd64 systems with > 4GB RAM. We're still seeing it with 10.1. >I know older versions had it, quite badly. > >setting hw.memtest.tests=3D0 in /boot/loader.conf makes most of the delay >go away, and the default on -CURRENT is now 0 instead of 1. > >IIRC, in 10, it defaults to 1, but is switched to 0 in the case of >virtual machines, because touching every page of memory ruined memory >over-commit type features. > >Is this feature still useful with memtest.tests=3D0? You're right; when I set hw.memtest.tests=3D0, the pause goes away in 10.1. I thought it defaulted to 0 in 10.1. So that changes this slightly - could we either change the default in 10-STABLE, or commit the following patch against 10-STABLE? Thanks, Ravi Index: sys/amd64/amd64/machdep.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D diff --git a/stable/10/sys/amd64/amd64/machdep.c b/stable/10/sys/amd64/amd64/machdep.c --- a/stable/10/sys/amd64/amd64/machdep.c (revision 277085) +++ b/stable/10/sys/amd64/amd64/machdep.c (working copy) @@ -1541,6 +1541,9 @@ struct bios_smap *smapbase; struct efi_map_header *efihdr; quad_t dcons_addr, dcons_size; + u_int32_t page_counter; + int PAGES_PER_MB =3D ((1024 * 1024) / PAGE_SIZE); + int PAGES_PER_GB =3D (PAGES_PER_MB * 1024); =20 bzero(physmap, sizeof(physmap)); basemem =3D 0; @@ -1651,6 +1654,8 @@ * physmap is in bytes, so when converting to page boundaries, * round up the start address and round down the end address. */ + printf("Mapping system memory"); + page_counter =3D 0; for (i =3D 0; i <=3D physmap_idx; i +=3D 2) { vm_paddr_t end; =20 @@ -1661,6 +1666,14 @@ int tmp, page_bad, full; int *ptr =3D (int *)CADDR1; =20 + /* + * Print a "." every GB, to show we're making progress + */ + page_counter++; + if ((page_counter % PAGES_PER_GB) =3D=3D 0) { + printf("."); + } + full =3D FALSE; /* * block out kernel memory as not available. @@ -1765,6 +1778,9 @@ break; } } + printf("\nMapped %d GB + %d MB total\n", + (page_counter / PAGES_PER_GB), + ((page_counter % PAGES_PER_GB) / PAGES_PER_MB)); *pte =3D 0; invltlb(); =20 From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 12 20:40:40 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EE7D0AEC; Mon, 12 Jan 2015 20:40:40 +0000 (UTC) Received: from mail-la0-x22a.google.com (mail-la0-x22a.google.com [IPv6:2a00:1450:4010:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 695E5A36; Mon, 12 Jan 2015 20:40:40 +0000 (UTC) Received: by mail-la0-f42.google.com with SMTP id gd6so26329159lab.1; Mon, 12 Jan 2015 12:40:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:reply-to:to:cc:references:in-reply-to:subject:date :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=HqgBRkoe3+YQFAf/BWroAktJ5yD5hXGh5YEtIOgvcTs=; b=Umx82QVDgjou4o87/tqalUfkl76pUfdSycQYZYdHl0PUa0lq99ruCMB1GD0lprpJrw 6LhfIJ4o7whQOFRqKZ+jQ5frn7hU+CzACEa7OclFpVjGq3GVyWmWMG6ezfxpq+4iotDq Phi3Oou/grSgBq+Ly9b2Zob9O/17+lxodPltDzQw8ARfuILoNCEwfzDkxj1dvAcCsgP2 ZuLTC/U6iwOtD+enL7YS98Wxhhf0oxyF7ZQaaFIcbxDkA1SSB2sPYKq4NTeMsEOxjPef UtHyPzX1OCEmY+izc4THKHex/NVZS4FH+Rvk/+3rApJAjlW1j4XV0zz0on4bx0oP0uz7 EJZQ== X-Received: by 10.152.18.135 with SMTP id w7mr37788821lad.47.1421095237965; Mon, 12 Jan 2015 12:40:37 -0800 (PST) Received: from rimwks1w7x64 ([2001:470:1f15:8e:b007:2759:7397:9491]) by mx.google.com with ESMTPSA id o13sm271111laa.16.2015.01.12.12.40.35 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 12 Jan 2015 12:40:36 -0800 (PST) Message-ID: <54b43144.2d08980a.437b.0f8f@mx.google.com> X-Google-Original-Message-ID: <018e01d02ea8$04f7b370$0ee71a50$@IM@gmail.com> From: rozhuk.im@gmail.com X-Google-Original-From: Reply-To: To: "'John-Mark Gurney'" References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <20150112072249.GM1949@funkthat.com> In-Reply-To: <20150112072249.GM1949@funkthat.com> Subject: RE: ChaCha8/12/20 and GEOM ELI tests Date: Mon, 12 Jan 2015 23:40:34 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdAuOJeW3qYugAzIRtOIqcPClX0Q6gABULag Content-Language: ru Cc: freebsd-hackers@freebsd.org, freebsd-geom@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jan 2015 20:40:41 -0000 > > Cha?ha patch: > > > http://netlab.linkpc.net/download/software/FreeBSD/patches/chacha.patch > > What's the difference between CHACHA and XCHACHA? Same as between SALSA and XSALSA. XChaCha20 uses a 256-bit key as well as the first 128 bits of the nonce in order to compute a subkey. This subkey, as well as the remaining 64 bits of the nonce, are the parameters of the ChaCha20 function used to actually generate the stream. But with XChaCha20's longer nonce, it is safe to generate nonces using randombytes_buf() for every message encrypted with the same key without having to worry about a collision. More details: http://cr.yp.to/snuffle/xsalsa-20081128.pdf > Also, where are the man page diffs? They might have explained the > difference between the two, and explained why two versions of chacha > are needed... No man page diffs. Man pages does not explain difference between AES-CBC and AES-XTS... > Is there a reason you decided to write your own ChaCha implementation > instead of using one of the standard ones? Did you run performance > tests between your implementation and others? Reference ChaCha and reference (FreeBSD) XTS (4k sector): ChaCha8-XTS-256 = 199518722 bytes/sec ChaCha12-XTS-256 = 179029849 bytes/sec ChaCha20-XTS-256 = 149447317 bytes/sec XChaCha8-XTS-256 = 195675728 bytes/sec XChaCha12-XTS-256 = 175790196 bytes/sec XChaCha20-XTS-256 = 147939263 bytes/sec This is the reference version adapted for use in /dev/crypto. chacha_block_unaligneg() - processing the reference version of a data block. Macros are used for readability. chacha_block_aligned() - the same but the work on the aligned data. To increase speed, instead of one byte is processed for 4/8 byte times. The data in the context of an 8-byte aligned. To increase security, all data, including temporary, saved in a context that on completion of the work is filled with zeros. > > HW: Core Duo E8500, 8Gb DDR2-800. > > dd if=/dev/zero of=/dev/md0 bs=1m > > 2148489421 bytes/sec > > > > > > # sector = 512b > > 3DES-CBC-192 = 20773120 bytes/sec > > AES-CBC-128 = 85276853 bytes/sec > > AES-CBC-256 = 68893016 bytes/sec > > AES-XTS-128 = 68194868 bytes/sec > > AES-XTS-256 = 56611573 bytes/sec > > Blowfish-CBC-128 = 11169657 bytes/sec > > Blowfish-CBC-256 = 11185891 bytes/sec > > Camellia-CBC-128 = 78077243 bytes/sec > > Camellia-CBC-256 = 65732219 bytes/sec > > ChaCha8-XTS-256 = 258042765 bytes/sec > > ChaCha12-XTS-256 = 223616967 bytes/sec > > ChaCha20-XTS-256 = 176005366 bytes/sec > > XChaCha8-XTS-256 = 228292624 bytes/sec > > XChaCha12-XTS-256 = 195577624 bytes/sec > > XChaCha20-XTS-256 = 152247267 bytes/sec > > XChaCha20-XTS-128 = 152717737 bytes/sec ! 128 bit key have same speed > > as 256 > > > > > > # sector = 4kb > > 3DES-CBC-192 = 22018189 bytes/sec > > AES-CBC-128 = 104097143 bytes/sec > > AES-CBC-256 = 81983833 bytes/sec > > AES-XTS-128 = 78559346 bytes/sec > > AES-XTS-256 = 66047200 bytes/sec > > Blowfish-CBC-128 = 38635464 bytes/sec > > Blowfish-CBC-256 = 38810555 bytes/sec > > Camellia-CBC-128 = 92814510 bytes/sec > > Camellia-CBC-256 = 75949489 bytes/sec > > ChaCha8-XTS-256 = 337336982 bytes/sec > > ChaCha12-XTS-256 = 284740187 bytes/sec > > ChaCha20-XTS-256 = 217326865 bytes/sec > > XChaCha8-XTS-256 = 328424551 bytes/sec > > XChaCha12-XTS-256 = 278579692 bytes/sec > > XChaCha20-XTS-256 = 211660225 bytes/sec > > > > Optimized AES-XTS - speed like AES-CBC: > > AES-XTS-128 = 102841051 bytes/sec > > AES-XTS-256 = 80813644 bytes/sec > > Is this from a different patch or what? Can you talk more about this? No patch at this moment. After optimization ChaCha-XTS I applied these optimizations to the AES-XTS and get this result. All changes were aes_xts_reinit() and aes_xts_crypt(), just slightly changed the structure aes_xts_ctx. aes_xts_ctx: u_int8_t tweak[] -> u_int64_t tweak[] aes_xts_reinit -> same as chacha_xts_reinit() aes_xts_crypt -> same as chacha_xts_crypt(): block[] - temp buf removed; xor 1 byte -> xor 8 bytes at once; tweak[i] << 1: rotl 1 bit: 1 byte -> 8 bytes; unroll loops; Final: struct aes_xts_ctx { rijndael_ctx key1; rijndael_ctx key2; uint64_t tweak[(AES_XTS_BLOCKSIZE / sizeof(uint64_t))]; }; void aes_xts_reinit(caddr_t key, u_int8_t *iv) { struct aes_xts_ctx *ctx = (struct aes_xts_ctx *)key; /* * Prepare tweak as E_k2(IV). IV is specified as LE representation * of a 64-bit block number which we allow to be passed in directly. */ if (ALIGNED_POINTER(iv, uint64_t)) { ctx->tweak[0] = (*((uint64_t*)(void*)iv)); } else { bcopy(iv, ctx->tweak, sizeof(uint64_t)); } /* Convert to LE. */ ctx->tweak[0] = htole64(ctx->tweak[0]); /* Last 64 bits of IV are always zero */ ctx->tweak[1] = 0; rijndael_encrypt(&ctx->key2, (uint8_t*)ctx->tweak, (uint8_t*)ctx->tweak); } static void aes_xts_crypt(struct aes_xts_ctx *ctx, u_int8_t *data, u_int do_encrypt) { size_t i; uint64_t crr, tm; if (ALIGNED_POINTER(blk, uint64_t)) { ((uint64_t*)(void*)data)[0] ^= ctx->tweak[0]; ((uint64_t*)(void*)data)[1] ^= ctx->tweak[1]; } else { for (i = 0; i < AES_XTS_BLOCKSIZE; i ++) data[i] ^= ((uint8_t*)ctx->tweak)[i]; } if (do_encrypt) rijndael_encrypt(&ctx->key1, data, data); else rijndael_decrypt(&ctx->key1, data, data); if (ALIGNED_POINTER(blk, uint64_t)) { ((uint64_t*)(void*)data)[0] ^= ctx->tweak[0]; ((uint64_t*)(void*)data)[1] ^= ctx->tweak[1]; } else { for (i = 0; i < AES_XTS_BLOCKSIZE; i ++) data[i] ^= ((uint8_t*)ctx->tweak)[i]; } /* Exponentiate tweak */ crr = (ctx->tweak[0] >> ((sizeof(uint64_t) * 8) - 1)); ctx->tweak[0] = (ctx->tweak[0] << 1); tm = ctx->tweak[1]; ctx->tweak[1] = ((tm << 1) | crr); crr = (tm >> ((sizeof(uint64_t) * 8) - 1)); if (crr) ctx->tweak[0] ^= 0x87; /* GF(2^128) generator polynomial. */ } From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 12 20:51:29 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 56BF3FE8; Mon, 12 Jan 2015 20:51:29 +0000 (UTC) Received: from mail-ig0-x233.google.com (mail-ig0-x233.google.com [IPv6:2607:f8b0:4001:c05::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1B88EC02; Mon, 12 Jan 2015 20:51:29 +0000 (UTC) Received: by mail-ig0-f179.google.com with SMTP id l13so463340iga.0; Mon, 12 Jan 2015 12:51:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=b6Er2ed7CRZrQJ9MwlBgtrW6o0pwGk4pZ6UuxNPbCZM=; b=Q14mkiZoIrtxBGto/81IZn5wFrnU3SX97lKU7zUx4z9kpXalJ1uWdbSNo6QF22ELfg T6hqj9Ukz3OU3v1r/WXiwxeGlMDmSpiO9tfZVAzcMzvJiKQY3B8OY2gnZewNbWXCqYUO ADLk9tjpjQuh8hhDiWSabfq14tsIowGpRMz6V10xNrmY87/O3ZfS8MxH5yr6hkzbuvyT yYNpJvofXQQpUCqqY4+6YjdQfGpMaVZnD7zmzpTpBUd/28yZ7Z/OsPovDfavvZ72yrM8 F9AS2+YqC1KcgutEHxeVTt82zJRRdQEk46gaPXqgYvuyKXGjwUD814PgJKh/IzfEbQZx Eizw== X-Received: by 10.42.78.137 with SMTP id n9mr5914136ick.53.1421095888635; Mon, 12 Jan 2015 12:51:28 -0800 (PST) MIME-Version: 1.0 Sender: carpeddiem@gmail.com Received: by 10.107.0.85 with HTTP; Mon, 12 Jan 2015 12:51:08 -0800 (PST) In-Reply-To: <20150105135856.GA65607@gate> References: <20150101192219.GA46601@voyager> <20150102072620.GB65505@gate> <9F362787-EC79-4ADC-B414-4EDBF592B868@gmail.com> <20150105135856.GA65607@gate> From: Ed Maste Date: Mon, 12 Jan 2015 15:51:08 -0500 X-Google-Sender-Auth: kyOvyXYEHjvtHdmF-K5pe7rMAo4 Message-ID: Subject: Re: [PATCH] Fixing panic in vt_fb_blank() if fb_size is not a multiple of fb_stride To: Andre Albsmeier Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" , Adrian Chadd , "freebsd-x11@freebsd.org" , Garrett Cooper X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jan 2015 20:51:29 -0000 On 5 January 2015 at 08:58, Andre Albsmeier wrote: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=196510 Thanks, this should now be fixed in stable/9 with r277083. From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 12 23:34:14 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2C5BE151; Mon, 12 Jan 2015 23:34:14 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 00AB7F11; Mon, 12 Jan 2015 23:34:13 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t0CNYCw0016538 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 12 Jan 2015 15:34:12 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t0CNYBAF016537; Mon, 12 Jan 2015 15:34:11 -0800 (PST) (envelope-from jmg) Date: Mon, 12 Jan 2015 15:34:11 -0800 From: John-Mark Gurney To: rozhuk.im@gmail.com Subject: Re: ChaCha8/12/20 and GEOM ELI tests Message-ID: <20150112233411.GP1949@funkthat.com> References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <20150112072249.GM1949@funkthat.com> <54b43144.2d08980a.437b.0f8f@mx.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54b43144.2d08980a.437b.0f8f@mx.google.com> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Mon, 12 Jan 2015 15:34:12 -0800 (PST) Cc: freebsd-hackers@freebsd.org, freebsd-geom@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jan 2015 23:34:14 -0000 rozhuk.im@gmail.com wrote this message on Mon, Jan 12, 2015 at 23:40 +0300: > > > Cha?ha patch: > > > > > http://netlab.linkpc.net/download/software/FreeBSD/patches/chacha.patch > > > > What's the difference between CHACHA and XCHACHA? > > Same as between SALSA and XSALSA. > > XChaCha20 uses a 256-bit key as well as the first 128 bits of the nonce in > order to compute a subkey. This subkey, as well as the remaining 64 bits of > the nonce, are the parameters of the ChaCha20 function used to actually > generate the stream. > > But with XChaCha20's longer nonce, it is safe to generate nonces using > randombytes_buf() for every message encrypted with the same key without > having to worry about a collision. > > More details: http://cr.yp.to/snuffle/xsalsa-20081128.pdf Ahh, thanks.. > > Also, where are the man page diffs? They might have explained the > > difference between the two, and explained why two versions of chacha > > are needed... > > No man page diffs. You need to document the new defines in crypto(9), and document the various parameters in crypto(7)... Yes, not all modes are documented in crypto(7), but going forward, at a minimum we need to document new additions... I'll admit I didn't document the other algorithms as I'm not as familar w/ those as the ones that I worked one... > Man pages does not explain difference between AES-CBC and AES-XTS... True, but CBC and XTS (which includes a reference to the standard) are a lot more searchable/common knowlege than xchacha.. google thinks you mean chacha, and xchacha just turns up a bunch of people on various networks... Not until you search on xchacha crypto do you get a relevant page... Also, wikipedia doesn't have an entry for xchacha, nor does the chacha (cipher) page list it... So, when documenting xchacha in crypto(7), include a link to the description/standard... > > Is there a reason you decided to write your own ChaCha implementation > > instead of using one of the standard ones? Did you run performance > > tests between your implementation and others? > > Reference ChaCha and reference (FreeBSD) XTS (4k sector): > ChaCha8-XTS-256 = 199518722 bytes/sec > ChaCha12-XTS-256 = 179029849 bytes/sec > ChaCha20-XTS-256 = 149447317 bytes/sec > XChaCha8-XTS-256 = 195675728 bytes/sec > XChaCha12-XTS-256 = 175790196 bytes/sec > XChaCha20-XTS-256 = 147939263 bytes/sec So, you're seeing a 33%-50% improvement, good to hear... Also, do you publish this implementation somewhere? If so, it'd be helpful to include a url to where up to date versions can be obtained... If you don't plan on publishing/maintaining it outside of FreeBSD, then we need to unifdef out the Windows parts of it for our tree... > This is the reference version adapted for use in /dev/crypto. > chacha_block_unaligneg() - processing the reference version of a data block. > Macros are used for readability. > chacha_block_aligned() - the same but the work on the aligned data. Please use the macro __NO_STRICT_ALIGNMENT to decide if special work is necessary to handle the alignment... What is the CHACHA_X64 macro for? If that is to detect LP64 platforms, please use the macro __LP64__ to decide this... Have you done performance evaluations on 32bit arches to make sure double rounds aren't a benefit there too? Use the byteorder(9) macros to encode/decode integers instead of rolling your own (U8TO32_LITTLE and U32TO8_LITTLE)... Turns out compilers aren't good at optimizing this type of code, and platforms may have assembly optimized versions for these... > To increase speed, instead of one byte is processed for 4/8 byte times. > The data in the context of an 8-byte aligned. > To increase security, all data, including temporary, saved in a context that > on completion of the work is filled with zeros. Please use the function explicite_bzero that is available for all of these instead of creating your own.. > > > HW: Core Duo E8500, 8Gb DDR2-800. > > > dd if=/dev/zero of=/dev/md0 bs=1m > > > 2148489421 bytes/sec > > > > > > > > > # sector = 512b > > > 3DES-CBC-192 = 20773120 bytes/sec > > > AES-CBC-128 = 85276853 bytes/sec > > > AES-CBC-256 = 68893016 bytes/sec > > > AES-XTS-128 = 68194868 bytes/sec > > > AES-XTS-256 = 56611573 bytes/sec > > > Blowfish-CBC-128 = 11169657 bytes/sec > > > Blowfish-CBC-256 = 11185891 bytes/sec > > > Camellia-CBC-128 = 78077243 bytes/sec > > > Camellia-CBC-256 = 65732219 bytes/sec > > > ChaCha8-XTS-256 = 258042765 bytes/sec > > > ChaCha12-XTS-256 = 223616967 bytes/sec > > > ChaCha20-XTS-256 = 176005366 bytes/sec > > > XChaCha8-XTS-256 = 228292624 bytes/sec > > > XChaCha12-XTS-256 = 195577624 bytes/sec > > > XChaCha20-XTS-256 = 152247267 bytes/sec > > > XChaCha20-XTS-128 = 152717737 bytes/sec ! 128 bit key have same speed > > > as 256 > > > > > > > > > # sector = 4kb > > > 3DES-CBC-192 = 22018189 bytes/sec > > > AES-CBC-128 = 104097143 bytes/sec > > > AES-CBC-256 = 81983833 bytes/sec > > > AES-XTS-128 = 78559346 bytes/sec > > > AES-XTS-256 = 66047200 bytes/sec > > > Blowfish-CBC-128 = 38635464 bytes/sec > > > Blowfish-CBC-256 = 38810555 bytes/sec > > > Camellia-CBC-128 = 92814510 bytes/sec > > > Camellia-CBC-256 = 75949489 bytes/sec > > > ChaCha8-XTS-256 = 337336982 bytes/sec > > > ChaCha12-XTS-256 = 284740187 bytes/sec > > > ChaCha20-XTS-256 = 217326865 bytes/sec > > > XChaCha8-XTS-256 = 328424551 bytes/sec > > > XChaCha12-XTS-256 = 278579692 bytes/sec > > > XChaCha20-XTS-256 = 211660225 bytes/sec > > > > > > Optimized AES-XTS - speed like AES-CBC: > > > AES-XTS-128 = 102841051 bytes/sec > > > AES-XTS-256 = 80813644 bytes/sec > > > > Is this from a different patch or what? Can you talk more about this? > > No patch at this moment. > After optimization ChaCha-XTS I applied these optimizations to the AES-XTS > and get this result. > All changes were aes_xts_reinit() and aes_xts_crypt(), just slightly changed > the structure aes_xts_ctx. > > aes_xts_ctx: > u_int8_t tweak[] -> u_int64_t tweak[] > > aes_xts_reinit -> same as chacha_xts_reinit() > > aes_xts_crypt -> same as chacha_xts_crypt(): > block[] - temp buf removed; > xor 1 byte -> xor 8 bytes at once; > tweak[i] << 1: rotl 1 bit: 1 byte -> 8 bytes; > unroll loops; Ahh, I thought I had done some similar optimizations, but I only did them to the aesni version of the routines... You should use the macro above to decide if things are aligned or not... > > Final: > > struct aes_xts_ctx { > rijndael_ctx key1; > rijndael_ctx key2; > uint64_t tweak[(AES_XTS_BLOCKSIZE / sizeof(uint64_t))]; > }; > > void > aes_xts_reinit(caddr_t key, u_int8_t *iv) > { > struct aes_xts_ctx *ctx = (struct aes_xts_ctx *)key; > > /* > * Prepare tweak as E_k2(IV). IV is specified as LE representation > * of a 64-bit block number which we allow to be passed in directly. > */ > if (ALIGNED_POINTER(iv, uint64_t)) { > ctx->tweak[0] = (*((uint64_t*)(void*)iv)); > } else { > bcopy(iv, ctx->tweak, sizeof(uint64_t)); > } > /* Convert to LE. */ > ctx->tweak[0] = htole64(ctx->tweak[0]); Hmm... this line bothers me.. I'll need to spend more time reading up to decide if it is buggy or not... Is ctx->tweak in host order? or LE order? I believe it's suppose to be LE order, as it gets passed directly to _encryt.. I'm also not sure if the original code is BE clean, which is part of my problem... > /* Last 64 bits of IV are always zero */ > ctx->tweak[1] = 0; > > rijndael_encrypt(&ctx->key2, (uint8_t*)ctx->tweak, > (uint8_t*)ctx->tweak); > } > > static void > aes_xts_crypt(struct aes_xts_ctx *ctx, u_int8_t *data, u_int do_encrypt) > { > size_t i; > uint64_t crr, tm; > > if (ALIGNED_POINTER(blk, uint64_t)) { > ((uint64_t*)(void*)data)[0] ^= ctx->tweak[0]; > ((uint64_t*)(void*)data)[1] ^= ctx->tweak[1]; > } else { > for (i = 0; i < AES_XTS_BLOCKSIZE; i ++) > data[i] ^= ((uint8_t*)ctx->tweak)[i]; > } > > if (do_encrypt) > rijndael_encrypt(&ctx->key1, data, data); > else > rijndael_decrypt(&ctx->key1, data, data); > > if (ALIGNED_POINTER(blk, uint64_t)) { > ((uint64_t*)(void*)data)[0] ^= ctx->tweak[0]; > ((uint64_t*)(void*)data)[1] ^= ctx->tweak[1]; > } else { > for (i = 0; i < AES_XTS_BLOCKSIZE; i ++) > data[i] ^= ((uint8_t*)ctx->tweak)[i]; > } > > /* Exponentiate tweak */ > crr = (ctx->tweak[0] >> ((sizeof(uint64_t) * 8) - 1)); > ctx->tweak[0] = (ctx->tweak[0] << 1); > > tm = ctx->tweak[1]; > ctx->tweak[1] = ((tm << 1) | crr); > crr = (tm >> ((sizeof(uint64_t) * 8) - 1)); > > if (crr) > ctx->tweak[0] ^= 0x87; /* GF(2^128) generator polynomial. */ Please use the AES_XTS_ALPHA define instead of hardcoding the value.. Thanks. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 13 00:11:16 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 140D7ABA for ; Tue, 13 Jan 2015 00:11:16 +0000 (UTC) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.pix.net", Issuer "Pix.Com Technologies, LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B34142E0 for ; Tue, 13 Jan 2015 00:11:15 +0000 (UTC) Received: from torb.pix.net (torb.pix.net [IPv6:2001:470:e254:10:12dd:b1ff:febf:eca9]) (authenticated bits=0) by hydra.pix.net (8.15.1/8.15.1) with ESMTPA id t0D0BDRs084686; Mon, 12 Jan 2015 19:11:13 -0500 (EST) (envelope-from lidl@pix.net) Message-ID: <54B462A0.9080209@pix.net> Date: Mon, 12 Jan 2015 19:11:12 -0500 From: Kurt Lidl User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freeBSD-Hackers Subject: Import of xz 5.2? Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jan 2015 00:11:16 -0000 Are there planes to import the xz library version 5.2 (now that it has hit "stable") status? This opens the door to multi-threaded compression, which would be very nice to have. Thanks! -Kurt From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 13 03:40:23 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5FF2D1DD; Tue, 13 Jan 2015 03:40:23 +0000 (UTC) Received: from mail-qg0-x232.google.com (mail-qg0-x232.google.com [IPv6:2607:f8b0:400d:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0EAE2B96; Tue, 13 Jan 2015 03:40:23 +0000 (UTC) Received: by mail-qg0-f50.google.com with SMTP id z60so533874qgd.9; Mon, 12 Jan 2015 19:40:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :message-id:references:to; bh=PzzdCPv9Hx5qNsK2+XXEWq5CxfoyMWM+0BOrDAFC33c=; b=Hl3KgKIpwCb3LR/21Sz2DF09qcBg1tAEQvw+8LrnYjAGnyxjRgN0FkV13qglaBcAGD Boel1OnzCEnSOx9Gt0g5/J4EVTZvm3uuNGbgMRMHl0npOlOS4UV0RAmuZaZYlwHhcqt0 BC6oUDBY69L7pvJFA+xV7ZnDUA4MveLTXvBJwVa/wmgiwoZ/zSow+6h4d76tOCl0iWXA Ovdx35wz7aFZHm1i2yXaey6ScOSK47YDLyIlxstKR0gEYtnENbfDoMhBp5efcmayC29i T8ZAo1WVeooa3i70GuSB0dVDy/n5JYnRC1o3lQNnVgUnbCa0Kw6jGvjqgOrdbLeIsXTm 50ow== X-Received: by 10.140.34.204 with SMTP id l70mr53078358qgl.55.1421120422148; Mon, 12 Jan 2015 19:40:22 -0800 (PST) Received: from [10.0.0.155] (c-50-131-147-141.hsd1.ca.comcast.net. [50.131.147.141]) by mx.google.com with ESMTPSA id u1sm1276455qap.11.2015.01.12.19.40.21 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 12 Jan 2015 19:40:21 -0800 (PST) Subject: Re: ChaCha8/12/20 and GEOM ELI tests Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Content-Type: multipart/signed; boundary="Apple-Mail=_43A9D6FE-F5F9-4CCC-B6A3-B8B5171B44D8"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5b4 From: Alexey Ivanov In-Reply-To: <20150112233411.GP1949@funkthat.com> Date: Mon, 12 Jan 2015 19:40:13 -0800 Message-Id: <7A712B22-1151-4A80-970A-36C0C2A63653@gmail.com> References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <20150112072249.GM1949@funkthat.com> <54b43144.2d08980a.437b.0f8f@mx.google.com> <20150112233411.GP1949@funkthat.com> To: rozhuk.im@gmail.com, John-Mark Gurney X-Mailer: Apple Mail (2.1993) Cc: freebsd-hackers@freebsd.org, freebsd-geom@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jan 2015 03:40:23 -0000 --Apple-Mail=_43A9D6FE-F5F9-4CCC-B6A3-B8B5171B44D8 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Just curious: why does a stream cipher use mode of operation (e.g. XTS)? > On Jan 12, 2015, at 3:34 PM, John-Mark Gurney = wrote: >=20 > rozhuk.im@gmail.com wrote this message on Mon, Jan 12, 2015 at 23:40 = +0300: >>>> Cha?ha patch: >>>>=20 >>> = http://netlab.linkpc.net/download/software/FreeBSD/patches/chacha.patch >>>=20 >>> What's the difference between CHACHA and XCHACHA? >>=20 >> Same as between SALSA and XSALSA. >>=20 >> XChaCha20 uses a 256-bit key as well as the first 128 bits of the = nonce in >> order to compute a subkey. This subkey, as well as the remaining 64 = bits of >> the nonce, are the parameters of the ChaCha20 function used to = actually >> generate the stream. >>=20 >> But with XChaCha20's longer nonce, it is safe to generate nonces = using >> randombytes_buf() for every message encrypted with the same key = without >> having to worry about a collision. >>=20 >> More details: http://cr.yp.to/snuffle/xsalsa-20081128.pdf >=20 > Ahh, thanks.. >=20 >>> Also, where are the man page diffs? They might have explained the >>> difference between the two, and explained why two versions of chacha >>> are needed... >>=20 >> No man page diffs. >=20 > You need to document the new defines in crypto(9), and document the > various parameters in crypto(7)... Yes, not all modes are documented > in crypto(7), but going forward, at a minimum we need to document new > additions... >=20 > I'll admit I didn't document the other algorithms as I'm not as = familar > w/ those as the ones that I worked one... >=20 >> Man pages does not explain difference between AES-CBC and AES-XTS... >=20 > True, but CBC and XTS (which includes a reference to the standard) are > a lot more searchable/common knowlege than xchacha.. google thinks = you > mean chacha, and xchacha just turns up a bunch of people on various > networks... Not until you search on xchacha crypto do you get a = relevant > page... Also, wikipedia doesn't have an entry for xchacha, nor does > the chacha (cipher) page list it... So, when documenting xchacha in > crypto(7), include a link to the description/standard... >=20 >>> Is there a reason you decided to write your own ChaCha = implementation >>> instead of using one of the standard ones? Did you run performance >>> tests between your implementation and others? >>=20 >> Reference ChaCha and reference (FreeBSD) XTS (4k sector): >> ChaCha8-XTS-256 =3D 199518722 bytes/sec >> ChaCha12-XTS-256 =3D 179029849 bytes/sec >> ChaCha20-XTS-256 =3D 149447317 bytes/sec >> XChaCha8-XTS-256 =3D 195675728 bytes/sec >> XChaCha12-XTS-256 =3D 175790196 bytes/sec >> XChaCha20-XTS-256 =3D 147939263 bytes/sec >=20 > So, you're seeing a 33%-50% improvement, good to hear... >=20 > Also, do you publish this implementation somewhere? If so, it'd be > helpful to include a url to where up to date versions can be = obtained... > If you don't plan on publishing/maintaining it outside of FreeBSD, = then > we need to unifdef out the Windows parts of it for our tree... >=20 >> This is the reference version adapted for use in /dev/crypto. >> chacha_block_unaligneg() - processing the reference version of a data = block. >> Macros are used for readability. >> chacha_block_aligned() - the same but the work on the aligned data. >=20 > Please use the macro __NO_STRICT_ALIGNMENT to decide if special work > is necessary to handle the alignment... >=20 > What is the CHACHA_X64 macro for? If that is to detect LP64 = platforms, > please use the macro __LP64__ to decide this... Have you done > performance evaluations on 32bit arches to make sure double rounds = aren't > a benefit there too? >=20 > Use the byteorder(9) macros to encode/decode integers instead of = rolling > your own (U8TO32_LITTLE and U32TO8_LITTLE)... Turns out compilers = aren't > good at optimizing this type of code, and platforms may have assembly > optimized versions for these... >=20 >> To increase speed, instead of one byte is processed for 4/8 byte = times. >> The data in the context of an 8-byte aligned. >> To increase security, all data, including temporary, saved in a = context that >> on completion of the work is filled with zeros. >=20 > Please use the function explicite_bzero that is available for all of > these instead of creating your own.. >=20 >>>> HW: Core Duo E8500, 8Gb DDR2-800. >>>> dd if=3D/dev/zero of=3D/dev/md0 bs=3D1m >>>> 2148489421 bytes/sec >>>>=20 >>>>=20 >>>> # sector =3D 512b >>>> 3DES-CBC-192 =3D 20773120 bytes/sec >>>> AES-CBC-128 =3D 85276853 bytes/sec >>>> AES-CBC-256 =3D 68893016 bytes/sec >>>> AES-XTS-128 =3D 68194868 bytes/sec >>>> AES-XTS-256 =3D 56611573 bytes/sec >>>> Blowfish-CBC-128 =3D 11169657 bytes/sec >>>> Blowfish-CBC-256 =3D 11185891 bytes/sec >>>> Camellia-CBC-128 =3D 78077243 bytes/sec >>>> Camellia-CBC-256 =3D 65732219 bytes/sec >>>> ChaCha8-XTS-256 =3D 258042765 bytes/sec >>>> ChaCha12-XTS-256 =3D 223616967 bytes/sec >>>> ChaCha20-XTS-256 =3D 176005366 bytes/sec >>>> XChaCha8-XTS-256 =3D 228292624 bytes/sec >>>> XChaCha12-XTS-256 =3D 195577624 bytes/sec >>>> XChaCha20-XTS-256 =3D 152247267 bytes/sec >>>> XChaCha20-XTS-128 =3D 152717737 bytes/sec ! 128 bit key have same = speed >>>> as 256 >>>>=20 >>>>=20 >>>> # sector =3D 4kb >>>> 3DES-CBC-192 =3D 22018189 bytes/sec >>>> AES-CBC-128 =3D 104097143 bytes/sec >>>> AES-CBC-256 =3D 81983833 bytes/sec >>>> AES-XTS-128 =3D 78559346 bytes/sec >>>> AES-XTS-256 =3D 66047200 bytes/sec >>>> Blowfish-CBC-128 =3D 38635464 bytes/sec >>>> Blowfish-CBC-256 =3D 38810555 bytes/sec >>>> Camellia-CBC-128 =3D 92814510 bytes/sec >>>> Camellia-CBC-256 =3D 75949489 bytes/sec >>>> ChaCha8-XTS-256 =3D 337336982 bytes/sec >>>> ChaCha12-XTS-256 =3D 284740187 bytes/sec >>>> ChaCha20-XTS-256 =3D 217326865 bytes/sec >>>> XChaCha8-XTS-256 =3D 328424551 bytes/sec >>>> XChaCha12-XTS-256 =3D 278579692 bytes/sec >>>> XChaCha20-XTS-256 =3D 211660225 bytes/sec >>>>=20 >>>> Optimized AES-XTS - speed like AES-CBC: >>>> AES-XTS-128 =3D 102841051 bytes/sec >>>> AES-XTS-256 =3D 80813644 bytes/sec >>>=20 >>> Is this from a different patch or what? Can you talk more about = this? >>=20 >> No patch at this moment. >> After optimization ChaCha-XTS I applied these optimizations to the = AES-XTS >> and get this result. >> All changes were aes_xts_reinit() and aes_xts_crypt(), just slightly = changed >> the structure aes_xts_ctx. >>=20 >> aes_xts_ctx: >> u_int8_t tweak[] -> u_int64_t tweak[] >>=20 >> aes_xts_reinit -> same as chacha_xts_reinit() >>=20 >> aes_xts_crypt -> same as chacha_xts_crypt(): >> block[] - temp buf removed; >> xor 1 byte -> xor 8 bytes at once; >> tweak[i] << 1: rotl 1 bit: 1 byte -> 8 bytes; >> unroll loops; >=20 > Ahh, I thought I had done some similar optimizations, but I only did > them to the aesni version of the routines... You should use the macro > above to decide if things are aligned or not... >=20 >>=20 >> Final: >>=20 >> struct aes_xts_ctx { >> rijndael_ctx key1; >> rijndael_ctx key2; >> uint64_t tweak[(AES_XTS_BLOCKSIZE / sizeof(uint64_t))]; >> }; >>=20 >> void >> aes_xts_reinit(caddr_t key, u_int8_t *iv) >> { >> struct aes_xts_ctx *ctx =3D (struct aes_xts_ctx *)key; >>=20 >> /* >> * Prepare tweak as E_k2(IV). IV is specified as LE = representation >> * of a 64-bit block number which we allow to be passed in = directly. >> */ >> if (ALIGNED_POINTER(iv, uint64_t)) { >> ctx->tweak[0] =3D (*((uint64_t*)(void*)iv)); >> } else { >> bcopy(iv, ctx->tweak, sizeof(uint64_t)); >> } >> /* Convert to LE. */ >> ctx->tweak[0] =3D htole64(ctx->tweak[0]); >=20 > Hmm... this line bothers me.. I'll need to spend more time reading up > to decide if it is buggy or not... Is ctx->tweak in host order? or LE > order? I believe it's suppose to be LE order, as it gets passed > directly to _encryt.. I'm also not sure if the original code is BE > clean, which is part of my problem... >=20 >> /* Last 64 bits of IV are always zero */ >> ctx->tweak[1] =3D 0; >>=20 >> rijndael_encrypt(&ctx->key2, (uint8_t*)ctx->tweak, >> (uint8_t*)ctx->tweak); >> } >>=20 >> static void >> aes_xts_crypt(struct aes_xts_ctx *ctx, u_int8_t *data, u_int = do_encrypt) >> { >> size_t i; >> uint64_t crr, tm; >>=20 >> if (ALIGNED_POINTER(blk, uint64_t)) { >> ((uint64_t*)(void*)data)[0] ^=3D ctx->tweak[0]; >> ((uint64_t*)(void*)data)[1] ^=3D ctx->tweak[1]; >> } else { >> for (i =3D 0; i < AES_XTS_BLOCKSIZE; i ++) >> data[i] ^=3D ((uint8_t*)ctx->tweak)[i]; >> } >>=20 >> if (do_encrypt) >> rijndael_encrypt(&ctx->key1, data, data); >> else >> rijndael_decrypt(&ctx->key1, data, data); >>=20 >> if (ALIGNED_POINTER(blk, uint64_t)) { >> ((uint64_t*)(void*)data)[0] ^=3D ctx->tweak[0]; >> ((uint64_t*)(void*)data)[1] ^=3D ctx->tweak[1]; >> } else { >> for (i =3D 0; i < AES_XTS_BLOCKSIZE; i ++) >> data[i] ^=3D ((uint8_t*)ctx->tweak)[i]; >> } >>=20 >> /* Exponentiate tweak */ >> crr =3D (ctx->tweak[0] >> ((sizeof(uint64_t) * 8) - 1)); >> ctx->tweak[0] =3D (ctx->tweak[0] << 1); >>=20 >> tm =3D ctx->tweak[1]; >> ctx->tweak[1] =3D ((tm << 1) | crr); >> crr =3D (tm >> ((sizeof(uint64_t) * 8) - 1)); >>=20 >> if (crr) >> ctx->tweak[0] ^=3D 0x87; /* GF(2^128) generator = polynomial. */ >=20 > Please use the AES_XTS_ALPHA define instead of hardcoding the value.. >=20 > Thanks. >=20 > -- > John-Mark Gurney Voice: +1 415 225 5579 >=20 > "All that I will do, has been done, All that I have, has not." > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to = "freebsd-hackers-unsubscribe@freebsd.org" --Apple-Mail=_43A9D6FE-F5F9-4CCC-B6A3-B8B5171B44D8 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJUtJOjAAoJECvXQw+IBr2aY+8P/iuzKXskTgHrRJUYnLcL6B9M JXD/KGCX38n5jEt7wzkv5dlvihnXvYaHxdvA0xQ7ehCEWdBwV4w/lwWnMcl10a1n SIbzyDtk5diYsbHBKLEQE3uuWXG1dcC08+LS3J4QYz0oYJzdJkVe/8Ci3FSCNhGX tt2RZJjMikVZcMU9/4TD51zvKbJfWaZOiS6Z/BTU/gWmPx0+HzelbudR8zrs6w3+ 0ow8PZE39qaj+RIxHjUhQyHGXRMnGW2ebrX/7nanVTO2j6Hxxip1Kqfc3Aa3wSIx S2NrL2VCA+vOfAcHqeAFOjAPrnasYivR3Rjw1aJ8u7m7wwn2ZVTSfGgykR+rvuIp wNWCb7N+487yLTxVH4+xso8hUnxEAJ/rkVQaS44JR3Bm0hGUkDaPZ5obp+7Szu3S BJAqHLkKn5NqHyXENfKdZQEFYHEot9m9H1gNWXqWSmk/0sed7bC1CjD3LQ3MRCQk tRjr6REATviqRT/DRKwQ7ldX1GUe3WN6t2ozA4xbxM/H7IGdKkztmZ9p4urnhIgp 3B6NhWzhX7bVkHZbEu/dq8WC8ZQMF+PlfcOTyDb8wl8Dfb9va/+vriV6zOosKOMU tzAbK/kDgSE/m2Aum1xYlCC1NxW02VfHrEVYGP2YHfA1i9a1fa+yqkR3gMYZjpHE qLjhXxVTMebG60ru9R84 =IQih -----END PGP SIGNATURE----- --Apple-Mail=_43A9D6FE-F5F9-4CCC-B6A3-B8B5171B44D8-- From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 13 05:41:41 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C5BB2D0C; Tue, 13 Jan 2015 05:41:41 +0000 (UTC) Received: from platinum.linux.pl (platinum.edu.pl [81.161.192.4]) by mx1.freebsd.org (Postfix) with ESMTP id 5035B850; Tue, 13 Jan 2015 05:41:37 +0000 (UTC) Received: by platinum.linux.pl (Postfix, from userid 87) id D237145218C; Tue, 13 Jan 2015 06:35:09 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on platinum.linux.pl X-Spam-Level: X-Spam-Status: No, score=-1.3 required=3.0 tests=ALL_TRUSTED,AWL autolearn=disabled version=3.4.0 Received: from [10.255.0.2] (unknown [83.151.38.73]) by platinum.linux.pl (Postfix) with ESMTPA id 5719B452086; Tue, 13 Jan 2015 06:35:09 +0100 (CET) Message-ID: <54B4AE55.9090205@platinum.linux.pl> Date: Tue, 13 Jan 2015 06:34:13 +0100 From: Adam Nowacki User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org, freebsd-geom@FreeBSD.org Subject: Re: ChaCha8/12/20 and GEOM ELI tests References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> In-Reply-To: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jan 2015 05:41:42 -0000 Maybe faster but a stream cipher is unusable for disk encryption - iv is derived from sector number and doesn't change. Being able to write a known plaintext and read resulting ciphertext allows you to recover the cipher stream and decrypt any past or future data stored on that sector. Also use of XTS in this context is a no-op since: plain text XOR tweak XOR cipher stream XOR tweak = plain text XOR cipher stream On 2015-01-12 04:13, rozhuk.im@gmail.com wrote: > > FreeBSD firewall 11.0-CURRENT FreeBSD 11.0-CURRENT #3 r276867M: Fri Jan 9 > 09:34:39 MSK 2015 root@firewall:/usr/obj/usr/src/sys/RIMx64 amd64 > > ChaСha patch: > http://netlab.linkpc.net/download/software/FreeBSD/patches/chacha.patch > > HW: Core Duo E8500, 8Gb DDR2-800. > dd if=/dev/zero of=/dev/md0 bs=1m > 2148489421 bytes/sec > > > # sector = 512b > 3DES-CBC-192 = 20773120 bytes/sec > AES-CBC-128 = 85276853 bytes/sec > AES-CBC-256 = 68893016 bytes/sec > AES-XTS-128 = 68194868 bytes/sec > AES-XTS-256 = 56611573 bytes/sec > Blowfish-CBC-128 = 11169657 bytes/sec > Blowfish-CBC-256 = 11185891 bytes/sec > Camellia-CBC-128 = 78077243 bytes/sec > Camellia-CBC-256 = 65732219 bytes/sec > ChaCha8-XTS-256 = 258042765 bytes/sec > ChaCha12-XTS-256 = 223616967 bytes/sec > ChaCha20-XTS-256 = 176005366 bytes/sec > XChaCha8-XTS-256 = 228292624 bytes/sec > XChaCha12-XTS-256 = 195577624 bytes/sec > XChaCha20-XTS-256 = 152247267 bytes/sec > XChaCha20-XTS-128 = 152717737 bytes/sec ! 128 bit key have same speed as 256 > > > # sector = 4kb > 3DES-CBC-192 = 22018189 bytes/sec > AES-CBC-128 = 104097143 bytes/sec > AES-CBC-256 = 81983833 bytes/sec > AES-XTS-128 = 78559346 bytes/sec > AES-XTS-256 = 66047200 bytes/sec > Blowfish-CBC-128 = 38635464 bytes/sec > Blowfish-CBC-256 = 38810555 bytes/sec > Camellia-CBC-128 = 92814510 bytes/sec > Camellia-CBC-256 = 75949489 bytes/sec > ChaCha8-XTS-256 = 337336982 bytes/sec > ChaCha12-XTS-256 = 284740187 bytes/sec > ChaCha20-XTS-256 = 217326865 bytes/sec > XChaCha8-XTS-256 = 328424551 bytes/sec > XChaCha12-XTS-256 = 278579692 bytes/sec > XChaCha20-XTS-256 = 211660225 bytes/sec > > Optimized AES-XTS - speed like AES-CBC: > AES-XTS-128 = 102841051 bytes/sec > AES-XTS-256 = 80813644 bytes/sec > > > > Prepare env: > mdmfs -S -o async -s 4g md /media > > Per test: > geli init -v -e ALGO_NAME -i 8 -l KEY_LEN -s SECTOR_SIZE /dev/md0 > geli attach /dev/md0 > dd if=/dev/zero of=/dev/md0.eli bs=1m > geli detach /dev/md0.eli > > > top -aSCHIP > > CPU 0: 0.0% user, 0.0% nice, 45.8% system, 0.0% interrupt, 54.2% idle > CPU 1: 0.0% user, 0.0% nice, 54.2% system, 0.0% interrupt, 45.8% idle > Mem: 4104M Active, 364M Inact, 558M Wired, 828M Buf, 2927M Free > Swap: > > PID USERNAME PRI NICE SIZE RES STATE C TIME CPU COMMAND > 10 root 155 ki31 0K 32K RUN 0 842:15 54.04% [idle{idle: > cpu0}] > 5319 root 43 - 0K 16K CPU1 1 0:30 51.55% [g_eli[1] > md0] > 10 root 155 ki31 0K 32K RUN 1 842:36 45.69% [idle{idle: > cpu1}] > 5318 root 43 - 0K 16K RUN 0 0:32 43.47% [g_eli[0] > md0] > 3490 root -8 - 0K 16K mdwait 1 2:11 2.79% [md0] > 12 root -8 - 0K 48K - 1 0:48 1.25% > [geom{g_up}] > 5399 root -8 0 12188K 3904K physwr 1 0:00 0.81% dd > if=/dev/zero of=/dev/md0.eli bs=1m > 3506 root 40 0 21668K 3688K CPU0 0 0:11 0.16% top -aSCHIP > 12 root -8 - 0K 48K - 1 0:06 0.14% > [geom{g_down}] > > > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 13 06:54:59 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 257F6B22; Tue, 13 Jan 2015 06:54:59 +0000 (UTC) Received: from mail-la0-x22f.google.com (mail-la0-x22f.google.com [IPv6:2a00:1450:4010:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B0C0EF17; Tue, 13 Jan 2015 06:54:58 +0000 (UTC) Received: by mail-la0-f47.google.com with SMTP id hz20so1038540lab.6; Mon, 12 Jan 2015 22:54:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:reply-to:to:references:in-reply-to:subject:date :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=hArpmUxCLW2B6v1yvIiKRBksxKyPMhQT5HGBdZCdltk=; b=nTGll3v39pVahE1pUXOeWQXw0RGOHbPTdOlotg80I3AG2WklUbarZRnvzC0+zstOyT eMlwvfgkqH+GyhAMgINjV3cin4oUKXZkoOklBeoosTe0Y6C6XTT0NgFfaRUNPUvxSLC7 M8UD62dNYZdiLuT72k9OxQhIq/FZQalJ7DtEBSp9nVmS31gcmQy04SJjysOvFV7k7Los GqgzUnV5p4/pfI+SHLZimf2+F48SXh3ItkUTPUkQ1xhGe9PFAopZisABfkElPSCEng+m 5uFaPdY4QtcXy08MOOGamv9lCiyXoG/uvh9Og5d+EXsYk2I89GbnxVKWLUu3IBL9ZSsj LIRg== X-Received: by 10.152.6.132 with SMTP id b4mr41106234laa.59.1421132096742; Mon, 12 Jan 2015 22:54:56 -0800 (PST) Received: from rimwks1w7x64 ([2001:470:1f15:8e:b007:2759:7397:9491]) by mx.google.com with ESMTPSA id is5sm583460lac.41.2015.01.12.22.54.54 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 12 Jan 2015 22:54:55 -0800 (PST) Message-ID: <54b4c13f.45c5980a.6b2c.1d44@mx.google.com> X-Google-Original-Message-ID: <026801d02efd$d6bcb7c0$84362740$@IM@gmail.com> From: rozhuk.im@gmail.com X-Google-Original-From: Reply-To: To: "'Adam Nowacki'" , , References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> In-Reply-To: <54B4AE55.9090205@platinum.linux.pl> Subject: RE: ChaCha8/12/20 and GEOM ELI tests Date: Tue, 13 Jan 2015 09:54:53 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdAu85//yn4xS/63R0GS0vN078AwhQACNKYg Content-Language: ru X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jan 2015 06:54:59 -0000 > Maybe faster but a stream cipher is unusable for disk encryption - iv > is derived from sector number and doesn't change. Being able to write = a > known plaintext and read resulting ciphertext allows you to recover = the > cipher stream and decrypt any past or future data stored on that > sector. > Also use of XTS in this context is a no-op since: > plain text XOR tweak XOR cipher stream XOR tweak =3D plain text XOR > cipher stream Looks like you're right. Shame on me. 1. ChaCha and XChaCha and can be left in /dev/crypto for future = applications 2. Geom GELI can leave some small changes for the future - it will be = easier to add XTS algorithms. 3. AES-XTC can work faster. From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 13 09:11:28 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35883136; Tue, 13 Jan 2015 09:11:28 +0000 (UTC) Received: from mail.0x20.net (mail.0x20.net [IPv6:2001:aa8:fffb:1::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 90AF1E6E; Tue, 13 Jan 2015 09:11:27 +0000 (UTC) Received: from e-new.0x20.net (mail.0x20.net [IPv6:2001:aa8:fffb:1::3]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.0x20.net (Postfix) with ESMTPS id 6C5A26A6004; Tue, 13 Jan 2015 10:11:22 +0100 (CET) Received: from e-new.0x20.net (localhost [127.0.0.1]) by e-new.0x20.net (8.14.7/8.14.7) with ESMTP id t0D9BMeX067019; Tue, 13 Jan 2015 10:11:22 +0100 (CET) (envelope-from lars@e-new.0x20.net) Received: (from lars@localhost) by e-new.0x20.net (8.14.7/8.14.7/Submit) id t0D9BM7d066113; Tue, 13 Jan 2015 10:11:22 +0100 (CET) (envelope-from lars) Date: Tue, 13 Jan 2015 10:11:22 +0100 From: Lars Engels To: Allan Jude Subject: Re: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged Message-ID: <20150113091122.GK67556@e-new.0x20.net> References: <54B35B36.4040504@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="9Jdw4pA1x1k2W7MG" Content-Disposition: inline In-Reply-To: <54B35B36.4040504@freebsd.org> X-Editor: VIM - Vi IMproved 7.4 X-Operation-System: FreeBSD 8.4-RELEASE-p16 User-Agent: Mutt/1.5.23 (2014-03-12) X-Mailman-Approved-At: Tue, 13 Jan 2015 12:48:13 +0000 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jan 2015 09:11:28 -0000 --9Jdw4pA1x1k2W7MG Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jan 12, 2015 at 12:27:18AM -0500, Allan Jude wrote: > On 2015-01-12 00:24, Pokala, Ravi wrote: > > Hi folks, > >=20 > > Several of us have noticed that there's a long pause at the start of > > booting the kernel on amd64 systems with large amounts of RAM. During t= his > > pause, the kernel is mapping in the memory ranges, but does not emit any > > progress indicators. Because this can take quite a while, it can look l= ike > > the kernel has hung. I filed PR 196650 about this issue; this patch just > > prints out a dot for every GB that's mapped in. We've been using this > > patch for years at Panasas, and I'm hoping someone can submit it for me. > >=20 > > Thanks, > >=20 > > Ravi > >=20 > >=20 > > Index: sys/amd64/amd64/machdep.c > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > --- sys/amd64/amd64/machdep.c (revision 276903) > > +++ sys/amd64/amd64/machdep.c (working copy) > > @@ -1575,6 +1575,9 @@ > > u_long physmem_start, physmem_tunable, memtest; > > pt_entry_t *pte; > > quad_t dcons_addr, dcons_size; > > + u_int32_t page_counter; > > + int PAGES_PER_MB =3D ((1024 * 1024) / PAGE_SIZE); > > + int PAGES_PER_GB =3D (PAGES_PER_MB * 1024); > > =20 > > bzero(physmap, sizeof(physmap)); > > basemem =3D 0; > > @@ -1678,6 +1681,8 @@ > > * physmap is in bytes, so when converting to page boundaries, > > * round up the start address and round down the end address. > > */ > > + printf("Mapping system memory"); > > + page_counter =3D 0; > > for (i =3D 0; i <=3D physmap_idx; i +=3D 2) { > > vm_paddr_t end; > > =20 > > @@ -1688,6 +1693,14 @@ > > int tmp, page_bad, full; > > int *ptr =3D (int *)CADDR1; > > =20 > > + /* > > + * Print a "." every GB, to show we're making progress > > + */ > > + page_counter++; > > + if ((page_counter % PAGES_PER_GB) =3D=3D 0) { > > + printf("."); > > + } > > + > > full =3D FALSE; > > /* > > * block out kernel memory as not available. > > @@ -1792,6 +1805,9 @@ > > break; > > } > > } > > + printf("\nMapped %d GB + %d MB total\n", > > + (page_counter / PAGES_PER_GB), > > + ((page_counter % PAGES_PER_GB) / PAGES_PER_MB)); > > *pte =3D 0; > > invltlb(); > > =20 > >=20 > >=20 > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" > >=20 >=20 > Which version(s) of FreeBSD were you seeing this delay on? >=20 > I know older versions had it, quite badly. >=20 > setting hw.memtest.tests=3D0 in /boot/loader.conf makes most of the delay > go away, and the default on -CURRENT is now 0 instead of 1. >=20 > IIRC, in 10, it defaults to 1, but is switched to 0 in the case of > virtual machines, because touching every page of memory ruined memory > over-commit type features. >=20 > Is this feature still useful with memtest.tests=3D0? This feature is useful for everyone who has it set to 1, if they know about it or not. So it's a very useful feature. --9Jdw4pA1x1k2W7MG Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAEBCgBmBQJUtOE6XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ4RjQwMDE3RTRERjUzMTI1N0FGRTUxNDlF NTRDQjM3RDNBMDg5RDZEAAoJEOVMs306CJ1trFUH/iOnxon1rZIsHxGVUbE5/gdz 2WSkNeLSjOvXSjinInDcOV4nE3TIEVH+FIL8SqOceM1zCwIiGuRU6Wh0VPlU2nlD thM20CU7aqhY+JXvSt9wEhONspDcCDTpF+xDLVH/HHKbj1d6DnolYMQQ6NSrulym raRlMzsA8hjo6VRSR9F/dQYTDnFhm4hA4yb3M1eNNSmwGyeSJdOey1EN/JLQagBZ 1spwt4Kw6lptf+HexqAkTOx677CCm/4DyOy1PFOBdLBC+78zE+dWFlW/XZ1nk7L2 IjRY9wbU66ZSRE4zqZU4JlDA4huFgMRBeSbWuOdpoYMuqPNIVGguSHzZYOGxDDs= =St1w -----END PGP SIGNATURE----- --9Jdw4pA1x1k2W7MG-- From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 13 15:22:33 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 121E7E97 for ; Tue, 13 Jan 2015 15:22:33 +0000 (UTC) Received: from astart2.astart.com (108-248-95-193.lightspeed.sndgca.sbcglobal.net [108.248.95.193]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E2143D89 for ; Tue, 13 Jan 2015 15:22:32 +0000 (UTC) Received: from laptop_93.private (localhost [127.0.0.1]) by astart2.astart.com (8.14.4/8.14.4) with ESMTP id t0DErEJh024979 for ; Tue, 13 Jan 2015 06:53:15 -0800 (PST) (envelope-from papowell@astart.com) Message-ID: <54B5315A.9000108@astart.com> Date: Tue, 13 Jan 2015 06:53:14 -0800 From: Patrick Powell Reply-To: papowell@astart.com Organization: Astart Technologies User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jan 2015 15:22:33 -0000 On 01/12/15 11:43, Pokala, Ravi wrote: > -----Original Message----- >> Date: Mon, 12 Jan 2015 00:27:18 -0500 >> From: Allan Jude >> To: freebsd-hackers@freebsd.org >> Subject: Re: [PATCH] Display progress during getmemsize() so the >> kernel doesn't look like it hanged >> Message-ID: <54B35B36.4040504@freebsd.org> >> Content-Type: text/plain; charset="windows-1252" >> >> On 2015-01-12 00:24, Pokala, Ravi wrote: >>> Hi folks, >>> >>> Several of us have noticed that there's a long pause at the start of >>> booting the kernel on amd64 systems with large amounts of RAM. During >>> This pause, the kernel is mapping in the memory ranges, but does not >>> emit any progress indicators. Because this can take quite a while, it >>> can look like the kernel has hung. I filed PR 196650 about this issue; >>> this patch just prints out a dot for every GB that's mapped in. We've >>> been using this patch for years at Panasas, and I'm hoping someone can >>> submit it for me. >>> >>> Thanks, >>> >>> Ravi >>> >>> >>> >>> ... >>> >>> >>> >> Which version(s) of FreeBSD were you seeing this delay on? > We first noted it with 7.2, because that's what we used on our first amd64 > systems with > 4GB RAM. We're still seeing it with 10.1. > >> I know older versions had it, quite badly. >> >> setting hw.memtest.tests=0 in /boot/loader.conf makes most of the delay >> go away, and the default on -CURRENT is now 0 instead of 1. >> >> IIRC, in 10, it defaults to 1, but is switched to 0 in the case of >> virtual machines, because touching every page of memory ruined memory >> over-commit type features. >> >> Is this feature still useful with memtest.tests=0? > You're right; when I set hw.memtest.tests=0, the pause goes away in 10.1. > I thought it defaulted to 0 in 10.1. > > So that changes this slightly - could we either change the default in > 10-STABLE, or commit the following patch against 10-STABLE? > > Thanks, > > Ravi > > > Index: sys/amd64/amd64/machdep.c > =================================================================== > diff --git a/stable/10/sys/amd64/amd64/machdep.c > b/stable/10/sys/amd64/amd64/machdep.c > --- a/stable/10/sys/amd64/amd64/machdep.c (revision 277085) > +++ b/stable/10/sys/amd64/amd64/machdep.c (working copy) > @@ -1541,6 +1541,9 @@ > struct bios_smap *smapbase; > struct efi_map_header *efihdr; > quad_t dcons_addr, dcons_size; > + u_int32_t page_counter; > + int PAGES_PER_MB = ((1024 * 1024) / PAGE_SIZE); > + int PAGES_PER_GB = (PAGES_PER_MB * 1024); > > bzero(physmap, sizeof(physmap)); > basemem = 0; > @@ -1651,6 +1654,8 @@ > * physmap is in bytes, so when converting to page boundaries, > * round up the start address and round down the end address. > */ > + printf("Mapping system memory"); > + page_counter = 0; > for (i = 0; i <= physmap_idx; i += 2) { > vm_paddr_t end; > > @@ -1661,6 +1666,14 @@ > int tmp, page_bad, full; > int *ptr = (int *)CADDR1; > > + /* > + * Print a "." every GB, to show we're making progress > + */ > + page_counter++; > + if ((page_counter % PAGES_PER_GB) == 0) { > + printf("."); > + } > + > full = FALSE; > /* > * block out kernel memory as not available. > @@ -1765,6 +1778,9 @@ > break; > } > } > + printf("\nMapped %d GB + %d MB total\n", > + (page_counter / PAGES_PER_GB), > + ((page_counter % PAGES_PER_GB) / PAGES_PER_MB)); > *pte = 0; > invltlb(); > > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > I like the idea of the busy dots. It would make new sysadmins doing installs feel much happier when the system pauses for a couple of minutes during boot. Patrick ("Its dead, Jim!" "Its not dead, its just checking memory.") Powell -- Patrick Powell Astart Technologies papowell@astart.com 1530 Jamacha Rd, Suite X Network and System San Diego, CA 92019 Consulting 858-874-6543 FAX 858-751-2435 Web: www.astart.com From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 13 15:48:25 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4DBF02C6; Tue, 13 Jan 2015 15:48:25 +0000 (UTC) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [IPv6:2607:fc50:1000:7400:216:3eff:fe72:314f]) by mx1.freebsd.org (Postfix) with ESMTP id 2F1B998; Tue, 13 Jan 2015 15:48:25 +0000 (UTC) Received: from marvin.lab.vangyzen.net (c-24-125-214-90.hsd1.va.comcast.net [24.125.214.90]) by smtp.vangyzen.net (Postfix) with ESMTPSA id BE6CA56467; Tue, 13 Jan 2015 09:48:17 -0600 (CST) Message-ID: <54B53E40.1030903@vangyzen.net> Date: Tue, 13 Jan 2015 10:48:16 -0500 From: Eric van Gyzen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.0 MIME-Version: 1.0 To: Lars Engels , Allan Jude Subject: Re: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged References: <54B35B36.4040504@freebsd.org> <20150113091122.GK67556@e-new.0x20.net> In-Reply-To: <20150113091122.GK67556@e-new.0x20.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jan 2015 15:48:25 -0000 On 01/13/2015 04:11, Lars Engels wrote: > On Mon, Jan 12, 2015 at 12:27:18AM -0500, Allan Jude wrote: >> Is this feature still useful with memtest.tests=0? > This feature is useful for everyone who has it set to 1, if they know > about it or not. So it's a very useful feature. Agreed. Comments on the patch: The patch will divide by zero when PAGE_SIZE > 1 MiB. Maybe remove PAGES_PER_MB, and just use PAGES_PER_GB. Make it const, too. The "total" line is mostly redundant with the later messages regarding "real memory" and "avail memory". I suggest removing it. Eric From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 13 17:55:44 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4341663; Tue, 13 Jan 2015 17:55:44 +0000 (UTC) Received: from mail.myota.org (mail.myota.org [85.10.206.105]) by mx1.freebsd.org (Postfix) with ESMTP id 4766A1DD; Tue, 13 Jan 2015 17:55:43 +0000 (UTC) Received: from e181126138.adsl.alicedsl.de (e181126138.adsl.alicedsl.de [85.181.126.138]) (authenticated bits=128) by mail.myota.org (8.14.9/8.14.9) with ESMTP id t0DHsvpl024759; Tue, 13 Jan 2015 18:55:03 +0100 (CET) (envelope-from andre@albsmeier.net) Received: from stationary.client ([192.168.128.2]) by gate.local (8.14.9/8.14.9) with ESMTP id t0DHsu8Z035667; Tue, 13 Jan 2015 18:54:56 +0100 (CET) (envelope-from andre@gate.home.albsmeier.net) Received: from submit.client ([127.0.0.1]) by voyager.local (8.14.9/8.14.9) with ESMTP id t0DHsuLJ007307; Tue, 13 Jan 2015 18:54:56 +0100 (CET) (envelope-from andre@voyager.home.albsmeier.net) Received: (from user@localhost) by voyager.local (8.14.9/8.14.9/Submit) id t0DHsuKr007306; Tue, 13 Jan 2015 18:54:56 +0100 (CET) (envelope-from andre@voyager.home.albsmeier.net) Date: Tue, 13 Jan 2015 18:54:56 +0100 From: Andre Albsmeier To: Ed Maste Subject: Re: [PATCH] Fixing panic in vt_fb_blank() if fb_size is not a multiple of fb_stride Message-ID: <20150113175456.GA7271@voyager> References: <20150101192219.GA46601@voyager> <20150102072620.GB65505@gate> <9F362787-EC79-4ADC-B414-4EDBF592B868@gmail.com> <20150105135856.GA65607@gate> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Echelon: F-15, plutonium, Blowfish, 737, assault X-Advice: Drop that crappy M$-Outlook, I'm tired of your viruses! User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Not delayed on 85.10.206.105, ACL: AUTH(59), Origin: DE, OS: FreeBSD 9.x X-Greylist: Not delayed on 192.168.128.1, ACL: RFC_Nets(54), Origin: , OS: unknown X-Virus-Scanned: clamav-milter 0.98.5 at colo X-Virus-Status: Clean X-Mailman-Approved-At: Tue, 13 Jan 2015 18:05:52 +0000 Cc: Andre Albsmeier , Adrian Chadd , "freebsd-x11@freebsd.org" , Garrett Cooper , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jan 2015 17:55:44 -0000 On Mon, 12-Jan-2015 at 15:51:08 -0500, Ed Maste wrote: > On 5 January 2015 at 08:58, Andre Albsmeier wrote: > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=196510 > > Thanks, this should now be fixed in stable/9 with r277083. Confirmed to be fixed, thanks! From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 13 18:53:31 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 92BF4A56; Tue, 13 Jan 2015 18:53:31 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6970FB3E; Tue, 13 Jan 2015 18:53:31 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 3D882B924; Tue, 13 Jan 2015 13:53:30 -0500 (EST) From: John Baldwin To: Eric van Gyzen , Lars Engels , Allan Jude Subject: Re: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged Date: Tue, 13 Jan 2015 13:53:19 -0500 Message-ID: <3221643.2Qu3vC2WD5@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: <54B53E40.1030903@vangyzen.net> References: <54B35B36.4040504@freebsd.org> <20150113091122.GK67556@e-new.0x20.net> <54B53E40.1030903@vangyzen.net> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 13 Jan 2015 13:53:30 -0500 (EST) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jan 2015 18:53:31 -0000 On 1/13/15 10:48 AM, Eric van Gyzen wrote: > On 01/13/2015 04:11, Lars Engels wrote: >> On Mon, Jan 12, 2015 at 12:27:18AM -0500, Allan Jude wrote: >>> Is this feature still useful with memtest.tests=0? >> This feature is useful for everyone who has it set to 1, if they know >> about it or not. So it's a very useful feature. > > Agreed. > > Comments on the patch: > > The patch will divide by zero when PAGE_SIZE > 1 MiB. Maybe remove > PAGES_PER_MB, and just use PAGES_PER_GB. Make it const, too. > > The "total" line is mostly redundant with the later messages regarding > "real memory" and "avail memory". I suggest removing it. I agree with these. One other nit is that FreeBSD tends to use all caps for a macro rather than variables. Ravi, how about this variant, it also only displays the dots if the memory test is enabled. Also, I think the commit to disable the test should probably be merged to 10, yes. Index: machdep.c =================================================================== --- machdep.c (revision 277075) +++ machdep.c (working copy) @@ -1557,6 +1557,8 @@ native_parse_memmap(caddr_t kmdp, vm_paddr_t *phys } } +#define PAGES_PER_GB (1024 * 1024 * 1024 / PAGE_SIZE) + /* * Populate the (physmap) array with base/bound pairs describing the * available physical memory in the system, then test this memory and @@ -1575,6 +1577,7 @@ getmemsize(caddr_t kmdp, u_int64_t first) u_long physmem_start, physmem_tunable, memtest; pt_entry_t *pte; quad_t dcons_addr, dcons_size; + int page_counter; bzero(physmap, sizeof(physmap)); basemem = 0; @@ -1678,6 +1681,9 @@ getmemsize(caddr_t kmdp, u_int64_t first) * physmap is in bytes, so when converting to page boundaries, * round up the start address and round down the end address. */ + page_counter = 0; + if (memtest != 0) + printf("Testing system memory"); for (i = 0; i <= physmap_idx; i += 2) { vm_paddr_t end; @@ -1708,6 +1714,14 @@ getmemsize(caddr_t kmdp, u_int64_t first) goto skip_memtest; /* + * Print a "." every GB to show we're making + * progress. + */ + page_counter++; + if ((page_counter % PAGES_PER_GB) == 0) + printf("."); + + /* * map page into kernel: valid, read/write,non-cacheable */ *pte = pa | PG_V | PG_RW | PG_NC_PWT | PG_NC_PCD; @@ -1794,6 +1808,8 @@ do_next: } *pte = 0; invltlb(); + if (memtest != 0) + printf("\n"); /* * XXX -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 02:21:17 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 523F0933; Wed, 14 Jan 2015 02:21:17 +0000 (UTC) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DF840FC3; Wed, 14 Jan 2015 02:21:16 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id pn19so5821571lab.9; Tue, 13 Jan 2015 18:21:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:reply-to:to:references:in-reply-to:subject:date :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=80SjMj+qafb2xMO6td5tScnjSBfmsPI0+U5SLnp1u80=; b=YbMtrc+pjYDp7M9LS6XUTFp6M9/RUiFsO4utI1t32qZSbVM2orvXRVzoS1/N8SNfFH ovYKfishFc680LsBXY8mA4TUPhfykQjVzyp/cK+kQ8+bc/LkOshHJeb+HAn+kSQOEaDM 6zV90X8RoibMpV31qgd55pUVKulxNy9iBkFpMJI0U4ro+vmO1wxTJqWjPch/lxMnxpGZ eXLfx3DVmckrV8whsci85Du+fvQTvl8KcrblVbqYkGxy5BqAUJRTpnIW7C3LFWoS8AHn gJyWEddASZGBuPipImZcmePaOY2kJ7ZlPcAma/lnf+oIOXeJXkDLpWUxIFnKH7vFwkdo /xXw== X-Received: by 10.152.5.132 with SMTP id s4mr1333650las.39.1421202074856; Tue, 13 Jan 2015 18:21:14 -0800 (PST) Received: from rimwks1w7x64 ([2001:470:1f15:8e:b007:2759:7397:9491]) by mx.google.com with ESMTPSA id l9sm1324777lae.0.2015.01.13.18.21.12 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 13 Jan 2015 18:21:13 -0800 (PST) Message-ID: <54b5d299.4914980a.61cd.43a6@mx.google.com> X-Google-Original-Message-ID: <027201d02fa0$c49b9db0$4dd2d910$@IM@gmail.com> From: rozhuk.im@gmail.com X-Google-Original-From: Reply-To: To: "'Adam Nowacki'" , , References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> In-Reply-To: <54B4AE55.9090205@platinum.linux.pl> Subject: RE: ChaCha8/12/20 and GEOM ELI tests Date: Wed, 14 Jan 2015 05:21:10 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdAu85//yn4xS/63R0GS0vN078AwhQAqcwFg Content-Language: ru X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 02:21:17 -0000 > Maybe faster but a stream cipher is unusable for disk encryption - iv > is derived from sector number and doesn't change. Being able to write = a > known plaintext and read resulting ciphertext allows you to recover = the > cipher stream and decrypt any past or future data stored on that > sector. Depends on the capabilities of the attacker. To be able to continuously read encrypted sectors for data collection is = too much. Ability to read encrypted sectors has a transmission network, for = example when the container=3Ddisk is stored somewhere in the cloud. In many cases, the attacker gets Encrypted disk along with other = equipment, often in the off state. Without encryption keys and the ability to write / read through the = GELI. I do not see any weaknesses stream ciphers in cases when the attacker is = not able to access the disk when it is mounted in the GEOM GELI. Another possibility is the use of ChaCha (without XTS) - encryption swap = file: there every time a new key is generated, besides the speed is = particularly important. These aspects of the application must necessarily be reflected in the = documentation. There are objections to add ChaCha and XChaCha (without XTS) in GEOM = GELI? From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 02:26:00 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 83420A72; Wed, 14 Jan 2015 02:26:00 +0000 (UTC) Received: from mail-la0-x235.google.com (mail-la0-x235.google.com [IPv6:2a00:1450:4010:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 18511FF1; Wed, 14 Jan 2015 02:26:00 +0000 (UTC) Received: by mail-la0-f53.google.com with SMTP id gm9so5852445lab.12; Tue, 13 Jan 2015 18:25:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:reply-to:to:cc:references:in-reply-to:subject:date :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=WtWIMdgddjNSyrE9FRhtTfO8AdExmF/uhKli+hAv+XQ=; b=Uvc2roI/+JpncOnUdaGNWLp7FhvogE4HnAXQ0IS6uxc05MVCFM3nPOZImLZVM/qzK6 1eo6hH3tpxPe7XqwFsImd7fIbSliulLPLP+MS0VV1ZqqL9Dh8HIKNbLFK8wdlxnpPypZ lfDOpDTFnhQWrXaIuQ4dbiyfgKuhui4kzCCkhOHSHIa86tbL7dkgr8CqtpoVHPfMsxYM ivbGJMvTb6nhjaCRyViTisPa7m4dLBgPxoxLXA9KNvyH6vR/eR4/L5OarkKz2NmP+q50 PflMQq9P8OhSrc7UkvXZSjpwnutzCg9Dg6qWCQKypz3dfyF8PlXpNlOpcYmhrcg1IB8I QZlQ== X-Received: by 10.153.6.6 with SMTP id cq6mr1375672lad.23.1421202358077; Tue, 13 Jan 2015 18:25:58 -0800 (PST) Received: from rimwks1w7x64 ([2001:470:1f15:8e:b007:2759:7397:9491]) by mx.google.com with ESMTPSA id yf10sm5457573lbb.9.2015.01.13.18.25.55 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 13 Jan 2015 18:25:56 -0800 (PST) Message-ID: <54b5d3b4.2aa3700a.3a6c.25f4@mx.google.com> X-Google-Original-Message-ID: <027901d02fa1$6d929ef0$48b7dcd0$@IM@gmail.com> From: rozhuk.im@gmail.com X-Google-Original-From: Reply-To: To: "'Eric van Gyzen'" , "'Lars Engels'" , "'Allan Jude'" References: <54B35B36.4040504@freebsd.org> <20150113091122.GK67556@e-new.0x20.net> <54B53E40.1030903@vangyzen.net> In-Reply-To: <54B53E40.1030903@vangyzen.net> Subject: RE: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged Date: Wed, 14 Jan 2015 05:25:54 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdAvSGCvsyctOzXxRRqAdAYIqL1KVAAWNGAA Content-Language: ru Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 02:26:00 -0000 > >> Is this feature still useful with memtest.tests=0? > > This feature is useful for everyone who has it set to 1, if they know > > about it or not. So it's a very useful feature. > > Agreed. > > Comments on the patch: > > The patch will divide by zero when PAGE_SIZE > 1 MiB. Maybe remove > PAGES_PER_MB, and just use PAGES_PER_GB. Make it const, too. > > The "total" line is mostly redundant with the later messages regarding > "real memory" and "avail memory". I suggest removing it. > Maybe just write: checked XX megabytes of memory? From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 03:21:46 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 046DE779; Wed, 14 Jan 2015 03:21:46 +0000 (UTC) Received: from mail-qa0-x22c.google.com (mail-qa0-x22c.google.com [IPv6:2607:f8b0:400d:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B24A8971; Wed, 14 Jan 2015 03:21:45 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id w8so5063898qac.3; Tue, 13 Jan 2015 19:21:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=cG6cBMrclQkp5FCKx0KpNV6uyfA8VZKDikVk9J2EdGM=; b=moRGCgLOKmeDvdYnGjAKarPI90axjsgGbWuMYjoe5yO8zTslhG7qITKXt+3UlcDNtt UCQG00J4TiVVWWPLuAFNxkIkdw1NNTQkvTAmQMf6LPyHWmhpsUdP8nxF3zTlz+aUOfO5 qee8ivHOe3PDe5DP/GJHv9BWgA/75Qz5xL4tr4N4Ip48naUrpkHEvBmPZF+2HhnWH3r7 8YWJr7NBHRugZulqZrdHQbi5MtWwwaUrrD8m33pnqJS/PqnCb2u1/OQY2Wooq9jmMIqd JIzM+wZ/wpbj08bXiQys1ecYDVw7Ap7q1VnRUDJRD26hoVd/fVmiGtQf3Fwz+CXCtuIf Tljw== MIME-Version: 1.0 X-Received: by 10.224.147.197 with SMTP id m5mr3067026qav.51.1421205704938; Tue, 13 Jan 2015 19:21:44 -0800 (PST) Received: by 10.96.39.2 with HTTP; Tue, 13 Jan 2015 19:21:44 -0800 (PST) In-Reply-To: <54b5d299.4914980a.61cd.43a6@mx.google.com> References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> Date: Wed, 14 Jan 2015 05:21:44 +0200 Message-ID: Subject: Re: ChaCha8/12/20 and GEOM ELI tests From: Kimmo Paasiala To: Rozhuk.IM@gmail.com Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Hackers , freebsd-geom@freebsd.org, Adam Nowacki X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 03:21:46 -0000 > Depends on the capabilities of the attacker. > > To be able to continuously read encrypted sectors for data collection is too much. > When talking about disk encryption the first assumption is that the attacker always has this capability, even with so much power the attacker shouldn't be able to break the encryption scheme. If he can then the encryption scheme is not secure. -Kimmo From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 03:30:53 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F3F6940; Wed, 14 Jan 2015 03:30:53 +0000 (UTC) Received: from mail-qa0-x232.google.com (mail-qa0-x232.google.com [IPv6:2607:f8b0:400d:c00::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BBFB6A5A; Wed, 14 Jan 2015 03:30:52 +0000 (UTC) Received: by mail-qa0-f50.google.com with SMTP id k15so5097173qaq.9; Tue, 13 Jan 2015 19:30:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=V/qUu2d+VaKGMnVeYnl9rOGI/UiigUYj3x2NWfx8l00=; b=N5w3ln8Q6v6b2OxV7LWzKc4rlZ7pz2K2+GV4LDDPk8H/Ov1io8tMBq7xrzZZbAdhL4 XIgpK8dcL6bUTexMyn0MRMmatu3IJwc3XF7dXIVirhPhe5Z/wWLWT2m8PF6bRsTXwOco CpflyyoXR0XDwvLulJ96WjMYsNvMLjWvqdZCsdTM6knksl0si/w7r18+4YBzyH7DcY7X NZaJnqVNJzWRp89JeXiQ3ik8qI8P6GO6Xil4+3/uO4JE10RRo9awqxkIf6hRcL0DvUUA tUxODdrxDkGpYf8CJhSHppyJNquxtV0Snd/65b2jsh3cII3xvPA/0OmEouzzod/oYujB 0q1A== MIME-Version: 1.0 X-Received: by 10.224.26.4 with SMTP id b4mr3410263qac.26.1421206251865; Tue, 13 Jan 2015 19:30:51 -0800 (PST) Received: by 10.96.39.2 with HTTP; Tue, 13 Jan 2015 19:30:51 -0800 (PST) In-Reply-To: References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> Date: Wed, 14 Jan 2015 05:30:51 +0200 Message-ID: Subject: Re: ChaCha8/12/20 and GEOM ELI tests From: Kimmo Paasiala To: Rozhuk.IM@gmail.com Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Hackers , freebsd-geom@freebsd.org, Adam Nowacki X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 03:30:53 -0000 On Wed, Jan 14, 2015 at 5:21 AM, Kimmo Paasiala wrote: >> Depends on the capabilities of the attacker. >> >> To be able to continuously read encrypted sectors for data collection is too much. >> > > When talking about disk encryption the first assumption is that the > attacker always has this capability, even with so much power the > attacker shouldn't be able to break the encryption scheme. If he can > then the encryption scheme is not secure. > > -Kimmo Sorry pressed sent too fast. The last sentence should have been: Ift the attacker can learn anything about the unencrypted data or predict something about future encrypted or unencrypted blocks by analyzing the previous encrypted blocks the encryption scheme should be considered insecure. -Kimmo From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 04:16:03 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EE3C632B; Wed, 14 Jan 2015 04:16:02 +0000 (UTC) Received: from mail-pd0-x22f.google.com (mail-pd0-x22f.google.com [IPv6:2607:f8b0:400e:c02::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BDCF7E5E; Wed, 14 Jan 2015 04:16:02 +0000 (UTC) Received: by mail-pd0-f175.google.com with SMTP id g10so7372018pdj.6; Tue, 13 Jan 2015 20:16:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-type:content-disposition :in-reply-to:user-agent; bh=uQOxL7nFniBWpyX10UG7fpVjyHgcA2QjtqD8rqj9JvE=; b=USpLqIiP2vRLCIk790VVhJfEbGOXnG1Jy9eFGxVDw6flD9x/vmxJ/6OTbN5Hv/dPoA QSQVYhGmNXPIzwlazrQPfMvXMZDKxFb8BOsFtMo8PDVwWLkUCLCHSEp+U4e4t/ihMTwF LtAMCwRWafjG3dHULXq0FuGFk+F2xMazD0kGAoPoylDi22TVVT3vGc4eRdKg2v2Q2iMF flZ7eqNzt0XDyFaFMxA5xTl1NwBugNgcOqeUni+Sc2pglJdP+JSJhusJJwXKxp2smfOt xGyauufy6AaKh7NDOvPBOqybyRiD4bhRQzpGTNB7hfG7O1qjnoGduoqotSxjZgT+gRSj gv7A== X-Received: by 10.66.228.200 with SMTP id sk8mr2523618pac.134.1421208962259; Tue, 13 Jan 2015 20:16:02 -0800 (PST) Received: from localhost (c-76-21-76-83.hsd1.ca.comcast.net. [76.21.76.83]) by mx.google.com with ESMTPSA id or4sm18542532pab.30.2015.01.13.20.16.01 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Jan 2015 20:16:01 -0800 (PST) Sender: Gleb Kurtsou Date: Tue, 13 Jan 2015 20:17:08 -0800 From: Gleb Kurtsou To: rozhuk.im@gmail.com Subject: Re: ChaCha8/12/20 and GEOM ELI tests Message-ID: <20150114041708.GA3189@reks> Mail-Followup-To: rozhuk.im@gmail.com, 'Adam Nowacki' , freebsd-hackers@freebsd.org, freebsd-geom@FreeBSD.org References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <54b5d299.4914980a.61cd.43a6@mx.google.com> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-hackers@freebsd.org, freebsd-geom@FreeBSD.org, 'Adam Nowacki' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 04:16:03 -0000 On (14/01/2015 05:21), rozhuk.im@gmail.com wrote: > > Maybe faster but a stream cipher is unusable for disk encryption - iv > > is derived from sector number and doesn't change. Being able to write a > > known plaintext and read resulting ciphertext allows you to recover the > > cipher stream and decrypt any past or future data stored on that > > sector. > > Depends on the capabilities of the attacker. > > To be able to continuously read encrypted sectors for data collection > is too much. I disagree. It's the most basic attack scenario. Assuming attacker was able to get access to encrypted disk once, odds are high it may happen again. > Ability to read encrypted sectors has a transmission network, for example when the container=disk is stored somewhere in the cloud. > > In many cases, the attacker gets Encrypted disk along with other equipment, often in the off state. > Without encryption keys and the ability to write / read through the GELI. > > I do not see any weaknesses stream ciphers in cases when the attacker is not able to access the disk when it is mounted in the GEOM GELI. > > Another possibility is the use of ChaCha (without XTS) - encryption > swap file: there every time a new key is generated, besides the speed > is particularly important. Stream cipher (or similarly functioning block cipher mode) should not be used for disk encryption. IMHO swap encryption hardly justifies adding insecure encryption mode to geli. Fast swap is certainly nice to have, but rather remains a snake oil, system will remain trashed due to swapping no matter how fast swap is. > These aspects of the application must necessarily be reflected in the documentation. > > > There are objections to add ChaCha and XChaCha (without XTS) in GEOM GELI? I strongly object. Having XTS mode for a stream cipher in the first place looks really fishy.. Originally encryption is defined as: T = AES-ENC(key2, i) (*) alpha_j PP = P (*) T CC = AES-ENC(key1, PP) C = CC (*) T In stream cipher case: T = CHACHA(key2, state2) (*) i (*) alpha_j PP = P (*) T CC = CHACHA(key1, state1) (*) PP C = CC (*) T CC = CHACHA(key1, state1) (*) P (*) T C = CC (*) T C = CHACHA(key1, state1) (*) P It doesn't depend on i, j or key2. state1 should be the same as well. From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 05:16:50 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4CBB2D49; Wed, 14 Jan 2015 05:16:50 +0000 (UTC) Received: from mail-la0-x234.google.com (mail-la0-x234.google.com [IPv6:2a00:1450:4010:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B9183679; Wed, 14 Jan 2015 05:16:49 +0000 (UTC) Received: by mail-la0-f52.google.com with SMTP id hs14so6276667lab.11; Tue, 13 Jan 2015 21:16:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:reply-to:to:cc:references:in-reply-to:subject:date :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=XJ7qMCtLV4K9HTcdJZZOJSkCKazP+zKnkbNJ/diEDB8=; b=d3j2ZJmJEYVIyRTL2MSnXKMYnfU3r/OtPIE0pzKvBaKUCFEOdsAk6xM2+gsnlfH+i2 hjrDM/beyxaWucPgcSISGidSKP/P7+4AC2pVo6EmdLRy86nIRsQ4JIVwjUzmguyLW+mt DASBz3Ipc9iWCRn2r3ckIGTbiogNSRZpQk7g0NqxWThHGdpWJatBQIb4Grl/JM1mKqpx OMb7DR1YM4gnkjPFuvov18EQm06nN97yR0Mn8SPPd/yy1tAEZOHBedkjHq/wVhgOY6vT AAyQIXyIzla5yoBfmxC7Q+71vmu2ffYCMSqtgXVtDB1DoAFkc6VnNQ9T6hIAGuRn5z2o 4kiA== X-Received: by 10.112.162.4 with SMTP id xw4mr1768643lbb.89.1421212607749; Tue, 13 Jan 2015 21:16:47 -0800 (PST) Received: from rimwks1w7x64 ([2001:470:1f15:8e:b007:2759:7397:9491]) by mx.google.com with ESMTPSA id v4sm2021348lbz.12.2015.01.13.21.16.45 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 13 Jan 2015 21:16:46 -0800 (PST) Message-ID: <54b5fbbe.4457700a.2456.6944@mx.google.com> X-Google-Original-Message-ID: <028401d02fb9$4b11b010$e1351030$@IM@gmail.com> From: rozhuk.im@gmail.com X-Google-Original-From: Reply-To: To: "'Kimmo Paasiala'" References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> In-Reply-To: Subject: RE: ChaCha8/12/20 and GEOM ELI tests Date: Wed, 14 Jan 2015 08:16:44 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdAvqn9GJSXYqkLGQlKI120JgveVWwADsClQ Content-Language: ru Cc: 'FreeBSD Hackers' , freebsd-geom@freebsd.org, 'Adam Nowacki' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 05:16:50 -0000 > >> Depends on the capabilities of the attacker. > >> > >> To be able to continuously read encrypted sectors for data > collection is too much. > >> > > > When talking about disk encryption the first assumption is that the=20 > attacker always has this capability, even with so much power the=20 > attacker shouldn't be able to break the encryption scheme. If he can=20 > then the encryption scheme is not secure. >=20 > Ift the attacker can learn anything about the unencrypted data or=20 > predict something about future encrypted or unencrypted blocks by=20 > analyzing the previous encrypted blocks the encryption scheme should=20 > be considered insecure. I consider the case when the disk can be obtained by physically an = attacker. All the rest of the disk directly connected to the computer. If an attacker can read encrypted data directly to disk means that the = system is already compromised by an attacker, and probably in this case = can read the data from the disk and through read() already decrypted and = get the key from the kernel memory. From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 05:25:46 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7DD5EEF; Wed, 14 Jan 2015 05:25:46 +0000 (UTC) Received: from mail-pd0-x231.google.com (mail-pd0-x231.google.com [IPv6:2607:f8b0:400e:c02::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44FF67C4; Wed, 14 Jan 2015 05:25:46 +0000 (UTC) Received: by mail-pd0-f177.google.com with SMTP id ft15so7654689pdb.8; Tue, 13 Jan 2015 21:25:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :message-id:references:to; bh=TlxlDMH+iv+RU7IP7qPiknWeyPjnTQb/aCCjQR+KKmA=; b=E0agUvY+5Ea2uJyftX5egvBVPOYOTWTx1/Z+a9dM1zZTwQFXi9XbWhsqjdhA3nivTU Cgr9yUjWPBKRLTkSjlfHokQqJxGxYiH+Bx133W0soKeiqVgNK4fh1+rQ4HtTqgGBPnrn 0091Us6gny6iAwkcwk+KhJXs7UxSZxrQUokp/jjlUmqK6DIA4t8GLVV3+eTCSz9goKA5 sQ1Rp4IKMNowwBglE81ko9p6odS2IIe28hrd+/vS8aUI6H6rPoOI9sqNVc/mgT7vnvrA IiPP8V5XWvLiNRZgE8vP1iHH9VUlENUw1iM6f0TCw114Nk4PK3YSWTlDocLTh/yemWFK AJVg== X-Received: by 10.66.181.136 with SMTP id dw8mr2807311pac.117.1421213145812; Tue, 13 Jan 2015 21:25:45 -0800 (PST) Received: from [172.20.10.2] ([172.56.40.24]) by mx.google.com with ESMTPSA id ku12sm18714862pab.39.2015.01.13.21.25.43 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 13 Jan 2015 21:25:45 -0800 (PST) Subject: Re: ChaCha8/12/20 and GEOM ELI tests Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Content-Type: multipart/signed; boundary="Apple-Mail=_E97B72A1-5F74-45E8-A7F5-D4A9E766D7DB"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5b4 From: Alexey Ivanov In-Reply-To: <20150114041708.GA3189@reks> Date: Tue, 13 Jan 2015 21:25:36 -0800 Message-Id: <29DB9466-3DF9-4191-9476-C46BF350848D@gmail.com> References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> <20150114041708.GA3189@reks> To: rozhuk.im@gmail.com, Gleb Kurtsou X-Mailer: Apple Mail (2.1993) Cc: freebsd-hackers@freebsd.org, Adam Nowacki , freebsd-geom@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 05:25:46 -0000 --Apple-Mail=_E97B72A1-5F74-45E8-A7F5-D4A9E766D7DB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jan 13, 2015, at 8:17 PM, Gleb Kurtsou wrote: >=20 > On (14/01/2015 05:21), rozhuk.im@gmail.com wrote: >>> Maybe faster but a stream cipher is unusable for disk encryption - = iv >>> is derived from sector number and doesn't change. Being able to = write a >>> known plaintext and read resulting ciphertext allows you to recover = the >>> cipher stream and decrypt any past or future data stored on that >>> sector. >>=20 >> Depends on the capabilities of the attacker. >>=20 >> To be able to continuously read encrypted sectors for data collection >> is too much. >=20 > I disagree. It's the most basic attack scenario. Assuming attacker was > able to get access to encrypted disk once, odds are high it may happen > again. Agreed: In day to day life if anyone had an ability to read content off = laptop=E2=80=99s hdd when it is hibernated - he would break the = encryption. In server world if one has access to two HDDs from the same raid1 from = different points in time - he also will have the ability to decrypt = data. >=20 >=20 >> Ability to read encrypted sectors has a transmission network, for = example when the container=3Ddisk is stored somewhere in the cloud. >>=20 >> In many cases, the attacker gets Encrypted disk along with other = equipment, often in the off state. >> Without encryption keys and the ability to write / read through the = GELI. >>=20 >> I do not see any weaknesses stream ciphers in cases when the attacker = is not able to access the disk when it is mounted in the GEOM GELI. >>=20 >> Another possibility is the use of ChaCha (without XTS) - encryption >> swap file: there every time a new key is generated, besides the speed >> is particularly important. >=20 > Stream cipher (or similarly functioning block cipher mode) should not = be > used for disk encryption. IMHO swap encryption hardly justifies adding > insecure encryption mode to geli. Fast swap is certainly nice to have, > but rather remains a snake oil, system will remain trashed due to > swapping no matter how fast swap is. Agreed again, if one really wants to add stream cipher he should then = securely generate random IV and write it to disk along with the data = (e.g. how g_eli_integrity.c does for HMAC, or even something simpler = since one does not have a requirement of storing IV on the same sector = as data) >=20 >=20 >> These aspects of the application must necessarily be reflected in the = documentation. >>=20 >>=20 >> There are objections to add ChaCha and XChaCha (without XTS) in GEOM = GELI? >=20 > I strongly object. Yep, though having ChaCha as a replacement for arc4 in both = userland[1][2] and kernel would be nice. [1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D182610 [2] http://marc.info/?l=3Dopenbsd-tech&m=3D141807224826859&w=3D2 >=20 > Having XTS mode for a stream cipher in the first place looks really = fishy.. >=20 > Originally encryption is defined as: > T =3D AES-ENC(key2, i) (*) alpha_j > PP =3D P (*) T > CC =3D AES-ENC(key1, PP) > C =3D CC (*) T >=20 > In stream cipher case: > T =3D CHACHA(key2, state2) (*) i (*) alpha_j > PP =3D P (*) T > CC =3D CHACHA(key1, state1) (*) PP > C =3D CC (*) T >=20 > CC =3D CHACHA(key1, state1) (*) P (*) T > C =3D CC (*) T >=20 > C =3D CHACHA(key1, state1) (*) P >=20 > It doesn't depend on i, j or key2. state1 should be the same as well. >=20 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to = "freebsd-hackers-unsubscribe@freebsd.org" --Apple-Mail=_E97B72A1-5F74-45E8-A7F5-D4A9E766D7DB Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJUtf3VAAoJECvXQw+IBr2abfcP/Ag+K0OOlD63DGyNcM00bQMs BX130Ek+km9tPCgRNTEVMWTgcSET0FHLv/UcdLmuQT7nzPk8bBgMUpL4cMfG+G/x w4e86yGF6ltpYTHDzUdxmjjUC2udzN7wUEesZg/DEOUJdSSEYFLXEvwSk/YxCo9J qnVKf9mbkBIyaIpfkb2i9uF3LE3KA4whpqhVMKV0RpiZ14uY6thj3SGw/l+X/64q 8fAKcWdkGleVyYiR72Hd+vpCMxqnBeVMzMGXd6RxpRZh0jdZvJ1RDOCk1t5Fuoly OdYzd31b8OUbGWxzCv6/6CVY4qPi6LaTAYWTu04M6rddvgFYfc1Yz6eSRaI4w8Bi 1gUi87OSHFG6EsUkpgXDMjhqKNKn484HKsFGtzO3AdGwaQNE0yrHDiePyk8Afych 0aOTVD8RzGGYdvimubuTD9jd6ud8LVXr+Pce62LYbX7RfGhUYiuKmx8siYRrnItO byeZDeBb/WMYh3AngDGF/aeG6yqbf9BALCvmMfUHohzPyQHOKIlt4iLZtigDNe7K dWyN/iNj0ZpEC9cwKyJVYItN2HhboonIsM/Fk8w1nGtrQWR080udweWHH4jMnint hr6CBdF1VyE7Di3+iCmGDZyy1njA1WqvLf3v+rJKhkvrVwVXAbWyHOvlgQ8vNfSz 1VJXu14iAlmxhNtsYAVT =WSPn -----END PGP SIGNATURE----- --Apple-Mail=_E97B72A1-5F74-45E8-A7F5-D4A9E766D7DB-- From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 05:43:11 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A4719C5B; Wed, 14 Jan 2015 05:43:11 +0000 (UTC) Received: from mail-lb0-x22b.google.com (mail-lb0-x22b.google.com [IPv6:2a00:1450:4010:c04::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3B83B9B0; Wed, 14 Jan 2015 05:43:11 +0000 (UTC) Received: by mail-lb0-f171.google.com with SMTP id w7so6241394lbi.2; Tue, 13 Jan 2015 21:43:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:reply-to:to:cc:references:in-reply-to:subject:date :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=VL3zogd6IIJQCqzWvDQ0xP4E93l/WLJ2vd2SXm6nLsk=; b=WjiINAYC1SXt4oMe++fGZUzXWunfmn79z701spip7HFy1HkJO5nrM9xqJtu1+LU13q 2LA4sAzvrRzI9kp5elS+iqeX97t3z6MHsFKSzaZbJASwsc7uwbIeilxKRe/11LwmsHyw 3H7uUEjW6Ks2xN7AjRaf//D1uyzXVpaghOWgFRRTYYHwoJk9BEwtZbY4SI/LkTH58yeZ QEUyOTIZsH4iXSyUkXCxhRKYu/oDpalWBTo8JuIcDU//umVbbZ0Y2Ifw01+869ba4Tdr bclqz0+E7Ql04x4w2LAH/jsZnaeXWvosaKDh49yZwIODBHijceokpcIRvsk5ff5uP8ae 9ttA== X-Received: by 10.112.160.33 with SMTP id xh1mr2047293lbb.60.1421214189260; Tue, 13 Jan 2015 21:43:09 -0800 (PST) Received: from rimwks1w7x64 ([2001:470:1f15:8e:b007:2759:7397:9491]) by mx.google.com with ESMTPSA id r5sm1427772lae.34.2015.01.13.21.43.07 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 13 Jan 2015 21:43:08 -0800 (PST) Message-ID: <54b601ec.0515980a.0c9c.47e1@mx.google.com> X-Google-Original-Message-ID: <028f01d02fbc$f98e6400$ecab2c00$@IM@gmail.com> From: rozhuk.im@gmail.com X-Google-Original-From: Reply-To: To: "'Gleb Kurtsou'" References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> <20150114041708.GA3189@reks> In-Reply-To: <20150114041708.GA3189@reks> Subject: RE: ChaCha8/12/20 and GEOM ELI tests Date: Wed, 14 Jan 2015 08:43:05 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdAvsM8oThNDkVedQwqTv+5SFSD+pwACGN9Q Content-Language: ru Cc: freebsd-hackers@freebsd.org, freebsd-geom@FreeBSD.org, 'Adam Nowacki' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 05:43:11 -0000 > On (14/01/2015 05:21), rozhuk.im@gmail.com wrote: > > > Maybe faster but a stream cipher is unusable for disk encryption - > > > iv is derived from sector number and doesn't change. Being able to > > > write a known plaintext and read resulting ciphertext allows you = to > > > recover the cipher stream and decrypt any past or future data > stored > > > on that sector. > > > > Depends on the capabilities of the attacker. > > > > To be able to continuously read encrypted sectors for data = collection > > is too much. >=20 > I disagree. It's the most basic attack scenario. Assuming attacker was > able to get access to encrypted disk once, odds are high it may happen > again. =20 >From different types of threats, there are various means of protection. AES-XTC more universal solution, encryption ChaSha more specialized. Each solution has its pluses and minuses. AES too slow for me, and options when someone magically going to read my = encrypted disk excluded. In my particular case. I think there will a lot of cases when you need the maximum speed of = encryption and all the other factors are not important or unlikely. For example, the value of the data may be too low to take the time to = decrypt, and the data can be very much and you want to write them down = quickly. =20 > > Ability to read encrypted sectors has a transmission network, for > example when the container=3Ddisk is stored somewhere in the cloud. > > > > In many cases, the attacker gets Encrypted disk along with other > equipment, often in the off state. > > Without encryption keys and the ability to write / read through the > GELI. > > > > I do not see any weaknesses stream ciphers in cases when the = attacker > is not able to access the disk when it is mounted in the GEOM GELI. > > > > Another possibility is the use of ChaCha (without XTS) - encryption > > swap file: there every time a new key is generated, besides the = speed > > is particularly important. >=20 > Stream cipher (or similarly functioning block cipher mode) should not > be used for disk encryption. IMHO swap encryption hardly justifies > adding insecure encryption mode to geli. Fast swap is certainly nice = to > have, but rather remains a snake oil, system will remain trashed due = to > swapping no matter how fast swap is. Can you describe what is weak encryption swap with ChaCha? > > These aspects of the application must necessarily be reflected in = the > documentation. > > > > > > There are objections to add ChaCha and XChaCha (without XTS) in GEOM > GELI? >=20 > I strongly object. Adam Nowacki already explained why XTS can not be used with ChaCha, I = realized my mistake. Now talking about ChaCha/XChaCha without(!!!) XTS. From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 05:56:56 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D36B6E69; Wed, 14 Jan 2015 05:56:56 +0000 (UTC) Received: from mail-lb0-x236.google.com (mail-lb0-x236.google.com [IPv6:2a00:1450:4010:c04::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 472D8AB4; Wed, 14 Jan 2015 05:56:56 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id u10so6185897lbd.13; Tue, 13 Jan 2015 21:56:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:reply-to:to:cc:references:in-reply-to:subject:date :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=iinTKq9P0BNrvbo0UngKpgmoTvC9PwFYVrUmOXS/paw=; b=oU/hZLzxFWSZaT2fCdR+yNix4El7BwKxpMTJrCJCFD0jaqjEogUDhDt4XZCLsHg8V8 ibGORoegbrAUKZnAgeLONrEPJneGTd38BX5n8qlEi5pu/nPGp9+G51MWS267zInHc6+l imkAaZYjoz9aPrI8eVEMEvKVvuDMhXNZBewnYPzUWtWk7BqUB6IgSqpvWFkaA+HXeQyR zIWsfih84GrRT/Tj68v1WJqvWT91LN/zF/cIZZIqf4sa6oFaJJ1K6WUGMHl70HBENMWX jeBped2Sx60iFiYn5n5RIz6+igMbJcGbp4gMuLod8d3AZkEgCBsexUkpKBrZzO3kazdc 3qrg== X-Received: by 10.112.181.106 with SMTP id dv10mr2040384lbc.88.1421215014337; Tue, 13 Jan 2015 21:56:54 -0800 (PST) Received: from rimwks1w7x64 ([2001:470:1f15:8e:b007:2759:7397:9491]) by mx.google.com with ESMTPSA id o13sm1444094laa.16.2015.01.13.21.56.52 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 13 Jan 2015 21:56:53 -0800 (PST) Message-ID: <54b60525.2d08980a.437b.4760@mx.google.com> X-Google-Original-Message-ID: <029001d02fbe$e5700580$b0501080$@IM@gmail.com> From: rozhuk.im@gmail.com X-Google-Original-From: Reply-To: To: "'Alexey Ivanov'" , "'Gleb Kurtsou'" References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> <20150114041708.GA3189@reks> <29DB9466-3DF9-4191-9476-C46BF350848D@gmail.com> In-Reply-To: <29DB9466-3DF9-4191-9476-C46BF350848D@gmail.com> Subject: RE: ChaCha8/12/20 and GEOM ELI tests Date: Wed, 14 Jan 2015 08:56:50 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdAvuoxrjwn5higZTmycb/oOCeD3dAAAm+QA Content-Language: ru Cc: freebsd-hackers@freebsd.org, 'Adam Nowacki' , freebsd-geom@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 05:56:57 -0000 > >>> Maybe faster but a stream cipher is unusable for disk encryption - > >>> iv is derived from sector number and doesn't change. Being able to > >>> write a known plaintext and read resulting ciphertext allows you = to > >>> recover the cipher stream and decrypt any past or future data > stored > >>> on that sector. > >> > >> Depends on the capabilities of the attacker. > >> > >> To be able to continuously read encrypted sectors for data > collection > >> is too much. > > > > I disagree. It's the most basic attack scenario. Assuming attacker > was > > able to get access to encrypted disk once, odds are high it may > happen > > again. > Agreed: > In day to day life if anyone had an ability to read content off > laptop=E2=80=99s hdd when it is hibernated - he would break the = encryption. > In server world if one has access to two HDDs from the same raid1 from > different points in time - he also will have the ability to decrypt > data. While the laptop is sleeping from the dump, you can get any encryption = keys the mounted disk, no matter what they are encrypted. With servers agree, but I specifically focus on the conditions of = application of ChaCha. > > Stream cipher (or similarly functioning block cipher mode) should = not > > be used for disk encryption. IMHO swap encryption hardly justifies > > adding insecure encryption mode to geli. Fast swap is certainly nice > > to have, but rather remains a snake oil, system will remain trashed > > due to swapping no matter how fast swap is. > Agreed again, if one really wants to add stream cipher he should then > securely generate random IV and write it to disk along with the data > (e.g. how g_eli_integrity.c does for HMAC, or even something simpler > since one does not have a requirement of storing IV on the same sector > as data) ChaCha perfectly suitable where speed is needed, but you can go to some = risks. I do not see the possibility of an attacker can decrypt ChaCha if he is = unable to read the encrypted data in a different time and compare them = with the plain text. ChaCha is not a silver bullet when encrypting drives, but its area of = application it has. From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 07:21:30 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99AB0DB2; Wed, 14 Jan 2015 07:21:30 +0000 (UTC) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 102AF2DD; Wed, 14 Jan 2015 07:21:30 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id z11so6472311lbi.10; Tue, 13 Jan 2015 23:21:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:reply-to:to:cc:references:in-reply-to:subject:date :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=CDAHbmVJTfZf6aWn6zmWXm9ZYT+7uzHD+QIJmAr/fnk=; b=CX0J6YdNVWDKbG2/26nKfc9SEhTLMSLtvAOzKexoXhgCLOWyMRjmHY5z9uxiy0X3BG JWh5xTO//soxKhv8Hj42fh+cYJNM0cHrqr2ulkW0HYzHgP5W5VbrbN58rrg5kzVDD+Ha HAYSFcPOenkZN59U4giGcnk0SDlC5cLtRxO6mvwCZYLme5cSMTHgDvW9DMEs1ur6qqwi HjKbLfBN8BB06jkB0UNLpP3AFY/e78RYbPOI95LBUMXCEEIa74rF30cpZfsKxA4OGhiG bAOq0lUU1hu+KPaAaeEx50O1kHecSoLFHXq/bVnH0ocGzuNTDFuqIs1K8/g54AP3Oj4P qPbg== X-Received: by 10.152.27.228 with SMTP id w4mr2232368lag.75.1421220087861; Tue, 13 Jan 2015 23:21:27 -0800 (PST) Received: from rimwks1w7x64 ([2001:470:1f15:8e:b007:2759:7397:9491]) by mx.google.com with ESMTPSA id ba3sm5625307lbc.35.2015.01.13.23.21.25 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 13 Jan 2015 23:21:26 -0800 (PST) Message-ID: <54b618f6.43ac700a.3509.2eae@mx.google.com> X-Google-Original-Message-ID: <029d01d02fca$b5718780$20549680$@IM@gmail.com> From: rozhuk.im@gmail.com X-Google-Original-From: Reply-To: To: "'John-Mark Gurney'" References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <20150112072249.GM1949@funkthat.com> <54b43144.2d08980a.437b.0f8f@mx.google.com> <20150112233411.GP1949@funkthat.com> In-Reply-To: <20150112233411.GP1949@funkthat.com> Subject: RE: ChaCha8/12/20 and GEOM ELI tests Date: Wed, 14 Jan 2015 10:21:24 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdAuwEdie9wllU8ERNCixG4fdnBU9gAAR2JQ Content-Language: ru Cc: freebsd-hackers@freebsd.org, freebsd-geom@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 07:21:30 -0000 I've updated the patch. Deleted XTC mode. ChaCha/XChaCha added to GELI. http://netlab.linkpc.net/download/software/FreeBSD/patches/chacha.patch > > > Also, where are the man page diffs? They might have explained the > > > difference between the two, and explained why two versions of > chacha > > > are needed... > > > > No man page diffs. > > You need to document the new defines in crypto(9), and document the > various parameters in crypto(7)... Yes, not all modes are documented > in crypto(7), but going forward, at a minimum we need to document new > additions... > > I'll admit I didn't document the other algorithms as I'm not as familar > w/ those as the ones that I worked one... Agree. > > Man pages does not explain difference between AES-CBC and AES-XTS... > > True, but CBC and XTS (which includes a reference to the standard) are > a lot more searchable/common knowlege than xchacha.. google thinks you > mean chacha, and xchacha just turns up a bunch of people on various > networks... Not until you search on xchacha crypto do you get a > relevant page... Also, wikipedia doesn't have an entry for xchacha, > nor does the chacha (cipher) page list it... So, when documenting > xchacha in crypto(7), include a link to the description/standard... Agree. > > > Is there a reason you decided to write your own ChaCha > > > implementation instead of using one of the standard ones? Did you > > > run performance tests between your implementation and others? > > > > Reference ChaCha and reference (FreeBSD) XTS (4k sector): > > ChaCha8-XTS-256 = 199518722 bytes/sec > > ChaCha12-XTS-256 = 179029849 bytes/sec > > ChaCha20-XTS-256 = 149447317 bytes/sec > > XChaCha8-XTS-256 = 195675728 bytes/sec > > XChaCha12-XTS-256 = 175790196 bytes/sec > > XChaCha20-XTS-256 = 147939263 bytes/sec > > So, you're seeing a 33%-50% improvement, good to hear... > > Also, do you publish this implementation somewhere? If so, it'd be > helpful to include a url to where up to date versions can be > obtained... > If you don't plan on publishing/maintaining it outside of FreeBSD, then > we need to unifdef out the Windows parts of it for our tree... On my own site: http://www.netlab.linkpc.net/download/software/SDK/core/include/chacha.h (working copy) This is not FreeBSD kernel specific, I also test it under Windows - 32 bit and FreeBSD user space. geli (user space) also use this code to encrypt/decrypt password/metadata. > > This is the reference version adapted for use in /dev/crypto. > > chacha_block_unaligneg() - processing the reference version of a data > block. > > Macros are used for readability. > > chacha_block_aligned() - the same but the work on the aligned data. > > Please use the macro __NO_STRICT_ALIGNMENT to decide if special work is > necessary to handle the alignment... I`m already use ALIGNED_POINTER() macro. > What is the CHACHA_X64 macro for? If that is to detect LP64 platforms, > please use the macro __LP64__ to decide this... Have you done > performance evaluations on 32bit arches to make sure double rounds > aren't a benefit there too? __LP64__ - done. I run self test on x32, all passed Ok. No speed degradation. > Use the byteorder(9) macros to encode/decode integers instead of > rolling your own (U8TO32_LITTLE and U32TO8_LITTLE)... Turns out > compilers aren't good at optimizing this type of code, and platforms > may have assembly optimized versions for these... 1. U8TO32_LITTLE / U32TO8_LITTLE can read/write unaligned data. Can htonl() handle unaligned input on arm? 2. On LE systems no conversion required. > > To increase speed, instead of one byte is processed for 4/8 byte > times. > > The data in the context of an 8-byte aligned. > > To increase security, all data, including temporary, saved in a > > context that on completion of the work is filled with zeros. > > Please use the function explicite_bzero that is available for all of > these instead of creating your own.. explicite_bzero() available only in FreeBSD kernel space. I`m use bzero() in chacha_zerokey() / xchacha_zerokey() as all other ***_zerokey() functions in this file. > > Final: > > > > struct aes_xts_ctx { > > rijndael_ctx key1; > > rijndael_ctx key2; > > uint64_t tweak[(AES_XTS_BLOCKSIZE / sizeof(uint64_t))]; }; > > > > void > > aes_xts_reinit(caddr_t key, u_int8_t *iv) { > > struct aes_xts_ctx *ctx = (struct aes_xts_ctx *)key; > > > > /* > > * Prepare tweak as E_k2(IV). IV is specified as LE > representation > > * of a 64-bit block number which we allow to be passed in > directly. > > */ > > if (ALIGNED_POINTER(iv, uint64_t)) { > > ctx->tweak[0] = (*((uint64_t*)(void*)iv)); > > } else { > > bcopy(iv, ctx->tweak, sizeof(uint64_t)); > > } > > /* Convert to LE. */ > > ctx->tweak[0] = htole64(ctx->tweak[0]); > > Hmm... this line bothers me.. I'll need to spend more time reading up > to decide if it is buggy or not... Is ctx->tweak in host order? or LE > order? I believe it's suppose to be LE order, as it gets passed > directly to _encryt.. I'm also not sure if the original code is BE > clean, which is part of my problem... I hope to see an optimized version soon to 10x :) From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 08:19:14 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A7F708D4; Wed, 14 Jan 2015 08:19:14 +0000 (UTC) Received: from mail-pd0-x22f.google.com (mail-pd0-x22f.google.com [IPv6:2607:f8b0:400e:c02::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 70C3FAED; Wed, 14 Jan 2015 08:19:14 +0000 (UTC) Received: by mail-pd0-f175.google.com with SMTP id g10so8469922pdj.6; Wed, 14 Jan 2015 00:19:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-type:content-disposition :in-reply-to:user-agent; bh=z39auAyU1xAIfcO+9x6ZRvEFBILqgI+UTjBpEL6MgQ4=; b=MMuKIXXXI01+z0Q8wCazepjkdzNr2jrkWZMl/zZLjh4abnAbedbCuPcVGYcA5ckBtG v2lAnsJ/eD0mNRNEf2due18vHsSIO0DYBMsCiiIeJ43tak1QrXK/Yyomixpq9+KTBmmx 1JcN4MZDWJMxaFPszNzOHtqMNU5hpdUM+3QBPxRIcFeyllgq+Pih7obNkKQs6dnb/LJa 5Z/4YClMQnZdhWuxhfuMk9lVQEk7Gb9VuAOPkCqlzX21j6UdRs0R1nYVagMrktfZT3+Z fRjydn8lUSUPqzhVolEC6xQInHV+ymwO7sKtEwDr/N3yAG6DuMp5REOGvVfqvgAXUFWX +daA== X-Received: by 10.66.138.72 with SMTP id qo8mr3962018pab.84.1421223553973; Wed, 14 Jan 2015 00:19:13 -0800 (PST) Received: from localhost (c-76-21-76-83.hsd1.ca.comcast.net. [76.21.76.83]) by mx.google.com with ESMTPSA id ig1sm18952791pbc.41.2015.01.14.00.19.12 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Jan 2015 00:19:13 -0800 (PST) Sender: Gleb Kurtsou Date: Wed, 14 Jan 2015 00:20:19 -0800 From: 'Gleb Kurtsou' To: rozhuk.im@gmail.com Subject: Re: ChaCha8/12/20 and GEOM ELI tests Message-ID: <20150114082019.GA3669@reks> Mail-Followup-To: rozhuk.im@gmail.com, 'Adam Nowacki' , freebsd-hackers@freebsd.org, freebsd-geom@FreeBSD.org References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> <20150114041708.GA3189@reks> <54b601ec.0515980a.0c9c.47e1@mx.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <54b601ec.0515980a.0c9c.47e1@mx.google.com> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-hackers@freebsd.org, freebsd-geom@FreeBSD.org, 'Adam Nowacki' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 08:19:14 -0000 On (14/01/2015 08:43), rozhuk.im@gmail.com wrote: > > On (14/01/2015 05:21), rozhuk.im@gmail.com wrote: > > > > Maybe faster but a stream cipher is unusable for disk encryption - > > > > iv is derived from sector number and doesn't change. Being able to > > > > write a known plaintext and read resulting ciphertext allows you to > > > > recover the cipher stream and decrypt any past or future data > > stored > > > > on that sector. > > > > > > Depends on the capabilities of the attacker. > > > > > > To be able to continuously read encrypted sectors for data collection > > > is too much. > > > > I disagree. It's the most basic attack scenario. Assuming attacker was > > able to get access to encrypted disk once, odds are high it may happen > > again. > > From different types of threats, there are various means of protection. > AES-XTC more universal solution, encryption ChaSha more specialized. > Each solution has its pluses and minuses. "various means of protection" is rather confusing. It's either acceptable for disk encryption or not. > AES too slow for me, and options when someone magically going to read my encrypted disk excluded. > In my particular case. It's widely accepted threat model (as was also mentioned elsewhere). So it's rather your particular use case that has magic vibe to it. > I think there will a lot of cases when you need the maximum speed of encryption and all the other factors are not important or unlikely. > > For example, the value of the data may be too low to take the time to decrypt, and the data can be very much and you want to write them down quickly. > > > > > Ability to read encrypted sectors has a transmission network, for > > example when the container=disk is stored somewhere in the cloud. > > > > > > In many cases, the attacker gets Encrypted disk along with other > > equipment, often in the off state. > > > Without encryption keys and the ability to write / read through the > > GELI. > > > > > > I do not see any weaknesses stream ciphers in cases when the attacker > > is not able to access the disk when it is mounted in the GEOM GELI. > > > > > > Another possibility is the use of ChaCha (without XTS) - encryption > > > swap file: there every time a new key is generated, besides the speed > > > is particularly important. > > > > Stream cipher (or similarly functioning block cipher mode) should not > > be used for disk encryption. IMHO swap encryption hardly justifies > > adding insecure encryption mode to geli. Fast swap is certainly nice to > > have, but rather remains a snake oil, system will remain trashed due to > > swapping no matter how fast swap is. > > Can you describe what is weak encryption swap with ChaCha? Stream cipher, as is, is not acceptable for disk encryption. I think we all agree on that. It remains secure as soon as the same key stream is not used twice. By writing data to the same sector one breaks this requirement. I'm not objecting that there is a possibility of building swap encryption on top of stream cipher, but that is likely not to be trivial and would require careful design review. Although I don't see how block device "encrypted" with stream cipher can be secure under assumption that attacker has access to the disk. > > > These aspects of the application must necessarily be reflected in the > > documentation. > > > > > > > > > There are objections to add ChaCha and XChaCha (without XTS) in GEOM > > GELI? > > > > I strongly object. > > Adam Nowacki already explained why XTS can not be used with ChaCha, I realized my mistake. > Now talking about ChaCha/XChaCha without(!!!) XTS. Badly broken for me :) BTW You may want to get rid of SHA in g_eli_crypto_ivgen() for ChaCha in pursue for yet higher performance. As far as I remember Salsa20 has this nice property of letting one start encryption from arbitrary offset in the stream, I assumed that you are using the same in ChaCha for geli. Although having looked at the code it is not that obvious what is going on. There is a method to set iv being used, but not to change counter (stream offset). Is that intentional? Do we reset key for each geli sector to be encrypted? P.S. My objection should only be considered merely my personal opinion. You might have better luck contacting security team directly. From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 08:43:52 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0C905E72; Wed, 14 Jan 2015 08:43:52 +0000 (UTC) Received: from mail-pd0-x22d.google.com (mail-pd0-x22d.google.com [IPv6:2607:f8b0:400e:c02::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C7255DBB; Wed, 14 Jan 2015 08:43:51 +0000 (UTC) Received: by mail-pd0-f173.google.com with SMTP id ft15so8579762pdb.4; Wed, 14 Jan 2015 00:43:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-type:content-disposition :in-reply-to:user-agent; bh=yoz/1e1S/3kpIPEi7IupduhcYFDnsMd1Qv+/D3/Il+0=; b=DWtA4+sRpSzpHZKKyxnEQw8iq7NnnX4MHPDn5mue7ftWKAYbJjZY+/pUG62Gv5dqJd GL1FJ4kuqe8BMCdXaLG0y+WsuhInywzxH/uCg9sp0I8YKSQIjdA0l4W2muaqaKItyxBN 3RkS6FntONBcDvJAXxWqHeSFzbsR3nciEOWi0K3QK1xIiPhh8i2GDJf9Iq1+yxDxVKHg CRWZXolZPhkYs0OJSW7qCd/6qAnPMD7KYu/gFAmP5GFx+X+7ZZsop+gvYrsUkC8Q6wAB fNIafXuBMkigVArq7gQUebCWnfBzt0wwTUO30Qs2Q5Sn2g/h4Z0ou1CeIYHmsTc2wlpp T2OA== X-Received: by 10.70.31.197 with SMTP id c5mr3873477pdi.93.1421225031418; Wed, 14 Jan 2015 00:43:51 -0800 (PST) Received: from localhost (c-76-21-76-83.hsd1.ca.comcast.net. [76.21.76.83]) by mx.google.com with ESMTPSA id ma7sm19097669pdb.53.2015.01.14.00.43.50 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Jan 2015 00:43:50 -0800 (PST) Sender: Gleb Kurtsou Date: Wed, 14 Jan 2015 00:44:57 -0800 From: Gleb Kurtsou To: rozhuk.im@gmail.com Subject: Re: ChaCha8/12/20 and GEOM ELI tests Message-ID: <20150114084457.GA4099@reks> Mail-Followup-To: rozhuk.im@gmail.com, 'Kimmo Paasiala' , 'FreeBSD Hackers' , freebsd-geom@freebsd.org, 'Adam Nowacki' References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> <54b5fbbe.4457700a.2456.6944@mx.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <54b5fbbe.4457700a.2456.6944@mx.google.com> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: 'Kimmo Paasiala' , freebsd-geom@freebsd.org, 'Adam Nowacki' , 'FreeBSD Hackers' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 08:43:52 -0000 On (14/01/2015 08:16), rozhuk.im@gmail.com wrote: > > >> Depends on the capabilities of the attacker. > > >> > > >> To be able to continuously read encrypted sectors for data > > collection is too much. > > >> > > > > > When talking about disk encryption the first assumption is that the > > attacker always has this capability, even with so much power the > > attacker shouldn't be able to break the encryption scheme. If he can > > then the encryption scheme is not secure. > > > > Ift the attacker can learn anything about the unencrypted data or > > predict something about future encrypted or unencrypted blocks by > > analyzing the previous encrypted blocks the encryption scheme should > > be considered insecure. > > I consider the case when the disk can be obtained by physically an attacker. > All the rest of the disk directly connected to the computer. Do you imply that scheme you propose will not provide support for network storage? What about HAST, iSCSI, ATA over Ethernet, Cinder or whatever else is fancy those days? Documenting such limitation would make geli look terrible, not to mention confusing.. > If an attacker can read encrypted data directly to disk means that the > system is already compromised by an attacker, Unless you don't have knowledge of attacker possessing such power :) I would happily throw my encrypted disk away every time I find out it was tempered with offline. The good thing I never know whether is has happened (good way to save some money as well). That is why assumption that attacker has access to encrypted disk should hold. In case of block device encryption overall situation is much worse. Imagine you find out that your geli keys were compromised. There is no mechanism provided to switch to new encryption key while retaining access to old data and changing key for new data without creating full copy. > and probably in this > case can read the data from the disk and through read() already > decrypted and get the key from the kernel memory. From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 17:58:40 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E5E243D5; Wed, 14 Jan 2015 17:58:39 +0000 (UTC) Received: from mail-la0-x22d.google.com (mail-la0-x22d.google.com [IPv6:2a00:1450:4010:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3808E2D3; Wed, 14 Jan 2015 17:58:39 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id gq15so9528137lab.4; Wed, 14 Jan 2015 09:58:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:reply-to:to:cc:references:in-reply-to:subject:date :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=aGXqeB/y+2qGafeOfRLg2okfKjb1u8r5XpPstzGmoPU=; b=qdJZnMnbD3r9yApWmXa0cHH8hQFVaG0uM92wdAIpyRBJMJ13pbcwdmRsQx7FMfoM3K PkSPhlpBiaqdiBh6hFye/Ko7L8tY8oPTemSxC6QPAiVMFf0hazGb4nj5zb4eC9KAkPQO qRJIibdRTJyucfPKFzuTybbYs4XuZtOU9QvPkqD6brr0bBoh8xwOMGZvpJozKTaK7wsG /ic4ISWZJgOJmwv8TwXEOx0sDR6bMHB+Kqo1aD+NT02lBXqFe/jV9m5sSOx0+rIiO7zS EJ4WcTvkLylOp3ExcsfFthAKpaeOkz7Zc7owzCKHqJ6m7dnevAqRh2LL86yrRIIFTt9a MzVw== X-Received: by 10.152.44.167 with SMTP id f7mr5386219lam.92.1421258317214; Wed, 14 Jan 2015 09:58:37 -0800 (PST) Received: from rimwks1w7x64 ([2001:470:1f15:8e:b007:2759:7397:9491]) by mx.google.com with ESMTPSA id ci9sm1966982lad.21.2015.01.14.09.58.34 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 14 Jan 2015 09:58:36 -0800 (PST) Message-ID: <54b6ae4c.0905990a.6c9c.642e@mx.google.com> X-Google-Original-Message-ID: <029e01d03023$b7c56520$27502f60$@IM@gmail.com> From: rozhuk.im@gmail.com X-Google-Original-From: Reply-To: To: "'Gleb Kurtsou'" References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> <20150114041708.GA3189@reks> <54b601ec.0515980a.0c9c.47e1@mx.google.com> <20150114082019.GA3669@reks> In-Reply-To: <20150114082019.GA3669@reks> Subject: RE: ChaCha8/12/20 and GEOM ELI tests Date: Wed, 14 Jan 2015 20:58:33 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdAv0siqUUBsp0hPTjWoxAcK7FDFgwASmyEA Content-Language: ru Cc: freebsd-hackers@freebsd.org, freebsd-geom@FreeBSD.org, 'Adam Nowacki' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 17:58:40 -0000 > > > > > Maybe faster but a stream cipher is unusable for disk > encryption > > > > > - iv is derived from sector number and doesn't change. Being > > > > > able to write a known plaintext and read resulting ciphertext > > > > > allows you to recover the cipher stream and decrypt any past = or > > > > > future data > > > stored > > > > > on that sector. > > > > > > > > Depends on the capabilities of the attacker. > > > > > > > > To be able to continuously read encrypted sectors for data > > > > collection is too much. > > > > > > I disagree. It's the most basic attack scenario. Assuming attacker > > > was able to get access to encrypted disk once, odds are high it = may > > > happen again. > > > > From different types of threats, there are various means of > protection. > > AES-XTC more universal solution, encryption ChaSha more specialized. > > Each solution has its pluses and minuses. >=20 > "various means of protection" is rather confusing. It's either > acceptable for disk encryption or not. =20 This is acceptable for disk encryption in certain conditions: need speed = and have the assurance that an attacker does not have access to the data = flow while the disk is mounted. > > AES too slow for me, and options when someone magically going to = read > my encrypted disk excluded. > > In my particular case. >=20 > It's widely accepted threat model (as was also mentioned elsewhere). = So > it's rather your particular use case that has magic vibe to it. In my case, home use, my data is not so important to arrange a targeted = attack on the home server. Moreover, in the case of targeted attack, there is often much easier = ways to get the unencrypted data than analyzing encrypted disk sectors. I need to encrypt the case if the server can steal or take by force. So I think that for me personally for my purposes ChaCha is more = appropriate than AES-XTS. I am sure many other users will also be happy to get a faster encryption = in exchange for the possibility of targeted attacks unlikely. > > > > Ability to read encrypted sectors has a transmission network, = for > > > example when the container=3Ddisk is stored somewhere in the = cloud. > > > > > > > > In many cases, the attacker gets Encrypted disk along with other > > > equipment, often in the off state. > > > > Without encryption keys and the ability to write / read through > > > > the > > > GELI. > > > > > > > > I do not see any weaknesses stream ciphers in cases when the > > > > attacker > > > is not able to access the disk when it is mounted in the GEOM = GELI. > > > > > > > > Another possibility is the use of ChaCha (without XTS) - > > > > encryption swap file: there every time a new key is generated, > > > > besides the speed is particularly important. > > > > > > Stream cipher (or similarly functioning block cipher mode) should > > > not be used for disk encryption. IMHO swap encryption hardly > > > justifies adding insecure encryption mode to geli. Fast swap is > > > certainly nice to have, but rather remains a snake oil, system = will > > > remain trashed due to swapping no matter how fast swap is. > > > > Can you describe what is weak encryption swap with ChaCha? >=20 > Stream cipher, as is, is not acceptable for disk encryption. I think = we > all agree on that. Stream cipher disks acceptable in certain cases. > It remains secure as soon as the same key stream is not used twice. > By writing data to the same sector one breaks this requirement. > > I'm not objecting that there is a possibility of building swap > encryption on top of stream cipher, but that is likely not to be > trivial and would require careful design review. Although I don't see > how block device "encrypted" with stream cipher can be secure under > assumption that attacker has access to the disk. It all has no value if the attacker has no way to get multiple copies of = the same sector with different data. > BTW You may want to get rid of SHA in g_eli_crypto_ivgen() for ChaCha > in pursue for yet higher performance. IV =3D SHA256(key, sector num) ChaCha8-256 =3D 353804761 bytes/sec ChaCha12-256 =3D 299028184 bytes/sec ChaCha20-256 =3D 225262046 bytes/sec XChaCha8-256 =3D 347179985 bytes/sec XChaCha12-256 =3D 289285124 bytes/sec XChaCha20-256 =3D 219787078 bytes/sec IV =3D sector num ChaCha8-256 =3D 376476930 bytes/sec ChaCha12-256 =3D 312266331 bytes/sec ChaCha20-256 =3D 233636775 bytes/sec XChaCha has a 192-bit IV, so I would prefer to get IV from the hash. > As far as I remember Salsa20 has this nice property of letting one > start encryption from arbitrary offset in the stream, I assumed that > you are using the same in ChaCha for geli. Although having looked at > the code it is not that obvious what is going on. There is a method to > set iv being used, but not to change counter (stream offset). Is that > intentional? Do we reset key for each geli sector to be encrypted? chacha_iv_set() does not reset key. xchacha_set_key_iv_rounds() require key and IV. chacha_counter_set() - set block counter. chacha_key_set() - copy key and constant to chacha context. From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 18:27:25 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9050DF5B; Wed, 14 Jan 2015 18:27:25 +0000 (UTC) Received: from mail-lb0-x229.google.com (mail-lb0-x229.google.com [IPv6:2a00:1450:4010:c04::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 06D45868; Wed, 14 Jan 2015 18:27:25 +0000 (UTC) Received: by mail-lb0-f169.google.com with SMTP id p9so9438176lbv.0; Wed, 14 Jan 2015 10:27:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:reply-to:to:cc:references:in-reply-to:subject:date :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=HYyGqleDKtNcRlh6H/2FZTSmOerIrQNjL8pbWVascOE=; b=PXFQ5YYhQaSZ+xcVZ3kgaUQJ2dem5fbvyx28haOtbnTS3D+AFSTgRoKFqdKBDhrHDr gnrS92VitLF52BVdybLlOYgDbtLqskMBsz1GgM0Q2zPBp4XCLehjDfbMkNA8LQyUFiEj HI6LEDE4QdQwrsnHv1a3+45DSe55xb7SkeVrpSfK3Eu+x+0GUQtK/ZWhe5fRtArC6oUK lzK9Bkx2ROaCBzm+tcwrob/SL9Rdk4+xYIygzlcyrH6Uu1VAiTaszeqZGNvYM76icknR D5dRcc/ektgl3g80nTrY14h05hoU4822Z1PLjgVuMkjvey4sryrQqbnZ1+TsMtObSLjg fWeA== X-Received: by 10.112.200.103 with SMTP id jr7mr5600017lbc.0.1421260043084; Wed, 14 Jan 2015 10:27:23 -0800 (PST) Received: from rimwks1w7x64 ([2001:470:1f15:8e:b007:2759:7397:9491]) by mx.google.com with ESMTPSA id mc17sm6131147lbb.1.2015.01.14.10.27.21 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 14 Jan 2015 10:27:22 -0800 (PST) Message-ID: <54b6b50a.d17b700a.5b19.4bd3@mx.google.com> X-Google-Original-Message-ID: <02a601d03027$bcbbb3a0$36331ae0$@IM@gmail.com> From: rozhuk.im@gmail.com X-Google-Original-From: Reply-To: To: "'Gleb Kurtsou'" References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> <54b5fbbe.4457700a.2456.6944@mx.google.com> <20150114084457.GA4099@reks> In-Reply-To: <20150114084457.GA4099@reks> Subject: RE: ChaCha8/12/20 and GEOM ELI tests Date: Wed, 14 Jan 2015 21:27:20 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdAv1jkG5Xl3HuGtSFayFxpfG2E9rgAUXkmw Content-Language: ru Cc: 'Kimmo Paasiala' , freebsd-geom@freebsd.org, 'Adam Nowacki' , 'FreeBSD Hackers' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 18:27:25 -0000 > > > >> Depends on the capabilities of the attacker. > > > >> > > > >> To be able to continuously read encrypted sectors for data > > > collection is too much. > > > >> > > > > > > > When talking about disk encryption the first assumption is that=20 > > > the attacker always has this capability, even with so much power=20 > > > the attacker shouldn't be able to break the encryption scheme. If=20 > > > he > can > > > then the encryption scheme is not secure. > > > > > > Ift the attacker can learn anything about the unencrypted data or=20 > > > predict something about future encrypted or unencrypted blocks by=20 > > > analyzing the previous encrypted blocks the encryption scheme > should > > > be considered insecure. > > > > I consider the case when the disk can be obtained by physically an > attacker. > > All the rest of the disk directly connected to the computer. >=20 > Do you imply that scheme you propose will not provide support for=20 > network storage? What about HAST, iSCSI, ATA over Ethernet, Cinder or=20 > whatever else is fancy those days? >=20 > Documenting such limitation would make geli look terrible, not to=20 > mention confusing.. Provide, but insecure / less secure. :) > > If an attacker can read encrypted data directly to disk means that > the > > system is already compromised by an attacker, >=20 > Unless you don't have knowledge of attacker possessing such power :) God knows everything :) How to protect your data from God? :) > I would happily throw my encrypted disk away every time I find out it=20 > was tempered with offline. The good thing I never know whether is has=20 > happened (good way to save some money as well). >=20 > That is why assumption that attacker has access to encrypted disk=20 > should hold. ChaCha does not suit you. > In case of block device encryption overall situation is much worse. > Imagine you find out that your geli keys were compromised. There is no = > mechanism provided to switch to new encryption key while retaining=20 > access to old data and changing key for new data without creating full = > copy. Technically, you can change the encryption keys geli without creating a = complete copy, but if something happens in the process there is a risk = of losing some data. I think that's why it did not implement. From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 18:44:47 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6D53A56D; Wed, 14 Jan 2015 18:44:47 +0000 (UTC) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D8F55A46; Wed, 14 Jan 2015 18:44:46 +0000 (UTC) Received: by mail-la0-f50.google.com with SMTP id pn19so9692542lab.9; Wed, 14 Jan 2015 10:44:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:reply-to:to:cc:references:in-reply-to:subject:date :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=OVfyOr0RMru1lm8zEOuJnceyOfeOXl32O6+bVdSpbno=; b=N9YNYcAyQlcoQTr7HkCACWtCxjK9YkoxNJLjSSy3eUhc+iALDAn3xedH4z0yYrUwJz r9fLgYQhSbIsX1sqRxkvWPJnsmjU1ygJ0dodWguIAXBLAh6cvNj7WLMdT8QK/F6N4lHU QwSG0pLl7ED/J2gRnxoAnQxZqFgqCxazy0nLcStbhW+UTHPQeL4dmwQnpFm6bi4FhIlg 33I6YQK+X+7Yaztinn5ay3ZxxSdI9073lTC/QMGONFv25i0b4FLCux7X5WV+MEcYmBVZ qg1SK6arTJ/El1zVlOcEMt1FzvXpQB6WVKjudv2blqSgaIx0lL6AiTW4cYd3g9LoMoJd FyTg== X-Received: by 10.112.57.226 with SMTP id l2mr5515037lbq.27.1421261084945; Wed, 14 Jan 2015 10:44:44 -0800 (PST) Received: from rimwks1w7x64 ([2001:470:1f15:8e:b007:2759:7397:9491]) by mx.google.com with ESMTPSA id yf10sm6142040lbb.9.2015.01.14.10.44.42 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 14 Jan 2015 10:44:43 -0800 (PST) Message-ID: <54b6b91b.2aa3700a.3a6c.47b5@mx.google.com> X-Google-Original-Message-ID: <02a701d0302a$2991e010$7cb5a030$@IM@gmail.com> From: rozhuk.im@gmail.com X-Google-Original-From: Reply-To: To: "'Alaksiej'" References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> <20150114041708.GA3189@reks> <54b601ec.0515980a.0c9c.47e1@mx.google.com> <20150114082019.GA3669@reks> <54b6ae4c.0905990a.6c9c.642e@mx.google.com> In-Reply-To: Subject: RE: ChaCha8/12/20 and GEOM ELI tests Date: Wed, 14 Jan 2015 21:44:41 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdAwJPHzQEPWGlhIRJKyDUR4Gd9wQAAAu60w Content-Language: ru Cc: 'Gleb Kurtsou' , 'freebsd-geom' , 'Adam Nowacki' , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 18:44:47 -0000 > Excuse me, but if you think your physical medium is either 100% > inaccessible to an adversary, or simply not worth a real attack, and > the speed is the concern, then why do you want to use any encryption = at > all? 100% is not available yet introduced GELI keys / mounted drive. AES-XTS is good but too slow. ChaCha is already enough to cryptography was not a bottleneck. The case when the disks - local (SATA/SAS/IDE/USB), keys entered / disk = is mounted and the attacker has access I do not see because AES-XTS does = not help. From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 18:14:50 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2BB15C99 for ; Wed, 14 Jan 2015 18:14:50 +0000 (UTC) Received: from mail-ob0-f172.google.com (mail-ob0-f172.google.com [209.85.214.172]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E33196E7 for ; Wed, 14 Jan 2015 18:14:49 +0000 (UTC) Received: by mail-ob0-f172.google.com with SMTP id va8so9436267obc.3 for ; Wed, 14 Jan 2015 10:14:43 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=eFwB8Ww8III+gq/p29kHkxM9QZL+dDv8JUjsnmMM4q0=; b=V0EcapZazdRLGhxsPNmjGV4sgM6kr3BKsdyu2UVHB1+/AQPOXD8is22NfBR4uoqDX8 LuYejC1acOFxuJPj81hqK9nXD6tEIO+bvrAUqtHwkCo98udpdF0CmwSB2qHlcyr38shZ +9HG6QNC43uS5n5esaAGh3RgKjC1SygzbSN3ayCDay2jAiKBOuLelHsgFBROf+E2wZVK QFujrwx3PY3ELQFwFMEB7Q5fF7t5HerHjz75E8ueOMVLTbNo1uLcEUWosuH+YaJ0rdQ/ VbiYgziKroI2yLwB5eHKv+oXoVYm4aY1/4N3k+jOAIvFSK52DNpzKCLi01gn5PnHLu+p j2Ww== X-Gm-Message-State: ALoCoQnASya7Oa4SET9nm8WOyaSboXofCsjJjDJxk/8D/IgtVsOKeGliUcsWJLxeJG8Z5tid2+PS X-Received: by 10.182.215.163 with SMTP id oj3mr3283409obc.49.1421258842439; Wed, 14 Jan 2015 10:07:22 -0800 (PST) MIME-Version: 1.0 Sender: a@carniajeu.com Received: by 10.202.212.71 with HTTP; Wed, 14 Jan 2015 10:07:02 -0800 (PST) X-Originating-IP: [46.53.218.193] In-Reply-To: <54b6ae4c.0905990a.6c9c.642e@mx.google.com> References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> <20150114041708.GA3189@reks> <54b601ec.0515980a.0c9c.47e1@mx.google.com> <20150114082019.GA3669@reks> <54b6ae4c.0905990a.6c9c.642e@mx.google.com> From: Alaksiej Date: Wed, 14 Jan 2015 21:07:02 +0300 X-Google-Sender-Auth: F2KcXXts0ugobisBfH6pn1Aisl4 Message-ID: Subject: Re: ChaCha8/12/20 and GEOM ELI tests To: Rozhuk.IM@gmail.com X-Mailman-Approved-At: Wed, 14 Jan 2015 19:25:12 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Gleb Kurtsou , freebsd-geom , Adam Nowacki , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 18:14:50 -0000 Excuse me, but if you think your physical medium is either 100% inaccessible to an adversary, or simply not worth a real attack, and the speed is the concern, then why do you want to use any encryption at all? Best, Alaksiej On Wed, Jan 14, 2015 at 8:58 PM, wrote: > > > > > > > Maybe faster but a stream cipher is unusable for disk > > encryption > > > > > > - iv is derived from sector number and doesn't change. Being > > > > > > able to write a known plaintext and read resulting ciphertext > > > > > > allows you to recover the cipher stream and decrypt any past or > > > > > > future data > > > > stored > > > > > > on that sector. > > > > > > > > > > Depends on the capabilities of the attacker. > > > > > > > > > > To be able to continuously read encrypted sectors for data > > > > > collection is too much. > > > > > > > > I disagree. It's the most basic attack scenario. Assuming attacker > > > > was able to get access to encrypted disk once, odds are high it may > > > > happen again. > > > > > > From different types of threats, there are various means of > > protection. > > > AES-XTC more universal solution, encryption ChaSha more specialized. > > > Each solution has its pluses and minuses. > > > > "various means of protection" is rather confusing. It's either > > acceptable for disk encryption or not. > > This is acceptable for disk encryption in certain conditions: need speed > and have the assurance that an attacker does not have access to the data > flow while the disk is mounted. > > > > > AES too slow for me, and options when someone magically going to read > > my encrypted disk excluded. > > > In my particular case. > > > > It's widely accepted threat model (as was also mentioned elsewhere). So > > it's rather your particular use case that has magic vibe to it. > > In my case, home use, my data is not so important to arrange a targeted > attack on the home server. > Moreover, in the case of targeted attack, there is often much easier ways > to get the unencrypted data than analyzing encrypted disk sectors. > I need to encrypt the case if the server can steal or take by force. > So I think that for me personally for my purposes ChaCha is more > appropriate than AES-XTS. > > I am sure many other users will also be happy to get a faster encryption > in exchange for the possibility of targeted attacks unlikely. > > > > > > > > Ability to read encrypted sectors has a transmission network, for > > > > example when the container=disk is stored somewhere in the cloud. > > > > > > > > > > In many cases, the attacker gets Encrypted disk along with other > > > > equipment, often in the off state. > > > > > Without encryption keys and the ability to write / read through > > > > > the > > > > GELI. > > > > > > > > > > I do not see any weaknesses stream ciphers in cases when the > > > > > attacker > > > > is not able to access the disk when it is mounted in the GEOM GELI. > > > > > > > > > > Another possibility is the use of ChaCha (without XTS) - > > > > > encryption swap file: there every time a new key is generated, > > > > > besides the speed is particularly important. > > > > > > > > Stream cipher (or similarly functioning block cipher mode) should > > > > not be used for disk encryption. IMHO swap encryption hardly > > > > justifies adding insecure encryption mode to geli. Fast swap is > > > > certainly nice to have, but rather remains a snake oil, system will > > > > remain trashed due to swapping no matter how fast swap is. > > > > > > Can you describe what is weak encryption swap with ChaCha? > > > > Stream cipher, as is, is not acceptable for disk encryption. I think we > > all agree on that. > > Stream cipher disks acceptable in certain cases. > > > > It remains secure as soon as the same key stream is not used twice. > > By writing data to the same sector one breaks this requirement. > > > > I'm not objecting that there is a possibility of building swap > > encryption on top of stream cipher, but that is likely not to be > > trivial and would require careful design review. Although I don't see > > how block device "encrypted" with stream cipher can be secure under > > assumption that attacker has access to the disk. > > It all has no value if the attacker has no way to get multiple copies of > the same sector with different data. > > > > BTW You may want to get rid of SHA in g_eli_crypto_ivgen() for ChaCha > > in pursue for yet higher performance. > > IV = SHA256(key, sector num) > ChaCha8-256 = 353804761 bytes/sec > ChaCha12-256 = 299028184 bytes/sec > ChaCha20-256 = 225262046 bytes/sec > XChaCha8-256 = 347179985 bytes/sec > XChaCha12-256 = 289285124 bytes/sec > XChaCha20-256 = 219787078 bytes/sec > > IV = sector num > ChaCha8-256 = 376476930 bytes/sec > ChaCha12-256 = 312266331 bytes/sec > ChaCha20-256 = 233636775 bytes/sec > > XChaCha has a 192-bit IV, so I would prefer to get IV from the hash. > > > > As far as I remember Salsa20 has this nice property of letting one > > start encryption from arbitrary offset in the stream, I assumed that > > you are using the same in ChaCha for geli. Although having looked at > > the code it is not that obvious what is going on. There is a method to > > set iv being used, but not to change counter (stream offset). Is that > > intentional? Do we reset key for each geli sector to be encrypted? > > chacha_iv_set() does not reset key. > xchacha_set_key_iv_rounds() require key and IV. > chacha_counter_set() - set block counter. > > chacha_key_set() - copy key and constant to chacha context. > > _______________________________________________ > freebsd-geom@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-geom > To unsubscribe, send any mail to "freebsd-geom-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 14 19:43:56 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2177A8F9; Wed, 14 Jan 2015 19:43:56 +0000 (UTC) Received: from platinum.linux.pl (platinum.edu.pl [81.161.192.4]) by mx1.freebsd.org (Postfix) with ESMTP id D1DF9A2; Wed, 14 Jan 2015 19:43:55 +0000 (UTC) Received: by platinum.linux.pl (Postfix, from userid 87) id DF3E545219A; Wed, 14 Jan 2015 20:43:46 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on platinum.linux.pl X-Spam-Level: X-Spam-Status: No, score=-1.3 required=3.0 tests=ALL_TRUSTED,AWL autolearn=disabled version=3.4.0 Received: from [10.255.0.2] (unknown [83.151.38.73]) by platinum.linux.pl (Postfix) with ESMTPA id 7EFF9452198; Wed, 14 Jan 2015 20:43:46 +0100 (CET) Message-ID: <54B6C6B7.4070407@platinum.linux.pl> Date: Wed, 14 Jan 2015 20:42:47 +0100 From: Adam Nowacki User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: 'freebsd-geom' Subject: Re: ChaCha8/12/20 and GEOM ELI tests References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> <20150114041708.GA3189@reks> <54b601ec.0515980a.0c9c.47e1@mx.google.com> <20150114082019.GA3669@reks> <54b6ae4c.0905990a.6c9c.642e@mx.google.com> <54b6b91b.2aa3700a.3a6c.47b5@mx.google.com> In-Reply-To: <54b6b91b.2aa3700a.3a6c.47b5@mx.google.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2015 19:43:56 -0000 On 2015-01-14 19:44, rozhuk.im@gmail.com wrote: >> Excuse me, but if you think your physical medium is either 100% >> inaccessible to an adversary, or simply not worth a real attack, and >> the speed is the concern, then why do you want to use any encryption at >> all? > > 100% is not available yet introduced GELI keys / mounted drive. > AES-XTS is good but too slow. FreeBSD supports AES-NI - hardware accelerated AES available in many Intel and AMD processors. I'm seeing speeds of 1GB/s on i5 2500K. > ChaCha is already enough to cryptography was not a bottleneck. > > The case when the disks - local (SATA/SAS/IDE/USB), keys entered / disk is mounted and the attacker has access I do not see because AES-XTS does not help. A few scenarios that will break ChaCha encryption: - remapped bad sectors on spinning disks, - multiple copies of sectors on SSD due to wear leveling, - RAID with one of the drives dropping out due to bad cabling, bad sectors or other issues, - disk imaged at multiple points in time (for example powered-off laptop left unattended). From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 15 00:29:50 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E8D515E9; Thu, 15 Jan 2015 00:29:50 +0000 (UTC) Received: from mail-lb0-x22d.google.com (mail-lb0-x22d.google.com [IPv6:2a00:1450:4010:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5DBE793D; Thu, 15 Jan 2015 00:29:50 +0000 (UTC) Received: by mail-lb0-f173.google.com with SMTP id z12so10795306lbi.4; Wed, 14 Jan 2015 16:29:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:reply-to:to:cc:references:in-reply-to:subject:date :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=qP6OB4iPUbJ7AKacUtDRyoIZ4taz/x8ENNhTpPxXiW4=; b=VRIT4j5WCpdNkCddLZkHuiBlJ9/n8CQ3z+l2mV5xHy64fpG4j7alJrqSr5rfmuzMCa wa49GaeQUBTA3LC74Vtd/OjJtWCRuCgh12soMOG5FuCHBnxJOD/I2AOrJkitSX0St26V MwfTPZq3ErgG6Z0MttOoIfyEfb7tYy4EH/opRLy7poDgo3r3GzzGUrPz7bxaWGzqurpe TZrtHpr5bUPnseC8UX5N/o84zjUCLWgmo48rSsRCZpcXHhEPUCSjMX3ednhlw5rhEmJ9 rLPHJvFVteGRn1Nwevqj+C2YoM7Pblt0jkrFq9DaRXNaPdIr2gbqeNEPHpC6uR7grqvN m/Hw== X-Received: by 10.112.198.1 with SMTP id iy1mr1019739lbc.13.1421281788431; Wed, 14 Jan 2015 16:29:48 -0800 (PST) Received: from rimwks1w7x64 ([2001:470:1f15:8e:b007:2759:7397:9491]) by mx.google.com with ESMTPSA id e7sm3160379lbq.33.2015.01.14.16.29.46 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 14 Jan 2015 16:29:47 -0800 (PST) Message-ID: <54b709fb.0739700a.2970.ffffa14a@mx.google.com> X-Google-Original-Message-ID: <000601d0305a$5db5f4a0$1921dde0$@IM@gmail.com> From: rozhuk.im@gmail.com X-Google-Original-From: Reply-To: To: "'Adam Nowacki'" , "'freebsd-geom'" References: <54b33bfa.e31b980a.3e5d.ffffc823@mx.google.com> <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> <20150114041708.GA3189@reks> <54b601ec.0515980a.0c9c.47e1@mx.google.com> <20150114082019.GA3669@reks> <54b6ae4c.0905990a.6c9c.642e@mx.google.com> <54b6b91b.2aa3700a.3a6c.47b5@mx.google.com> <54B6C6B7.4070407@platinum.linux.pl> In-Reply-To: <54B6C6B7.4070407@platinum.linux.pl> Subject: RE: ChaCha8/12/20 and GEOM ELI tests Date: Thu, 15 Jan 2015 03:29:44 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdAwMnJtnO5xniq6RwaQFPotzQ32hwAIMuYA Content-Language: ru Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Jan 2015 00:29:51 -0000 > >> Excuse me, but if you think your physical medium is either 100% > >> inaccessible to an adversary, or simply not worth a real attack, and > >> the speed is the concern, then why do you want to use any encryption > >> at all? > > > > 100% is not available yet introduced GELI keys / mounted drive. > > AES-XTS is good but too slow. > > FreeBSD supports AES-NI - hardware accelerated AES available in many > Intel and AMD processors. I'm seeing speeds of 1GB/s on i5 2500K. Not all modern processors. In ARM / MIPS do not have this accelerator. > > ChaCha is already enough to cryptography was not a bottleneck. > > > > The case when the disks - local (SATA/SAS/IDE/USB), keys entered / > disk is mounted and the attacker has access I do not see because AES- > XTS does not help. > > A few scenarios that will break ChaCha encryption: > - remapped bad sectors on spinning disks, > - multiple copies of sectors on SSD due to wear leveling, Remaping done inside the drive for the OS sector number does not change. If the sector number changes this would hurt not only the data encrypted in any cryptographic algorithm Geom GELI but filesystems without encryption. > - RAID with one of the drives dropping out due to bad cabling, bad > sectors or other issues, This is the same affect file systems without encryption and any other encryption schemes. An exception will be an encryption scheme where one key to all sectors. > - disk imaged at multiple points in time (for example powered-off > laptop left unattended). Yes, but there are observations: 1. It is not critical to encrypt the swap file and TMP by onetime key 2. To get the key stream and read the encrypted data must be long enough for a long time to collect disk images and analyze them. A little help to increase the safety of pre-filling the entire encrypted disk with random data. 3. If this is probably better to use other encryption schemes. :) In general, the attacker will require significant investment of time and resources to carry out such an attack. The owner of such valuable data will not likely make a choice in favor of speed encryption. And yet, for the paranoid. At the moment, XTS implemented only for AES. For AES there are timing attack, allowing the unprivileged process to obtain the encryption key by analyzing the behavior of the system. Cache-timing attacks on AES: http://cr.yp.to/antiforgery/cachetiming-20050414.pdf https://cseweb.ucsd.edu/~kmowery/papers/aes-cache-timing.pdf Wait a minute! A fast, Cross-VM attack on AES: https://eprint.iacr.org/2014/435.pdf A Cache Timing Attack on AES in Virtualization Environments: http://fc12.ifca.ai/pre-proceedings/paper_70.pdf So it may be that the attacker suddenly get an encryption key from the AES as a whole, rather than trying to read the sectors. Note that the sector is more difficult to read is not the privileged process. ChaCha runs in constant time and is not subject to such attacks. From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 15 06:54:32 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 48943300; Thu, 15 Jan 2015 06:54:32 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1on0060.outbound.protection.outlook.com [157.56.110.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7CCEE14C; Thu, 15 Jan 2015 06:54:30 +0000 (UTC) Received: from DM2PR0801MB0944.namprd08.prod.outlook.com (25.160.131.27) by DM2PR0801MB0944.namprd08.prod.outlook.com (25.160.131.27) with Microsoft SMTP Server (TLS) id 15.1.53.17; Thu, 15 Jan 2015 06:54:27 +0000 Received: from DM2PR0801MB0944.namprd08.prod.outlook.com ([25.160.131.27]) by DM2PR0801MB0944.namprd08.prod.outlook.com ([25.160.131.27]) with mapi id 15.01.0053.000; Thu, 15 Jan 2015 06:54:27 +0000 From: "Pokala, Ravi" To: "jhb@freebsd.org" , "eric@vangyzen.net" , "lars.engels@0x20.net" , "allanjude@freebsd.org" Subject: Re: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged Thread-Topic: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged Thread-Index: AQHQMJAa6jIVY6e0Rka/enbEqt4SMA== Date: Thu, 15 Jan 2015 06:54:26 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.7.141117 x-originating-ip: [24.6.178.251] authentication-results: spf=none (sender IP is ) smtp.mailfrom=rpokala@panasas.com; x-dmarcaction-test: None x-microsoft-antispam: BCL:0;PCL:0;RULEID:(3005004);SRVR:DM2PR0801MB0944; x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:; SRVR:DM2PR0801MB0944; x-forefront-prvs: 0457F11EAF x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(13464003)(24454002)(51704005)(199003)(479174004)(189002)(377454003)(19580405001)(19580395003)(2501002)(83506001)(512954002)(106356001)(106116001)(105586002)(54356999)(2201001)(64706001)(46102003)(86362001)(99286002)(68736005)(101416001)(122556002)(102836002)(87936001)(2656002)(2900100001)(36756003)(40100003)(77156002)(97736003)(66066001)(92566002)(50986999)(62966003); DIR:OUT; SFP:1101; SCL:1; SRVR:DM2PR0801MB0944; H:DM2PR0801MB0944.namprd08.prod.outlook.com; FPR:; SPF:None; MLV:sfv; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: panasas.com does not designate permitted sender hosts) Content-Type: text/plain; charset="us-ascii" Content-ID: <1325952284E26D46B8B929DA86EA60D4@namprd08.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: panasas.com X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Jan 2015 06:54:26.5969 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: acf01c9d-c699-42af-bdbb-44bf582e60b0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM2PR0801MB0944 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Jan 2015 06:54:32 -0000 -----Original Message----- >Date: Tue, 13 Jan 2015 13:53:19 -0500 >From: John Baldwin >To: Eric van Gyzen , Lars Engels > , Allan Jude >Cc: freebsd-hackers@freebsd.org >Subject: Re: [PATCH] Display progress during getmemsize() so the > kernel doesn't look like it hanged >Message-ID: <3221643.2Qu3vC2WD5@ralph.baldwin.cx> >Content-Type: text/plain; charset=3D"us-ascii" > >On 1/13/15 10:48 AM, Eric van Gyzen wrote: >> On 01/13/2015 04:11, Lars Engels wrote: >>> On Mon, Jan 12, 2015 at 12:27:18AM -0500, Allan Jude wrote: >>>> Is this feature still useful with memtest.tests=3D0? >>> This feature is useful for everyone who has it set to 1, if they know >>> about it or not. So it's a very useful feature. >>=20 >> Agreed. >>=20 >> Comments on the patch: >>=20 >> The patch will divide by zero when PAGE_SIZE > 1 MiB. Maybe remove >> PAGES_PER_MB, and just use PAGES_PER_GB. Make it const, too. >>=20 >> The "total" line is mostly redundant with the later messages regarding >> "real memory" and "avail memory". I suggest removing it. > >I agree with these. One other nit is that FreeBSD tends to use all caps >for a macro rather than variables. Ravi, how about this variant, it >also only displays the dots if the memory test is enabled. Also, I >think the commit to disable the test should probably be merged to 10, yes. We have a winner. I regenerated the patch against 10-STABLE and attached it to the PR. Thanks all! -Ravi From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 15 06:59:47 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 425E0509 for ; Thu, 15 Jan 2015 06:59:47 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id 02FD41A0 for ; Thu, 15 Jan 2015 06:59:46 +0000 (UTC) Received: from [192.168.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 9CA07617AB for ; Thu, 15 Jan 2015 06:59:40 +0000 (UTC) Message-ID: <54B7656D.9000704@freebsd.org> Date: Thu, 15 Jan 2015 01:59:57 -0500 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged References: In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="RUdiQ49dWmj5EqVT5SVT5fJPCNA76Lles" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Jan 2015 06:59:47 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --RUdiQ49dWmj5EqVT5SVT5fJPCNA76Lles Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2015-01-15 01:54, Pokala, Ravi wrote: > -----Original Message----- >> Date: Tue, 13 Jan 2015 13:53:19 -0500 >> From: John Baldwin >> To: Eric van Gyzen , Lars Engels >> , Allan Jude >> Cc: freebsd-hackers@freebsd.org >> Subject: Re: [PATCH] Display progress during getmemsize() so the >> kernel doesn't look like it hanged >> Message-ID: <3221643.2Qu3vC2WD5@ralph.baldwin.cx> >> Content-Type: text/plain; charset=3D"us-ascii" >> >> On 1/13/15 10:48 AM, Eric van Gyzen wrote: >>> On 01/13/2015 04:11, Lars Engels wrote: >>>> On Mon, Jan 12, 2015 at 12:27:18AM -0500, Allan Jude wrote: >>>>> Is this feature still useful with memtest.tests=3D0? >>>> This feature is useful for everyone who has it set to 1, if they kno= w >>>> about it or not. So it's a very useful feature. >>> >>> Agreed. >>> >>> Comments on the patch: >>> >>> The patch will divide by zero when PAGE_SIZE > 1 MiB. Maybe remove >>> PAGES_PER_MB, and just use PAGES_PER_GB. Make it const, too. >>> >>> The "total" line is mostly redundant with the later messages regardin= g >>> "real memory" and "avail memory". I suggest removing it. >> >> I agree with these. One other nit is that FreeBSD tends to use all ca= ps >> for a macro rather than variables. Ravi, how about this variant, it >> also only displays the dots if the memory test is enabled. Also, I >> think the commit to disable the test should probably be merged to 10, = yes. >=20 > We have a winner. I regenerated the patch against 10-STABLE and attache= d > it to the PR. Thanks all! >=20 > -Ravi >=20 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" >=20 Glad to see this, thanks for doing the work Ravi. Also, I agree with jhb@, we should disable the test by default in stable/10 (i think it is off by default only for VMs currently). --=20 Allan Jude --RUdiQ49dWmj5EqVT5SVT5fJPCNA76Lles Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJUt2VwAAoJEJrBFpNRJZKfgXkP/AjxiYbrQHCdsC9cBxQCCgm3 ozxqkOhBg3A8Kc2HyCI00pXe2+sbJ7sPBUSy+eNuDhjj9zkQkOTZNe51qYyA3/cr GpVs9wuKa/goz45ExWR/aIt85wZdGMLCEwivvVNarxbbGccUQxI80obaGS7yKcn+ OHNM8uDxbAjibteABUfcapDaoXYL7Up14G+6MvZ3ImISIbJ8dddcd4PXwEPpfr83 Qwbmyf8HzPpCqG8OIq8jFm2fZMb9t0jl95cSul1UMX50a2l5lWBIlk8l6ccKR0yS 6898/MtYJNMEgv40KTJw7PZMC/ij0aLEIDpMgLQn8PzkF5EdqcpjNZZjVxO3Xnq8 ply/2QhuMuJb1DLPDF/gLJ2E9QIJcdHUpfQP9YDVFXl9SwM4Lu9R2PhAv5x5Zw+a /dA0CJvPdQW6PacJuU/ne1b9UF2vZdv5lFJXm4j7YyTTAF7bpOBbWdeMsxQf/8FI LpEeO6F5ggPNTsVPvEwzaCdcPycM9ateNsRuGogP5/FZoExOFCLSg8987Gbq+qZd peDnkqmd0xuIbtICdKX/o1prruTJhkaktDuS/VbaGsccut+1kVvbyBZ+g7wVpXG5 WoCjvSpYfAxu9fzyyLUPfriVZdvP7PlaJvCM12aIxl9jP/afnx8KolaWkAoXVj83 HJkLTm1wLN6Rl4ZYMcjm =XFqR -----END PGP SIGNATURE----- --RUdiQ49dWmj5EqVT5SVT5fJPCNA76Lles-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 15 15:01:49 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9B3921C6; Thu, 15 Jan 2015 15:01:49 +0000 (UTC) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id 0D7E0D8C; Thu, 15 Jan 2015 15:01:48 +0000 (UTC) Received: from localhost (apn-37-7-2-180.dynamic.gprs.plus.pl [37.7.2.180]) by mail.dawidek.net (Postfix) with ESMTPSA id 3A5DA2F5; Thu, 15 Jan 2015 16:01:45 +0100 (CET) Date: Thu, 15 Jan 2015 16:03:17 +0100 From: Pawel Jakub Dawidek To: rozhuk.im@gmail.com Subject: Re: ChaCha8/12/20 and GEOM ELI tests Message-ID: <20150115150316.GB1190@garage.freebsd.pl> References: <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> <20150114041708.GA3189@reks> <54b601ec.0515980a.0c9c.47e1@mx.google.com> <20150114082019.GA3669@reks> <54b6ae4c.0905990a.6c9c.642e@mx.google.com> <54b6b91b.2aa3700a.3a6c.47b5@mx.google.com> <54B6C6B7.4070407@platinum.linux.pl> <54b709fb.0739700a.2970.ffffa14a@mx.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54b709fb.0739700a.2970.ffffa14a@mx.google.com> X-OS: FreeBSD 11.0-CURRENT amd64 User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-hackers@freebsd.org, 'freebsd-geom' , 'Adam Nowacki' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Jan 2015 15:01:49 -0000 Hi. I'm very happy that you have spent the time to play with GELI code and I hope you will continue to work on it, but this particular change won't be accepted as part of GELI, please accept that even if you don't fully agree. Stream ciphers are not compatible with GELI design. Using chacha might be a better fit for GBDE, where encryption keys are generated and stored for every write, so there should be no risk with reusing a key stream. This of course also require further analysis. If you would like to spend some more time with GELI, I'd suggest for starters to preparing a patch that removes support for MD5, SHA1 and RIPEMD160. On Thu, Jan 15, 2015 at 03:29:44AM +0300, rozhuk.im@gmail.com wrote: > > >> Excuse me, but if you think your physical medium is either 100% > > >> inaccessible to an adversary, or simply not worth a real attack, and > > >> the speed is the concern, then why do you want to use any encryption > > >> at all? > > > > > > 100% is not available yet introduced GELI keys / mounted drive. > > > AES-XTS is good but too slow. > > > > FreeBSD supports AES-NI - hardware accelerated AES available in many > > Intel and AMD processors. I'm seeing speeds of 1GB/s on i5 2500K. > > Not all modern processors. > In ARM / MIPS do not have this accelerator. > > > > > ChaCha is already enough to cryptography was not a bottleneck. > > > > > > The case when the disks - local (SATA/SAS/IDE/USB), keys entered / > > disk is mounted and the attacker has access I do not see because AES- > > XTS does not help. > > > > A few scenarios that will break ChaCha encryption: > > - remapped bad sectors on spinning disks, > > - multiple copies of sectors on SSD due to wear leveling, > > Remaping done inside the drive for the OS sector number does not change. > If the sector number changes this would hurt not only the data encrypted in > any cryptographic algorithm Geom GELI but filesystems without encryption. > > > > - RAID with one of the drives dropping out due to bad cabling, bad > > sectors or other issues, > > This is the same affect file systems without encryption and any other > encryption schemes. An exception will be an encryption scheme where one key > to all sectors. > > > > - disk imaged at multiple points in time (for example powered-off > > laptop left unattended). > > Yes, but there are observations: > 1. It is not critical to encrypt the swap file and TMP by onetime key > > 2. To get the key stream and read the encrypted data must be long enough for > a long time to collect disk images and analyze them. > A little help to increase the safety of pre-filling the entire encrypted > disk with random data. > > 3. If this is probably better to use other encryption schemes. :) > In general, the attacker will require significant investment of time and > resources to carry out such an attack. > The owner of such valuable data will not likely make a choice in favor of > speed encryption. > > > And yet, for the paranoid. > At the moment, XTS implemented only for AES. > For AES there are timing attack, allowing the unprivileged process to obtain > the encryption key by analyzing the behavior of the system. > > Cache-timing attacks on AES: > http://cr.yp.to/antiforgery/cachetiming-20050414.pdf > https://cseweb.ucsd.edu/~kmowery/papers/aes-cache-timing.pdf > > Wait a minute! A fast, Cross-VM attack on AES: > https://eprint.iacr.org/2014/435.pdf > > A Cache Timing Attack on AES in Virtualization Environments: > http://fc12.ifca.ai/pre-proceedings/paper_70.pdf > > > So it may be that the attacker suddenly get an encryption key from the AES > as a whole, rather than trying to read the sectors. > Note that the sector is more difficult to read is not the privileged > process. > > ChaCha runs in constant time and is not subject to such attacks. > > > > > _______________________________________________ > freebsd-geom@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-geom > To unsubscribe, send any mail to "freebsd-geom-unsubscribe@freebsd.org" -- Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 16 00:00:21 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 504B9429; Fri, 16 Jan 2015 00:00:21 +0000 (UTC) Received: from mail-la0-x229.google.com (mail-la0-x229.google.com [IPv6:2a00:1450:4010:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D71F93E7; Fri, 16 Jan 2015 00:00:20 +0000 (UTC) Received: by mail-la0-f41.google.com with SMTP id hv19so16389655lab.0; Thu, 15 Jan 2015 16:00:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:reply-to:to:cc:references:in-reply-to:subject:date :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=OT7elAebrdJnXGFb7+knLSTrpkxXuYG7pN3DUyPHMk4=; b=i4TcRX8pcGNSLjBibwl4P25iqAo+9GciVaKFzSLFsX4XmH4JSv1agBDCP/kJ7z8xWH xdrZNSnNkZouWNpBHobJ/OjIyHGJacYVoZbCs9/Tu6n5DkgadMa1buN94HbR+jAEYANP lBATHLROnFFLHJt22owy6HAnCBC4YpEiD/RyoYJg1eHQ1CUyE3+/R0zfUF94hnMBITfw ArMtf46Sy0IfpiKuE7q8gA8pxoP/uccVSnzJm5u+bve3Ko0+o1NPK2q0lp5bpmvC+asn Jm09KTtEWHeULXNbs5fef3rnvXhKm+Lyh4wkZVf7BYK9Z72KU1yeQLRMj9IABeF9Y8KG mugA== X-Received: by 10.113.7.163 with SMTP id dd3mr12749655lbd.96.1421366419044; Thu, 15 Jan 2015 16:00:19 -0800 (PST) Received: from rimwks1w7x64 ([2001:470:1f15:8e:e966:4081:140e:40fe]) by mx.google.com with ESMTPSA id w9sm990597laj.13.2015.01.15.16.00.16 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 15 Jan 2015 16:00:17 -0800 (PST) Message-ID: <54b85491.4925980a.17c4.2b00@mx.google.com> X-Google-Original-Message-ID: <001601d0311f$698deb00$3ca9c100$@IM@gmail.com> From: rozhuk.im@gmail.com X-Google-Original-From: Reply-To: To: "'Pawel Jakub Dawidek'" References: <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> <20150114041708.GA3189@reks> <54b601ec.0515980a.0c9c.47e1@mx.google.com> <20150114082019.GA3669@reks> <54b6ae4c.0905990a.6c9c.642e@mx.google.com> <54b6b91b.2aa3700a.3a6c.47b5@mx.google.com> <54B6C6B7.4070407@platinum.linux.pl> <54b709fb.0739700a.2970.ffffa14a@mx.google.com> <20150115150316.GB1190@garage.freebsd.pl> In-Reply-To: <20150115150316.GB1190@garage.freebsd.pl> Subject: RE: ChaCha8/12/20 and GEOM ELI tests Date: Fri, 16 Jan 2015 03:00:15 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdAw1C/82uLB0fl4TM2tPUP37KTA4AAFlFpw Content-Language: ru Cc: freebsd-hackers@freebsd.org, 'freebsd-geom' , 'Adam Nowacki' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2015 00:00:21 -0000 > I'm very happy that you have spent the time to play with GELI code and > I hope you will continue to work on it, but this particular change > won't be accepted as part of GELI, please accept that even if you don't > fully agree. Stream ciphers are not compatible with GELI design. Hopefully ChaCha gets into /dev/crypto. > Using chacha might be a better fit for GBDE, where encryption keys are > generated and stored for every write, so there should be no risk with > reusing a key stream. This of course also require further analysis. > > If you would like to spend some more time with GELI, I'd suggest for > starters to preparing a patch that removes support for MD5, SHA1 and > RIPEMD160. Options I have not so much. 1. Drink vodka and use slow AES-XTS :) 2. Use ChaCha GELI private patch 3. Write Geom node. Cipher = ChaCha/XChaCha Hash = Blake2 - https://blake2.net/ Key1 = key for cipher Key2 = key hor HMAC IV = HMAC(Key2, ('plain text data' + 'sector num')) = (8/24 bytes) IV stored on disk in two tables: main and back up 1 GB (4kb sector) require 2097152 / 6291456 bytes per IV table or 16777216 for full 512 bit hmac +: 1. optional data integrity verification (authentication) 2. cache-timing attack resistant 3. keys can be changed without transferring data to other media and minimal risk -: 1. very slow write: it is necessary to calculate the hmac and update two tables with IV data 2. slow reading: IV need to get from the table (+ optional hmac calc) 3. the risk of damage IV table on disk hardware problems 4. part of the disk is busy service data (IV tables) ChaCha20 + blake2 will still be faster than AES-XTS, about half as much. It takes time, but I was happy with all ChaCha in geli. Even if implement, there is no guarantee that gets into the mainline. I'll think about Geom node with ChaCha, again and weigh all pros and cons, and now I have disks that need encrypt and fill in for 20 days lie idle. From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 16 00:24:16 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DBB129B3; Fri, 16 Jan 2015 00:24:16 +0000 (UTC) Received: from mail.highsecure.ru (l.highsecure.ru [5.9.155.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8BB79897; Fri, 16 Jan 2015 00:24:15 +0000 (UTC) Received: from [172.24.168.60] (global-2-11.nat.csx.cam.ac.uk [131.111.185.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: vsevolod@highsecure.ru) by mail.highsecure.ru (Postfix) with ESMTPSA id AC75F300162; Fri, 16 Jan 2015 01:24:04 +0100 (CET) Message-ID: <54B85A25.6010806@highsecure.ru> Date: Fri, 16 Jan 2015 00:24:05 +0000 From: Vsevolod Stakhov User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: Rozhuk.IM@gmail.com Subject: Re: ChaCha8/12/20 and GEOM ELI tests References: <54B4AE55.9090205@platinum.linux.pl> <54b5d299.4914980a.61cd.43a6@mx.google.com> <20150114041708.GA3189@reks> <54b601ec.0515980a.0c9c.47e1@mx.google.com> <20150114082019.GA3669@reks> <54b6ae4c.0905990a.6c9c.642e@mx.google.com> <54b6b91b.2aa3700a.3a6c.47b5@mx.google.com> <54B6C6B7.4070407@platinum.linux.pl> <54b709fb.0739700a.2970.ffffa14a@mx.google.com> <20150115150316.GB1190@garage.freebsd.pl> <54b85491.4925980a.17c4.2b00@mx.google.com> In-Reply-To: <54b85491.4925980a.17c4.2b00@mx.google.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=highsecure.ru; s=dkim; t=1421367845; bh=YaV7C2tjosHBG5urmT0V8dR3hAa/rv3G5D3ugOgrjHw=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=wQkFPGlXAVnwss7B41Nq9+mKduE0WKqUKG9SbKBfWfb2d2/OvnKNvpJGsiny1L+ZKnFUANtMWMChefe2fHp2acJOelnE+HK/do4+TL4ICWn0K8dHCX1LelKiBHeQGs7sDx7bL0vCg1wkzKnHo0cwogAVShN3Aqo5gKF1s0Mhm2k= Cc: freebsd-hackers@freebsd.org, 'Adam Nowacki' , 'freebsd-geom' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2015 00:24:17 -0000 On 16/01/15 00:00, rozhuk.im@gmail.com wrote: >> I'm very happy that you have spent the time to play with GELI code and >> I hope you will continue to work on it, but this particular change >> won't be accepted as part of GELI, please accept that even if you don't >> fully agree. Stream ciphers are not compatible with GELI design. > > Hopefully ChaCha gets into /dev/crypto. > > >> Using chacha might be a better fit for GBDE, where encryption keys are >> generated and stored for every write, so there should be no risk with >> reusing a key stream. This of course also require further analysis. >> >> If you would like to spend some more time with GELI, I'd suggest for >> starters to preparing a patch that removes support for MD5, SHA1 and >> RIPEMD160. > > Options I have not so much. > 1. Drink vodka and use slow AES-XTS :) > 2. Use ChaCha GELI private patch > 3. Write Geom node. > > Cipher = ChaCha/XChaCha > Hash = Blake2 - https://blake2.net/ > Key1 = key for cipher > Key2 = key hor HMAC > IV = HMAC(Key2, ('plain text data' + 'sector num')) = (8/24 bytes) > What about the fourth funny option - trying threefish which is claimed to be a very fast tweakable block cipher? From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 16 01:58:12 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5A023C14 for ; Fri, 16 Jan 2015 01:58:12 +0000 (UTC) Received: from know-smtprelay-omc-3.server.virginmedia.net (know-smtprelay-omc-3.server.virginmedia.net [80.0.253.67]) by mx1.freebsd.org (Postfix) with ESMTP id B7B2913C for ; Fri, 16 Jan 2015 01:58:11 +0000 (UTC) Received: from [192.168.1.100] ([86.20.122.200]) by know-smtprelay-3-imp with bizsmtp id gRx01p01h4KXVwe01Rx0AN; Fri, 16 Jan 2015 01:57:01 +0000 X-Originating-IP: [86.20.122.200] X-Spam: 0 X-Authority: v=2.1 cv=N7qnFgNB c=1 sm=1 tr=0 a=WByauD8lJrWvBFCNrxRoEQ==:117 a=WByauD8lJrWvBFCNrxRoEQ==:17 a=eyG8AT_ZZ_QA:10 a=N659UExz7-8A:10 a=NLZqzBF-AAAA:8 a=ECOha3hJE2g-2iAa_B8A:9 a=pILNOxqGKmIA:10 a=XdyKOaxJwVsA:10 a=CJS5E0KSweAA:10 Message-ID: <54B86FD5.3090203@NTLWorld.com> Date: Fri, 16 Jan 2015 01:56:37 +0000 From: Jonathan de Boyne Pollard User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: FreeBSD Hackers Subject: nosh version 1.12 References: <54430B41.3010301@NTLWorld.com> In-Reply-To: <54430B41.3010301@NTLWorld.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2015 01:58:12 -0000 nosh is now up to version 1.12 * http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh.html As I wrote before: If you also read the worked example, make sure that you read all of the way to the bottom. (-: If you want to read more, there's a whole Guide in the package, and lots of manual pages. I missed out the annoucements for 1.10 and 1.11. There have been quite a lot of additions, but concentrated in some very specific areas of the package. The developers of the "vt" subsystem in the kernel might be pleased to learn that the console-fb-realizer in nosh uses the same font format, and can use the supplied vt fonts. I myself use the 9x15 and 9x15B "misc fixed" fonts published by Markus Kuhn, as converted from BDF by "vtfontcvt". The keyboard map compiler, similarly, compiles kbdmap(5) files. Of course, there aren't the constraints in applications mode programming that there are in kernel mode programming. So console-fb-realizer can load multiple fonts, can do true italics (SGR 3), distinguishes italic from oblique, can use a true italic font (as long as it is monospace of course), does 24-bit RGB colour, overlays the BSD kbdmaps on top of an ISO 9995-3 "common secondary group", does Unicode dead keys, and does ISO 9993-5 so-called "peculiar" dead keys. If at this point you are wondering what console-fb-realizer is, crank up your favourite HTML viewer, point it at the nosh Guide, and read about user-space virtual terminals, made up of console-fb-realizer, console-terminal-emulator, console-multiplexor, and your choice of tty login program. Yes, it's intentionally decomposable. Yes, you can change realizers on the fly without affecting running programs. Yes, the number of multiplexed virtual terminals can be changed on the fly. Yes, it does UTF-8. Yes, it's BSD licensed. Yes, it's designed so that one can do interesting things like plug in BRLTTY and not need screen. * http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh/brltty.html As well as being compatible with the control sequences for the FreeBSD and Linux consoles (minus some highly device-specific stuff that really only applies if one is using real PC hardware with things like VGA registers and whatnot), console-terminal-emulator understands the DECSCPP and DECSNLS control sequences for changing screen sizes, has programmable background-colour erase, and has the DECBKM mode switch. In support of the latter, the compiler provides a "bspace" extension to kbdmap(5) files for doing the right thing when the backspace key is pressed. In other news: The previously FreeBSD-specific VirtualBox guest services services (sic!) are now more platform-agnostic. The fact that some of these services are kernel modules inspired the creation of a simple, minimal, but (importantly) portable load-kernel-module tool. Other new services include a set of OpenStack service bundles. And the convert-ttys-presets tool allows one to import from /etc/ttys to a set of ttylogin service bundles. Have some free bug reports that resulted from this: * kbdmap(5) documents 35 ASCII control characters. It uses the name "np" for Form Feed, which is a bit confusing when one is in the ECMA-48 world of terminals, since NP is the name of a quite different control sequence in ECMA-48. ECMA-48 uses FF for Form Feed. None of the FreeBSD 10 keymaps that I've looked at actually use "np". Meanwhile, 32 C0 control characters plus SPACE and DEL is of course 34. I'll leave it as an exercise to the reader to discover the extra bogus 35th ASCII control character that the FreeBSD manual just invents from whole cloth. The answer can be found in the source for the console-convert-kbdmap tool. (-: * In contrast, many of the keymaps use "nop", but that's not documented in the man page at all. * For some reason, the higher numbered function keys (F13 and above) on the FreeBSD kernel terminal emulator still generate the SCO XENIX control sequences, even when the lower numbered function keys generate the DEC VT sequences. I didn't replicate this behaviour, because it looks like a simple bug where the higher numbered function keys have been overlooked during DECification. console-terminal-emulator generates DEC VT sequences for all function keys from F1 to F22 when in DEC VT mode. * It turns out that clang++ lets one static_cast<> away constness sometimes when it shouldn't. You'll need to use syscons for console-fb-realizer until I get to the bottom of the ioctl() problem that it encounters with vt. And I know about the keyboard LEDs. From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 16 08:19:31 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1AD2460B; Fri, 16 Jan 2015 08:19:31 +0000 (UTC) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id A6A7FB9B; Fri, 16 Jan 2015 08:19:29 +0000 (UTC) Received: from localhost (58.wheelsystems.com [83.12.187.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 9EBBB564; Fri, 16 Jan 2015 09:19:27 +0100 (CET) Date: Fri, 16 Jan 2015 09:21:04 +0100 From: Pawel Jakub Dawidek To: rozhuk.im@gmail.com Subject: Re: ChaCha8/12/20 and GEOM ELI tests Message-ID: <20150116082104.GA1230@garage.freebsd.pl> References: <20150114041708.GA3189@reks> <54b601ec.0515980a.0c9c.47e1@mx.google.com> <20150114082019.GA3669@reks> <54b6ae4c.0905990a.6c9c.642e@mx.google.com> <54b6b91b.2aa3700a.3a6c.47b5@mx.google.com> <54B6C6B7.4070407@platinum.linux.pl> <54b709fb.0739700a.2970.ffffa14a@mx.google.com> <20150115150316.GB1190@garage.freebsd.pl> <54b85491.4925980a.17c4.2b00@mx.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54b85491.4925980a.17c4.2b00@mx.google.com> X-OS: FreeBSD 11.0-CURRENT amd64 User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-hackers@freebsd.org, 'freebsd-geom' , 'Adam Nowacki' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2015 08:19:31 -0000 On Fri, Jan 16, 2015 at 03:00:15AM +0300, rozhuk.im@gmail.com wrote: > > I'm very happy that you have spent the time to play with GELI code and > > I hope you will continue to work on it, but this particular change > > won't be accepted as part of GELI, please accept that even if you don't > > fully agree. Stream ciphers are not compatible with GELI design. > > Hopefully ChaCha gets into /dev/crypto. > > > > Using chacha might be a better fit for GBDE, where encryption keys are > > generated and stored for every write, so there should be no risk with > > reusing a key stream. This of course also require further analysis. > > > > If you would like to spend some more time with GELI, I'd suggest for > > starters to preparing a patch that removes support for MD5, SHA1 and > > RIPEMD160. > > Options I have not so much. > 1. Drink vodka and use slow AES-XTS :) > 2. Use ChaCha GELI private patch > 3. Write Geom node. 4. Look at GBDE. > Cipher = ChaCha/XChaCha > Hash = Blake2 - https://blake2.net/ > Key1 = key for cipher > Key2 = key hor HMAC > IV = HMAC(Key2, ('plain text data' + 'sector num')) = (8/24 bytes) > > IV stored on disk in two tables: main and back up > 1 GB (4kb sector) require 2097152 / 6291456 bytes per IV table > or 16777216 for full 512 bit hmac > > +: > 1. optional data integrity verification (authentication) > 2. cache-timing attack resistant > 3. keys can be changed without transferring data to other media and minimal > risk > > -: > 1. very slow write: it is necessary to calculate the hmac and update two > tables with IV data > 2. slow reading: IV need to get from the table (+ optional hmac calc) > 3. the risk of damage IV table on disk hardware problems > 4. part of the disk is busy service data (IV tables) This would be very hard to implement correctly, because writing the data and updating your tables are not atomic on disk. How do you handle the case where you write new data, but your system crash or you have a power outage before updating the table? I came into conclusion that data authentication doesn't belong in the layer of disk encryption, because of lack of atomicity. GBDE has this problem and GELI data authentication has similar problem. This problem is mitigated by ZFS, which is transactional, copy-on-write file system that never overwrites existing data. I personally use ZFS with SHA256 checksum on top of GELI. > ChaCha20 + blake2 will still be faster than AES-XTS, about half as much. > > It takes time, but I was happy with all ChaCha in geli. > Even if implement, there is no guarantee that gets into the mainline. > I'll think about Geom node with ChaCha, again and weigh all pros and cons, > and now I have disks that need encrypt and fill in for 20 days lie idle. -- Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 16 15:02:25 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CF62BB4B for ; Fri, 16 Jan 2015 15:02:25 +0000 (UTC) Received: from astart2.astart.com (108-248-95-193.lightspeed.sndgca.sbcglobal.net [108.248.95.193]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A5F71D20 for ; Fri, 16 Jan 2015 15:02:25 +0000 (UTC) Received: from laptop_93.private (localhost [127.0.0.1]) by astart2.astart.com (8.14.4/8.14.4) with ESMTP id t0GF2HPc037916 for ; Fri, 16 Jan 2015 07:02:18 -0800 (PST) (envelope-from papowell@astart.com) Message-ID: <54B927F9.1010401@astart.com> Date: Fri, 16 Jan 2015 07:02:17 -0800 From: Patrick Powell Reply-To: papowell@astart.com Organization: Astart Technologies User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged References: <54B7656D.9000704@freebsd.org> In-Reply-To: <54B7656D.9000704@freebsd.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2015 15:02:25 -0000 On 01/14/15 22:59, Allan Jude wrote: > Glad to see this, thanks for doing the work Ravi. Also, I agree with > jhb@, we should disable the test by default in stable/10 (i think it > is off by default only for VMs currently). Please please do not disable memory tests by default. If you do, when trying to boot from a CD/Memory Stick image on a system with bad memory (which would be found by the tests) then it gets quite difficult to find this problem. You need to have some boot level option setting experience/wizardry to set the flags during boot. Also this information should be added to the Hardware or Install portion of the FreeBSD Handbook. I have vivid memories to trying to boot up a system with bad memory. -- Patrick Powell Astart Technologies papowell@astart.com 1530 Jamacha Rd, Suite X Network and System San Diego, CA 92019 Consulting 858-874-6543 FAX 858-751-2435 Web: www.astart.com From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 16 17:25:34 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5E871DAB for ; Fri, 16 Jan 2015 17:25:34 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3093AFF1 for ; Fri, 16 Jan 2015 17:25:34 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-226-91.lns20.per1.internode.on.net [121.45.226.91]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id t0GHPNnO049438 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 16 Jan 2015 09:25:25 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <54B9497D.8060404@freebsd.org> Date: Sat, 17 Jan 2015 01:25:17 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: Jonathan de Boyne Pollard , FreeBSD Hackers Subject: Re: nosh version 1.12 References: <54430B41.3010301@NTLWorld.com> <54B86FD5.3090203@NTLWorld.com> In-Reply-To: <54B86FD5.3090203@NTLWorld.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2015 17:25:34 -0000 On 1/16/15 9:56 AM, Jonathan de Boyne Pollard wrote: > nosh is now up to version 1.12 > > * > http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh.html > > As I wrote before: If you also read the worked example, make sure > that you read all of the way to the bottom. (-: If you want to > read more, there's a whole Guide in the package, and lots of manual > pages. > > It's all very cool, tough my head rebelled, and told me it's way too late at night, and refused to absorb anything past the first few paragraphs. I'll try again tomorrow :-) You've obviously researched the space a lot and done a lot of preparation. I hope the rest of the developers can take this seriously. Julian From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 16 17:43:33 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C3D8B6EC; Fri, 16 Jan 2015 17:43:33 +0000 (UTC) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 97AB62EE; Fri, 16 Jan 2015 17:43:33 +0000 (UTC) Received: from [73.34.117.227] (helo=ilsoft.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1YCAvk-0002EG-3m; Fri, 16 Jan 2015 17:43:32 +0000 Received: from revolution.hippie.lan (revolution.hippie.lan [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id t0GHhVfq001672; Fri, 16 Jan 2015 10:43:31 -0700 (MST) (envelope-from ian@freebsd.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 73.34.117.227 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1/MRcGYYUy2JUgNEIFZjGZj Message-ID: <1421430211.14601.296.camel@freebsd.org> Subject: Re: nosh version 1.12 From: Ian Lepore To: Julian Elischer Date: Fri, 16 Jan 2015 10:43:31 -0700 In-Reply-To: <54B9497D.8060404@freebsd.org> References: <54430B41.3010301@NTLWorld.com> <54B86FD5.3090203@NTLWorld.com> <54B9497D.8060404@freebsd.org> Content-Type: text/plain; charset="us-ascii" X-Mailer: Evolution 3.12.8 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Cc: FreeBSD Hackers , Jonathan de Boyne Pollard X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2015 17:43:33 -0000 On Sat, 2015-01-17 at 01:25 +0800, Julian Elischer wrote: > On 1/16/15 9:56 AM, Jonathan de Boyne Pollard wrote: > > nosh is now up to version 1.12 > > > > * > > http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh.html > > > > As I wrote before: If you also read the worked example, make sure > > that you read all of the way to the bottom. (-: If you want to > > read more, there's a whole Guide in the package, and lots of manual > > pages. > > > > > It's all very cool, tough my head rebelled, and told me it's way too > late at night, and refused to absorb anything past the first few > paragraphs. I'll try again tomorrow :-) > > > You've obviously researched the space a lot and done a lot of preparation. > > I hope the rest of the developers can take this seriously. Speaking as a developer (but only for myself) I can say that I read about 3 or 4 paragraphs of that posting and, not having discovered any hint of an answer to the basic question "WTF is nosh?" I moved on. -- Ian From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 16 17:46:37 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6DD1E834 for ; Fri, 16 Jan 2015 17:46:37 +0000 (UTC) Received: from mail-oi0-x230.google.com (mail-oi0-x230.google.com [IPv6:2607:f8b0:4003:c06::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2E60532B for ; Fri, 16 Jan 2015 17:46:37 +0000 (UTC) Received: by mail-oi0-f48.google.com with SMTP id u20so18416953oif.7 for ; Fri, 16 Jan 2015 09:46:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=l5N3r8irWt84JyrmZrXHbQELNzxsKm/QKBr0Znveb74=; b=QNp1VSz8S25AIWjWthz8DFGqhz57yEwAW0flag2ZqDy80Zww2jkuQ6XAhsnhXa6gpM AnoA0/kSmqWfY+KROcglH0PJWWm9Ee4eQbCkdovfNWuDNHTvHlmzedFhzO1qTolsVgCG Z5OJYNpy2d4uQIZfnsInhY4oYkCGhvg3hAUrg/UIt4SMPlr4FMhln3D+F70LynDReHNk AngKwGVUu0LuH3f0zCdOjPadS7QDRqNHRUiS7Qs3gyxO3Msj+BEk4DIlAyTinJQiHHWs Wv0L1DjUU0+RUpyIsXdQ+bO/LVq/JN58qQCIRdGlmmqBNEV7HR4PpnJlWvU696GTl6cg 6h8Q== MIME-Version: 1.0 X-Received: by 10.182.81.195 with SMTP id c3mr10367139oby.60.1421430396452; Fri, 16 Jan 2015 09:46:36 -0800 (PST) Received: by 10.202.76.71 with HTTP; Fri, 16 Jan 2015 09:46:36 -0800 (PST) In-Reply-To: <1421430211.14601.296.camel@freebsd.org> References: <54430B41.3010301@NTLWorld.com> <54B86FD5.3090203@NTLWorld.com> <54B9497D.8060404@freebsd.org> <1421430211.14601.296.camel@freebsd.org> Date: Fri, 16 Jan 2015 09:46:36 -0800 Message-ID: Subject: Re: nosh version 1.12 From: Freddie Cash To: FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2015 17:46:37 -0000 On Fri, Jan 16, 2015 at 9:43 AM, Ian Lepore wrote: > On Sat, 2015-01-17 at 01:25 +0800, Julian Elischer wrote: > > On 1/16/15 9:56 AM, Jonathan de Boyne Pollard wrote: > > > nosh is now up to version 1.12 > > > > > > * > > > > http://homepage.ntlworld.com./jonathan.deboynepollard/Softwares/nosh.html > > > > > > As I wrote before: If you also read the worked example, make sure > > > that you read all of the way to the bottom. (-: If you want to > > > read more, there's a whole Guide in the package, and lots of manual > > > pages. > > > > > > > > It's all very cool, tough my head rebelled, and told me it's way too > > late at night, and refused to absorb anything past the first few > > paragraphs. I'll try again tomorrow :-) > > > > > > You've obviously researched the space a lot and done a lot of > preparation. > > > > I hope the rest of the developers can take this seriously. > > Speaking as a developer (but only for myself) I can say that I read > about 3 or 4 paragraphs of that posting and, not having discovered any > hint of an answer to the basic question "WTF is nosh?" I moved on. =E2=80=8BThe very first line in the linked website: =E2=80=8B =E2=80=8B The nosh package is a suite of system-level utilities for initializing and =E2=80=8B running a BSD or Linux system, and for managing daemons. =E2=80=8BGranted, adding that to the e-mail would have been nice. But, it = wasn't hard to figure out what nosh does. (And I'm not even a developer.) ;)=E2= =80=8B --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 16 17:53:36 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9ADB8A85 for ; Fri, 16 Jan 2015 17:53:36 +0000 (UTC) Received: from mail-pd0-x232.google.com (mail-pd0-x232.google.com [IPv6:2607:f8b0:400e:c02::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 615A7645 for ; Fri, 16 Jan 2015 17:53:36 +0000 (UTC) Received: by mail-pd0-f178.google.com with SMTP id r10so23920039pdi.9 for ; Fri, 16 Jan 2015 09:53:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=mwtdaoVUK0DE8hAeKz9rMAyPg6OsMuWgFWjeyCH/OOE=; b=tKDdUr7RO2M+GApO2ndeUKO72PjLseMBZB9Ngky7ZzKGzjTHlnOEktwOmdTlR9vjBK iQn72V0KgDuEtwOT6dwxLG2L8WWV881u8zdcccHPF8BEL/qZjtxVegPb6yHeH2MEIT37 6/pwVadb0BBx+PcyESC3IzbGSnPo/27MHqB+tDeymUDzkIMlVJrVGgmebgMjnXi2thGe KkwZp/1Cov1kU6fSe22R5waq3/r/THYaXPzZnWBGlTPjn69+SLXRps8onax6eQBZ0Rfm yKFOi4MI92c4M9X6sfj0oJAgb/IaIM07O3mafroMM8NMJwZiCQwh/SqbnR/nS2yEvrfW GSpQ== X-Received: by 10.68.132.134 with SMTP id ou6mr15963993pbb.118.1421430815828; Fri, 16 Jan 2015 09:53:35 -0800 (PST) Received: from ?IPv6:2601:8:ab80:7d6:f8b3:fe2f:bf33:da64? ([2601:8:ab80:7d6:f8b3:fe2f:bf33:da64]) by mx.google.com with ESMTPSA id wt4sm4695781pab.4.2015.01.16.09.53.34 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 16 Jan 2015 09:53:34 -0800 (PST) Content-Type: multipart/signed; boundary="Apple-Mail=_B5A398C7-53F3-4B76-8DC4-C7C2C5C2B339"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged From: Garrett Cooper In-Reply-To: <54B927F9.1010401@astart.com> Date: Fri, 16 Jan 2015 09:53:32 -0800 Message-Id: References: <54B7656D.9000704@freebsd.org> <54B927F9.1010401@astart.com> To: papowell@astart.com X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2015 17:53:36 -0000 --Apple-Mail=_B5A398C7-53F3-4B76-8DC4-C7C2C5C2B339 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jan 16, 2015, at 7:02, Patrick Powell wrote: > On 01/14/15 22:59, Allan Jude wrote: >> Glad to see this, thanks for doing the work Ravi. Also, I agree with = jhb@, we should disable the test by default in stable/10 (i think it is = off by default only for VMs currently).=20 >=20 > Please please do not disable memory tests by default. If you do, when = trying to boot from a CD/Memory Stick > image on a system with bad memory (which would be found by the tests) = then it gets quite difficult to find this > problem. >=20 > You need to have some boot level option setting experience/wizardry to = set the flags during boot. >=20 > Also this information should be added to the Hardware or Install = portion of the FreeBSD Handbook. >=20 > I have vivid memories to trying to boot up a system with bad memory. Or=85 turn it off by default and provide an option in the boot loader to = turn it on manually? Cheers! --Apple-Mail=_B5A398C7-53F3-4B76-8DC4-C7C2C5C2B339 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJUuVAcAAoJEMZr5QU6S73eIDUH/1C17ro/r+hqoQ1qbE4Y/YU8 vRSBBVY1TFChTvRH2hGTI6mhzsSBJ4QEHYvX0Brz2HmJDA82S60n3uoE3MEJX7n9 S4TyTxDa8+WfBWs9LQDLMxtrMp0xi+PYFh4dvyQVWzWtU3m6Cy9dMzdW3mNomL/K VAgx2fuVaqQWUBh9rhnYrjGG06OY5CqMkZlTsKESnD3LVgCfYZtxu4Clk+1KWUFH /ghZ5afwIk9ZvcgRUb6/ZpnKQunhtooSYEGjsXRQqFkXSviAp6ufqiq8BPcV67Zx 7DZ1iI+IWN6hwegWZWWwhTsxO/mthBjz4YLFERpcIfRLyepe78sp+cpEhX8M/m4= =AtMY -----END PGP SIGNATURE----- --Apple-Mail=_B5A398C7-53F3-4B76-8DC4-C7C2C5C2B339-- From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 16 18:04:34 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC5AEE89 for ; Fri, 16 Jan 2015 18:04:34 +0000 (UTC) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [IPv6:2607:fc50:1000:7400:216:3eff:fe72:314f]) by mx1.freebsd.org (Postfix) with ESMTP id 9D5037EF for ; Fri, 16 Jan 2015 18:04:34 +0000 (UTC) Received: from marvin.lab.vangyzen.net (c-24-125-214-90.hsd1.va.comcast.net [24.125.214.90]) by smtp.vangyzen.net (Postfix) with ESMTPSA id D8F5756467; Fri, 16 Jan 2015 12:04:33 -0600 (CST) Message-ID: <54B952B0.1060503@vangyzen.net> Date: Fri, 16 Jan 2015 13:04:32 -0500 From: Eric van Gyzen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.0 MIME-Version: 1.0 To: Garrett Cooper , papowell@astart.com, freebsd-hackers@freebsd.org Subject: Re: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged References: <54B7656D.9000704@freebsd.org> <54B927F9.1010401@astart.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2015 18:04:34 -0000 On 01/16/2015 12:53, Garrett Cooper wrote: > On Jan 16, 2015, at 7:02, Patrick Powell wrote: > >> On 01/14/15 22:59, Allan Jude wrote: >>> Glad to see this, thanks for doing the work Ravi. Also, I agree with jhb@, we should disable the test by default in stable/10 (i think it is off by default only for VMs currently). >> Please please do not disable memory tests by default. If you do, when trying to boot from a CD/Memory Stick >> image on a system with bad memory (which would be found by the tests) then it gets quite difficult to find this >> problem. >> >> You need to have some boot level option setting experience/wizardry to set the flags during boot. >> >> Also this information should be added to the Hardware or Install portion of the FreeBSD Handbook. >> >> I have vivid memories to trying to boot up a system with bad memory. > Or… turn it off by default and provide an option in the boot loader to turn it on manually? > Cheers! I was thinking the same thing. In addition, maybe turn it on by default for CD/memstick images. Eric From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 16 19:07:40 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2BB871A8 for ; Fri, 16 Jan 2015 19:07:40 +0000 (UTC) Received: from puchar.net (puchar.net [188.252.31.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 93D58C4 for ; Fri, 16 Jan 2015 19:07:39 +0000 (UTC) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.14.9/8.14.9) with ESMTP id t0GJ7bHN010415 for ; Fri, 16 Jan 2015 20:07:37 +0100 (CET) (envelope-from wojtek@puchar.net) X-Authentication-Warning: puchar.net: Host laptop.wojtek.intra [10.1.0.2] claimed to be [10.1.0.2] Date: Fri, 16 Jan 2015 20:07:38 +0100 (CET) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: freebsd-hackers@freebsd.org Subject: etherswitchcfg - TPLINK TP-WR1043ND Message-ID: User-Agent: Alpine 2.11 (BSF 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Fri, 16 Jan 2015 20:07:37 +0100 (CET) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2015 19:07:40 -0000 what i am doing wrong. I want all 5 ports to act just like one subnet. i do /sbin/etherswitchcfg vlangroup0 vlan 1 members 0,1,2,3,4,5 /sbin/etherswitchcfg vlangroup1 members none port 1,2,3 or 4 works fine. port 0 shows connection as well, but no ping. what am i doing wrong? From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 16 21:56:26 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 193DDAB5 for ; Fri, 16 Jan 2015 21:56:26 +0000 (UTC) Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D3F135E7 for ; Fri, 16 Jan 2015 21:56:25 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id rp18so23082942iec.11 for ; Fri, 16 Jan 2015 13:56:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=2JPjvhLVuwgV6wCPhwFJdN6o2uDuHYYA1VrYNcpnD1k=; b=O7pw78SsA6RIvcI/KSLA2O432eQQlnwD8LC9HF9MbpLJj3FG4bZFm/ncGkjq/z5rdT dkOtqWV3tbgOvulVX9On56TPomXscGnePeRtRs16FsnuwORVf5i/SigNP44uX1myxPu9 zrSIJxOwKPcj4pafiMRu4xVhtVskr8vooGnFAj6FHz5H1SgYmFju1Aq6LTWFvfQYL4oz LYDECbL6pZDPJcsU6qPEM/8PzH/3HV8XBpIwWuzbVusXbPYCXhOlv5pcgLsjat6iF/aP b9yAqCtk5LXLxO5z0qwzhM1R68oX8ybVE/Wv96sISzg0E0N9MqZY0XjO+dDesaiy24s8 jzAw== X-Received: by 10.50.102.4 with SMTP id fk4mr6100561igb.28.1421445385387; Fri, 16 Jan 2015 13:56:25 -0800 (PST) MIME-Version: 1.0 Sender: carpeddiem@gmail.com Received: by 10.107.0.85 with HTTP; Fri, 16 Jan 2015 13:56:04 -0800 (PST) In-Reply-To: <54B927F9.1010401@astart.com> References: <54B7656D.9000704@freebsd.org> <54B927F9.1010401@astart.com> From: Ed Maste Date: Fri, 16 Jan 2015 16:56:04 -0500 X-Google-Sender-Auth: DG1xELFGmMroRTN9ij_sMAFaYH4 Message-ID: Subject: Re: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged To: papowell@astart.com Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2015 21:56:26 -0000 On 16 January 2015 at 10:02, Patrick Powell wrote: > On 01/14/15 22:59, Allan Jude wrote: >> >> Glad to see this, thanks for doing the work Ravi. Also, I agree with jhb@, >> we should disable the test by default in stable/10 (i think it is off by >> default only for VMs currently). > > > Please please do not disable memory tests by default. If you do, when > trying to boot from a CD/Memory Stick > image on a system with bad memory (which would be found by the tests) then > it gets quite difficult to find this > problem. The boot time "memory test" is not particularly valuable, especially on contemporary amd64 hardware. While it won't have any false positives, there are a huge number of failure modes it will not catch. It also does not inform the user of "failure" -- it just removes that memory from the kernel's map. It's really a test of memory presence, not quality. From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 17 03:36:44 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BAD1054B; Sat, 17 Jan 2015 03:36:44 +0000 (UTC) Received: from mail-lb0-x236.google.com (mail-lb0-x236.google.com [IPv6:2a00:1450:4010:c04::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CE98C22; Sat, 17 Jan 2015 03:36:44 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id u10so21203176lbd.13; Fri, 16 Jan 2015 19:36:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:from:reply-to:to:cc:references:in-reply-to:subject:date :mime-version:content-type:content-transfer-encoding:thread-index :content-language; bh=2iirWMuYglFNywDsavq0dA/4uH+mnIsx+DAk6QGOR7A=; b=VUGf09kdYWyWHrrITfHcNtlO27GwSVQF+nXMgUujxKC6AT3Y0emwsaMbVJKNSJa+NB ZtX3Hh245QiHA1TBrIF7y8GBq2WQs6DrsWCnKI89U8+jFhQ7/XBYcnFLDYy8cN16hlRp 3H5nAtf/MgAZMvkiJt9eHgOT5JQpd9KsG8eQMXk8TJFqicuww/RkAW4mq18X49PXMdDx J0D43j8d8gnig79PMaRX4y9ohW0icAjBYIOmzHaG+5Eq90q2LXRTcC2bt7by6zkv8IB5 BECjE/q3XYP8iLYuCLp8J7xTQDx1M59chwu7YS06hwpZv/QtZzTHRCbmvIdr5v+wwcJD Qygg== X-Received: by 10.152.8.225 with SMTP id u1mr18887079laa.21.1421465802304; Fri, 16 Jan 2015 19:36:42 -0800 (PST) Received: from rimwks1w7x64 ([2001:470:1f15:8e:e966:4081:140e:40fe]) by mx.google.com with ESMTPSA id ku11sm1850537lac.32.2015.01.16.19.36.40 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 16 Jan 2015 19:36:41 -0800 (PST) Message-ID: <54b9d8c9.0bcc980a.4215.5779@mx.google.com> X-Google-Original-Message-ID: <003801d03206$ce9a36b0$6bcea410$@IM@gmail.com> From: rozhuk.im@gmail.com X-Google-Original-From: Reply-To: To: "'Pawel Jakub Dawidek'" References: <20150114041708.GA3189@reks> <54b601ec.0515980a.0c9c.47e1@mx.google.com> <20150114082019.GA3669@reks> <54b6ae4c.0905990a.6c9c.642e@mx.google.com> <54b6b91b.2aa3700a.3a6c.47b5@mx.google.com> <54B6C6B7.4070407@platinum.linux.pl> <54b709fb.0739700a.2970.ffffa14a@mx.google.com> <20150115150316.GB1190@garage.freebsd.pl> <54b85491.4925980a.17c4.2b00@mx.google.com> <20150116082104.GA1230@garage.freebsd.pl> In-Reply-To: <20150116082104.GA1230@garage.freebsd.pl> Subject: RE: ChaCha8/12/20 and GEOM ELI tests Date: Sat, 17 Jan 2015 06:36:38 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdAxZSXWZ6IrmADnR9qgPKNlGTIWpgAn2jKw Content-Language: ru Cc: freebsd-hackers@freebsd.org, 'freebsd-geom' , 'Adam Nowacki' X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jan 2015 03:36:44 -0000 > > Options I have not so much. > > 1. Drink vodka and use slow AES-XTS :) 2. Use ChaCha GELI private > > patch 3. Write Geom node. > > 4. Look at GBDE. Already looked. Do not like it. > > Cipher = ChaCha/XChaCha > > Hash = Blake2 - https://blake2.net/ > > Key1 = key for cipher > > Key2 = key hor HMAC > > IV = HMAC(Key2, ('plain text data' + 'sector num')) = (8/24 bytes) > > > > IV stored on disk in two tables: main and back up > > 1 GB (4kb sector) require 2097152 / 6291456 bytes per IV table or > > 16777216 for full 512 bit hmac > > > > +: > > 1. optional data integrity verification (authentication) 2. > > cache-timing attack resistant 3. keys can be changed without > > transferring data to other media and minimal risk > > > > -: > > 1. very slow write: it is necessary to calculate the hmac and update > > two tables with IV data 2. slow reading: IV need to get from the > table > > (+ optional hmac calc) 3. the risk of damage IV table on disk > hardware > > problems 4. part of the disk is busy service data (IV tables) > > This would be very hard to implement correctly, because writing the > data and updating your tables are not atomic on disk. How do you handle > the case where you write new data, but your system crash or you have a > power outage before updating the table? > > I came into conclusion that data authentication doesn't belong in the > layer of disk encryption, because of lack of atomicity. GBDE has this > problem and GELI data authentication has similar problem. This problem > is mitigated by ZFS, which is transactional, copy-on-write file system > that never overwrites existing data. I personally use ZFS with SHA256 > checksum on top of GELI. That's why I wrote 2 tables and a very slow write: write IV in the main table, write the data in the sector, IV write back table. After entering key need to check both tables, even if they differ then try to decrypt the data from the sector using both IV thus determine which one is correct and update is not correct IV. It's all like me much less than private patch with ChaCha or slow AES-XTS. I try to make Threefish-XTS there any objections? PS: AMD 5350 AES-XTS-256 1 core = 38781004 bytes/sec 1 core AES-NI = 418601975 bytes/sec 4 core = 143620811 bytes/sec 4 core AES-NI = 652957361 bytes/sec ChaCha8-256-1-core = 243108762 bytes/sec ChaCha8-256-4-core = 544313467 bytes/sec ChaCha12-256-1-core = 197178668 bytes/sec ChaCha12-256-4-core = 480021949 bytes/sec From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 17 16:57:51 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BF945A4B; Sat, 17 Jan 2015 16:57:51 +0000 (UTC) Received: from astart2.astart.com (108-248-95-193.lightspeed.sndgca.sbcglobal.net [108.248.95.193]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 944A0E32; Sat, 17 Jan 2015 16:57:50 +0000 (UTC) Received: from laptop_93.private (localhost [127.0.0.1]) by astart2.astart.com (8.14.4/8.14.4) with ESMTP id t0HGvhkh043247; Sat, 17 Jan 2015 08:57:44 -0800 (PST) (envelope-from papowell@astart.com) Message-ID: <54BA9487.2040509@astart.com> Date: Sat, 17 Jan 2015 08:57:43 -0800 From: Patrick Powell Reply-To: papowell@astart.com Organization: Astart Technologies User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Ed Maste Subject: Re: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged References: <54B7656D.9000704@freebsd.org> <54B927F9.1010401@astart.com> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jan 2015 16:57:51 -0000 On 01/16/15 13:56, Ed Maste wrote: > On 16 January 2015 at 10:02, Patrick Powell wrote: >> On 01/14/15 22:59, Allan Jude wrote: >>> Glad to see this, thanks for doing the work Ravi. Also, I agree with jhb@, >>> we should disable the test by default in stable/10 (i think it is off by >>> default only for VMs currently). >> >> Please please do not disable memory tests by default. If you do, when >> trying to boot from a CD/Memory Stick >> image on a system with bad memory (which would be found by the tests) then >> it gets quite difficult to find this >> problem. > The boot time "memory test" is not particularly valuable, especially > on contemporary amd64 hardware. While it won't have any false > positives, there are a huge number of failure modes it will not catch. > It also does not inform the user of "failure" -- it just removes that > memory from the kernel's map. It's really a test of memory presence, > not quality. Right. But at least it gets you crawling... or staggering... so you can do further diagnostics. Bad memory is a *((*&^( NASTY problem. Since we seem to have diverged a bit on topic, any recommendations for stand-alone memory tests? -- Patrick Powell Astart Technologies papowell@astart.com 1530 Jamacha Rd, Suite X Network and System San Diego, CA 92019 Consulting 858-874-6543 FAX 858-751-2435 Web: www.astart.com From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 17 17:12:30 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DBEAFE80 for ; Sat, 17 Jan 2015 17:12:29 +0000 (UTC) Received: from mail-yk0-f175.google.com (mail-yk0-f175.google.com [209.85.160.175]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A7078FD3 for ; Sat, 17 Jan 2015 17:12:29 +0000 (UTC) Received: by mail-yk0-f175.google.com with SMTP id q9so636372ykb.6 for ; Sat, 17 Jan 2015 09:12:23 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=GqEn/9kyk31XCSwURLDDyHV80nOM5mvIxzeFl4yYDnI=; b=AxKj49O4hSG9FWdYm3UAeyXomtg2OxBqV1rAq5E6GsEVRIyXuF/iuUfFgDpIcaSFmZ Y1A5svFrfVzzgmrYVaM7pvQajKbL8BWnlxzcgZwYKGw5BnUMNqOCQHIBcgDGmQqbwpT6 lmAELHhRO82gJpuBjAUnuaS2SQj9C2KaE9Uc6RRD33NW9abpLDIZHMiPH2lahFLa2mNq N8SKhT/ohzYP7vK5kUL6x9r+1HxNke/1+yu5sBH8yuN7sbfDCR+HSQ/j142SkXCnESRj GVQC3cY0oHB1PgjNCIfuK0+/dTNIg+d97FS2dEHeCcQkEHxChqHD3xzqGNphlqgCwzl7 hDAQ== X-Gm-Message-State: ALoCoQlK7K4B4md91b0Jz+JTI+Nr4AgvGEMReKrLuG83gYnmSW3tcq6DtufX48sbNKHKiXmjUVwr MIME-Version: 1.0 X-Received: by 10.236.21.234 with SMTP id r70mr13175708yhr.70.1421514742978; Sat, 17 Jan 2015 09:12:22 -0800 (PST) Received: by 10.170.46.213 with HTTP; Sat, 17 Jan 2015 09:12:22 -0800 (PST) In-Reply-To: <54BA9487.2040509@astart.com> References: <54B7656D.9000704@freebsd.org> <54B927F9.1010401@astart.com> <54BA9487.2040509@astart.com> Date: Sat, 17 Jan 2015 18:12:22 +0100 Message-ID: Subject: Re: [PATCH] Display progress during getmemsize() so the kernel doesn't look like it hanged From: Oliver Pinter To: papowell@astart.com Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" , Ed Maste X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jan 2015 17:12:30 -0000 On Sat, Jan 17, 2015 at 5:57 PM, Patrick Powell wrote: > On 01/16/15 13:56, Ed Maste wrote: >> >> On 16 January 2015 at 10:02, Patrick Powell wrote: >>> >>> On 01/14/15 22:59, Allan Jude wrote: >>>> >>>> Glad to see this, thanks for doing the work Ravi. Also, I agree with >>>> jhb@, >>>> we should disable the test by default in stable/10 (i think it is off by >>>> default only for VMs currently). >>> >>> >>> Please please do not disable memory tests by default. If you do, when >>> trying to boot from a CD/Memory Stick >>> image on a system with bad memory (which would be found by the tests) >>> then >>> it gets quite difficult to find this >>> problem. >> >> The boot time "memory test" is not particularly valuable, especially >> on contemporary amd64 hardware. While it won't have any false >> positives, there are a huge number of failure modes it will not catch. >> It also does not inform the user of "failure" -- it just removes that >> memory from the kernel's map. It's really a test of memory presence, >> not quality. > > > Right. But at least it gets you crawling... or staggering... so you can do > further diagnostics. > > Bad memory is a *((*&^( NASTY problem. > > Since we seem to have diverged a bit on topic, any recommendations for > stand-alone memory tests? I tried memtest86+ from ports and from pkg, but when I load it from loader via load /boot/opt/memtest86+ I got constant reboot. And fully reproducible with real and virtual machine too. > > -- > Patrick Powell Astart Technologies > papowell@astart.com 1530 Jamacha Rd, Suite X > Network and System San Diego, CA 92019 > Consulting 858-874-6543 FAX 858-751-2435 > Web: www.astart.com > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 17 17:59:07 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 23F7DECC for ; Sat, 17 Jan 2015 17:59:07 +0000 (UTC) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE2A66AF for ; Sat, 17 Jan 2015 17:59:06 +0000 (UTC) Received: by mail-wi0-f179.google.com with SMTP id ho1so9399946wib.0 for ; Sat, 17 Jan 2015 09:59:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=rFNxXWIfYXbRoW3Rc1QGP9+WJZzydz1XVst2903cC6o=; b=utrCdSfx/aLW5e7y4Mnq9eqfgpivNbAOF8AgKxUzZXGXv+fTsyvmyYQUoZE1dnyESC 2vxWm5o9orvWmzMw9Y059BtBLZPTh72suQ4tp5YVKJdTCVepWqKh1PbtMD2BnUuPvdf1 AeKKaYfT3YKtyd/F//TEyZLamAAyYtZvffQnDG/AUHIFAZW0YJWfobDY9Q0fE6Bh9vEU Jfwr7VO3WkZiPJ2l2VEDmMMPAbCEPd2el066Ge7loN4dx19nLJ3faClSTlXverEW7IUz YB3jPMYgx2m43Iq8bg/k025zv+I+fA1Of7an4pZOu6K0EdKIhZUEtT2p2E+RhrlFWIV/ QiWQ== MIME-Version: 1.0 X-Received: by 10.194.84.240 with SMTP id c16mr41819682wjz.74.1421517545160; Sat, 17 Jan 2015 09:59:05 -0800 (PST) Received: by 10.27.228.201 with HTTP; Sat, 17 Jan 2015 09:59:05 -0800 (PST) Date: Sat, 17 Jan 2015 12:59:05 -0500 Message-ID: Subject: ctrl-d appends characters to output From: less xss To: freebsd-hackers@freebsd.org X-Mailman-Approved-At: Sat, 17 Jan 2015 19:20:53 +0000 Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jan 2015 17:59:07 -0000 I've searched around quite a bit with no luck on this matter. I currently have an issue where I send EOF (ctrl-d) to some simple K&R2 exercises and the terminal returns the D character appended to my data when EOF is sent. I wish to prevent any and all extra characters from being appended and I would also like to understand why it's happening. The following code is an example exercise from K&R2 that yield said problem. #include int main() { double nc; for (nc = 0; getchar() != EOF; ++nc) { ; /* syntactic null statement */ } printf("%.0f\n", nc); } $ ./a.out 0D $ From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 17 19:43:15 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3F481A6 for ; Sat, 17 Jan 2015 19:43:15 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 03925FD6 for ; Sat, 17 Jan 2015 19:43:14 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t0HJh8Z9023074 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 17 Jan 2015 11:43:08 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t0HJh8lr023073; Sat, 17 Jan 2015 11:43:08 -0800 (PST) (envelope-from jmg) Date: Sat, 17 Jan 2015 11:43:08 -0800 From: John-Mark Gurney To: less xss Subject: Re: ctrl-d appends characters to output Message-ID: <20150117194308.GH1949@funkthat.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Sat, 17 Jan 2015 11:43:08 -0800 (PST) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jan 2015 19:43:15 -0000 less xss wrote this message on Sat, Jan 17, 2015 at 12:59 -0500: > I've searched around quite a bit with no luck on this matter. I currently > have an issue where I send EOF (ctrl-d) to some simple K&R2 exercises and > the terminal returns the D character appended to my data when EOF is sent. > I wish to prevent any and all extra characters from being appended and I > would also like to understand why it's happening. The following code > is an example exercise from K&R2 that yield said problem. > > #include > > int main() { > double nc; > > for (nc = 0; getchar() != EOF; ++nc) { > ; /* syntactic null statement */ > } > > printf("%.0f\n", nc); > } > > $ ./a.out > 0D > $ This problem exists on pretty much any Unix based platform and is not FreeBSD specific... The D is not getting appended to your output... If you changed your printf to include a space or two before the new line, you wouldn't see the D... the D is left after the kernel echoed ^D in response to your ctrl-D. If you redirect the output to a file and examined the output, there would be no D in your output... Or if you provided input from another source that didn't require an EOF to trigger it, such as: $echo | ./t 1 -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 17 19:49:15 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 23F242CE for ; Sat, 17 Jan 2015 19:49:15 +0000 (UTC) Received: from ms-10.1blu.de (ms-10.1blu.de [178.254.4.101]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D6A4E8F for ; Sat, 17 Jan 2015 19:49:14 +0000 (UTC) Received: from [89.15.237.229] (helo=c720-r276659) by ms-10.1blu.de with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.76) (envelope-from ) id 1YCZMn-0006pl-OR; Sat, 17 Jan 2015 20:49:06 +0100 Date: Sat, 17 Jan 2015 20:49:02 +0100 From: Matthias Apitz To: John-Mark Gurney Subject: Re: ctrl-d appends characters to output Message-ID: <20150117194902.GA1491@c720-r276659> Reply-To: Matthias Apitz Mail-Followup-To: Matthias Apitz , John-Mark Gurney , less xss , freebsd-hackers@freebsd.org References: <20150117194308.GH1949@funkthat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20150117194308.GH1949@funkthat.com> X-Operating-System: FreeBSD 11.0-CURRENT r269739 (i386) User-Agent: Mutt/1.5.23 (2014-03-12) X-Con-Id: 51246 X-Con-U: 0-guru X-Originating-IP: 89.15.237.229 Cc: freebsd-hackers@freebsd.org, less xss X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jan 2015 19:49:15 -0000 El día Saturday, January 17, 2015 a las 11:43:08AM -0800, John-Mark Gurney escribió: > The D is not getting appended to your output... If you changed your > printf to include a space or two before the new line, you wouldn't > see the D... the D is left after the kernel echoed ^D in response to your > ctrl-D. If you redirect the output to a file and examined the output, > there would be no D in your output... Or if you provided input from > another source that didn't require an EOF to trigger it, such as: > $echo | ./t > 1 $ echo -n '' | ./a.out 0 matthias -- Matthias Apitz, guru@unixarea.de, http://www.unixarea.de/ +49-170-4527211 1989-2014: The Wall was torn down so that we go to war together again. El Muro ha sido derribado para que nos unimos en ir a la guerra otra vez. Diese Grenze wurde aufgehoben damit wir gemeinsam wieder in den Krieg ziehen. From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 17 19:58:15 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ABEB95CB for ; Sat, 17 Jan 2015 19:58:15 +0000 (UTC) Received: from ms-10.1blu.de (ms-10.1blu.de [178.254.4.101]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6AF2315F for ; Sat, 17 Jan 2015 19:58:14 +0000 (UTC) Received: from [89.15.237.229] (helo=c720-r276659) by ms-10.1blu.de with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.76) (envelope-from ) id 1YCZCw-0001Ru-Fu; Sat, 17 Jan 2015 20:38:54 +0100 Date: Sat, 17 Jan 2015 20:38:48 +0100 From: Matthias Apitz To: less xss Subject: Re: ctrl-d appends characters to output Message-ID: <20150117193848.GA1394@c720-r276659> Reply-To: Matthias Apitz Mail-Followup-To: Matthias Apitz , less xss , freebsd-hackers@freebsd.org References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Operating-System: FreeBSD 11.0-CURRENT r269739 (i386) User-Agent: Mutt/1.5.23 (2014-03-12) X-Con-Id: 51246 X-Con-U: 0-guru X-Originating-IP: 89.15.237.229 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jan 2015 19:58:15 -0000 El día Saturday, January 17, 2015 a las 12:59:05PM -0500, less xss escribió: > I've searched around quite a bit with no luck on this matter. I currently > have an issue where I send EOF (ctrl-d) to some simple K&R2 exercises and > the terminal returns the D character appended to my data when EOF is sent. > I wish to prevent any and all extra characters from being appended and I > would also like to understand why it's happening. The following code > is an example exercise from K&R2 that yield said problem. > > #include > > int main() { > double nc; > > for (nc = 0; getchar() != EOF; ++nc) { > ; /* syntactic null statement */ > } > > printf("%.0f\n", nc); > } > > $ ./a.out > 0D > $ This is just a display issue of your typed Ctrl-d; the '0D' is the '^D' where the '^' is overwritten by your output of the value of 'nc'; run your a.out as: $ ./a.out < /dev/null or change the printf(3) line to: printf("\n%.0f\n", nc); HIH matthias -- Matthias Apitz, guru@unixarea.de, http://www.unixarea.de/ +49-170-4527211 1989-2014: The Wall was torn down so that we go to war together again. El Muro ha sido derribado para que nos unimos en ir a la guerra otra vez. Diese Grenze wurde aufgehoben damit wir gemeinsam wieder in den Krieg ziehen. From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 17 20:17:20 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2ED0484B for ; Sat, 17 Jan 2015 20:17:20 +0000 (UTC) Received: from um-nip3-missouri-out.um.umsystem.edu (um-nip3-missouri-out.um.umsystem.edu [198.209.49.163]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "um-tip1.um.umsystem.edu", Issuer "InCommon RSA Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B6306324 for ; Sat, 17 Jan 2015 20:17:18 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AiUFABvCulTPoJ7U/2dsb2JhbABagwaBLsQJhUeCTwKBDkMBAQEBAQN6hA0BBXgRAgEIGAkWDwkDAgECASAlAgQBDAgBAYgoy0QBg1cBCwEfj0Y6hCkFjkybGSKDboI0fgEBAQ X-IPAS-Result: AiUFABvCulTPoJ7U/2dsb2JhbABagwaBLsQJhUeCTwKBDkMBAQEBAQN6hA0BBXgRAgEIGAkWDwkDAgECASAlAgQBDAgBAYgoy0QBg1cBCwEfj0Y6hCkFjkybGSKDboI0fgEBAQ Received: from um-ncas6.um.umsystem.edu ([207.160.158.212]) by um-nip3-exch-relay.um.umsystem.edu with ESMTP; 17 Jan 2015 14:16:07 -0600 Received: from UM-MBX-N02.um.umsystem.edu ([169.254.5.23]) by UM-NCAS6.um.umsystem.edu ([207.160.158.212]) with mapi id 14.03.0210.002; Sat, 17 Jan 2015 14:16:07 -0600 From: "Montgomery-Smith, Stephen" To: less xss , "freebsd-hackers@freebsd.org" Subject: Re: ctrl-d appends characters to output Thread-Topic: ctrl-d appends characters to output Thread-Index: AQHQMoq4BXOKvIQ6l0u+m3Cgq79pjZzFJFMA Date: Sat, 17 Jan 2015 20:16:07 +0000 Message-ID: <54BAC302.7050800@missouri.edu> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Mozilla/5.0 (X11; Linux i686; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 x-originating-ip: [207.160.158.206] Content-Type: text/plain; charset="Windows-1252" Content-ID: <311E36D74088C645B99EAA20E4AD076B@missouri.edu> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jan 2015 20:17:20 -0000 On 01/17/2015 11:59 AM, less xss wrote: > I've searched around quite a bit with no luck on this matter. I currently > have an issue where I send EOF (ctrl-d) to some simple K&R2 exercises and > the terminal returns the D character appended to my data when EOF is sent= . > I wish to prevent any and all extra characters from being appended and I > would also like to understand why it's happening. The following code > is an example exercise from K&R2 that yield said problem. >=20 > #include >=20 > int main() { > double nc; >=20 > for (nc =3D 0; getchar() !=3D EOF; ++nc) { > ; /* syntactic null statement */ > } >=20 > printf("%.0f\n", nc); > } >=20 > $ ./a.out > 0D > $ I did a bit of experimenting with this issue. First, I cannot reproduce it on my Linux box. Second, this simpler program does the same thing: #include int main() { while (getchar() !=3D EOF) { ; /* syntactic null statement */ } printf("\n"); } In this case I get: % ./a.out ^D % However, if I remove that last printf statement, then no ^D is displayed. Considering the inconsistent nature of when this ^D appears, I would prefer to call it a bug than a feature. But it must have been put there by design. From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 17 21:02:29 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 11091F10 for ; Sat, 17 Jan 2015 21:02:29 +0000 (UTC) Received: from um-nip4-missouri-out.um.umsystem.edu (um-nip4-missouri-out.um.umsystem.edu [198.209.49.177]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "um-tip1.um.umsystem.edu", Issuer "InCommon RSA Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 97F3395D for ; Sat, 17 Jan 2015 21:02:28 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AiUFAB/NulTPoJ7U/2dsb2JhbABZgwaBLsQJhUeCTwKBDkMBAQEBAQN6hA0BBXgRAgEIGAkWDwkDAgECASAlAgQBDAgBAYgoyzwBg1cBCgEBAR6PRjqEKQWOTJsZIoNugjR+AQEB X-IPAS-Result: AiUFAB/NulTPoJ7U/2dsb2JhbABZgwaBLsQJhUeCTwKBDkMBAQEBAQN6hA0BBXgRAgEIGAkWDwkDAgECASAlAgQBDAgBAYgoyzwBg1cBCgEBAR6PRjqEKQWOTJsZIoNugjR+AQEB Received: from um-ncas6.um.umsystem.edu ([207.160.158.212]) by um-nip4-exch-relay.um.umsystem.edu with ESMTP; 17 Jan 2015 15:01:06 -0600 Received: from UM-MBX-N02.um.umsystem.edu ([169.254.5.23]) by UM-NCAS6.um.umsystem.edu ([207.160.158.212]) with mapi id 14.03.0210.002; Sat, 17 Jan 2015 15:01:05 -0600 From: "Montgomery-Smith, Stephen" To: less xss , "freebsd-hackers@freebsd.org" Subject: Re: ctrl-d appends characters to output Thread-Topic: ctrl-d appends characters to output Thread-Index: AQHQMoq4BXOKvIQ6l0u+m3Cgq79pjZzFJFMAgAAMkQA= Date: Sat, 17 Jan 2015 21:01:04 +0000 Message-ID: <54BACD8C.3050703@missouri.edu> References: <54BAC302.7050800@missouri.edu> In-Reply-To: <54BAC302.7050800@missouri.edu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Mozilla/5.0 (X11; Linux i686; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 x-originating-ip: [207.160.158.206] Content-Type: text/plain; charset="Windows-1252" Content-ID: <9BBE76D5F2FDD34EAFCA44F867181257@missouri.edu> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jan 2015 21:02:29 -0000 On 01/17/2015 02:16 PM, Montgomery-Smith, Stephen wrote: > On 01/17/2015 11:59 AM, less xss wrote: >> I've searched around quite a bit with no luck on this matter. I currentl= y >> have an issue where I send EOF (ctrl-d) to some simple K&R2 exercises an= d >> the terminal returns the D character appended to my data when EOF is sen= t. >> I wish to prevent any and all extra characters from being appended and I >> would also like to understand why it's happening. The following code >> is an example exercise from K&R2 that yield said problem. >> >> #include >> >> int main() { >> double nc; >> >> for (nc =3D 0; getchar() !=3D EOF; ++nc) { >> ; /* syntactic null statement */ >> } >> >> printf("%.0f\n", nc); >> } >> >> $ ./a.out >> 0D >> $ >=20 > I did a bit of experimenting with this issue. First, I cannot reproduce > it on my Linux box. Second, this simpler program does the same thing: >=20 > #include >=20 > int main() { >=20 > while (getchar() !=3D EOF) { > ; /* syntactic null statement */ > } >=20 > printf("\n"); > } >=20 > In this case I get: >=20 > % ./a.out > ^D > % >=20 > However, if I remove that last printf statement, then no ^D is displayed. >=20 > Considering the inconsistent nature of when this ^D appears, I would > prefer to call it a bug than a feature. But it must have been put there > by design. OK, that last printf is NOT responsible for the ^D. It is just that the prompt wipes it out. Try this code: #include #include int main() { while (getchar() !=3D EOF) { ; /* syntactic null statement */ } sleep(10); } Then the ^D shows until the prompt appears 10 seconds later. From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 17 21:07:55 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE12E9F for ; Sat, 17 Jan 2015 21:07:55 +0000 (UTC) Received: from mst-rip6-missouri-out.um.umsystem.edu (mst-rip6-missouri-out.um.umsystem.edu [198.209.50.149]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "um-tip1.um.umsystem.edu", Issuer "InCommon RSA Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8AFB0988 for ; Sat, 17 Jan 2015 21:07:55 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AiUFAGHNulTPoJ7U/2dsb2JhbABZgwaBLsQJhUeCTwKBDkMBAQEBAQN6hA0BBXgRAgEIGAkWDwkDAgECASAlAgQBDAgBAYgoyzsBg1cBCgEBAR6PRjqEKQWOTJsZIoNugjR+AQEB X-IPAS-Result: AiUFAGHNulTPoJ7U/2dsb2JhbABZgwaBLsQJhUeCTwKBDkMBAQEBAQN6hA0BBXgRAgEIGAkWDwkDAgECASAlAgQBDAgBAYgoyzsBg1cBCgEBAR6PRjqEKQWOTJsZIoNugjR+AQEB Received: from um-ncas6.um.umsystem.edu ([207.160.158.212]) by mst-rip6-exch-relay.um.umsystem.edu with ESMTP; 17 Jan 2015 15:06:45 -0600 Received: from UM-MBX-N02.um.umsystem.edu ([169.254.5.23]) by UM-NCAS6.um.umsystem.edu ([207.160.158.212]) with mapi id 14.03.0210.002; Sat, 17 Jan 2015 15:06:45 -0600 From: "Montgomery-Smith, Stephen" To: less xss , "freebsd-hackers@freebsd.org" Subject: Re: ctrl-d appends characters to output Thread-Topic: ctrl-d appends characters to output Thread-Index: AQHQMoq4BXOKvIQ6l0u+m3Cgq79pjZzFJFMAgAAMkQCAAAGZgA== Date: Sat, 17 Jan 2015 21:06:44 +0000 Message-ID: <54BACEE3.9080500@missouri.edu> References: <54BAC302.7050800@missouri.edu> <54BACD8C.3050703@missouri.edu> In-Reply-To: <54BACD8C.3050703@missouri.edu> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Mozilla/5.0 (X11; Linux i686; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 x-originating-ip: [207.160.158.206] Content-Type: text/plain; charset="Windows-1252" Content-ID: <50E9E6513A254A408ACE9A432FF002C6@missouri.edu> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jan 2015 21:07:56 -0000 On 01/17/2015 03:01 PM, Montgomery-Smith, Stephen wrote: > On 01/17/2015 02:16 PM, Montgomery-Smith, Stephen wrote: >> On 01/17/2015 11:59 AM, less xss wrote: >>> I've searched around quite a bit with no luck on this matter. I current= ly >>> have an issue where I send EOF (ctrl-d) to some simple K&R2 exercises a= nd >>> the terminal returns the D character appended to my data when EOF is se= nt. >>> I wish to prevent any and all extra characters from being appended and = I >>> would also like to understand why it's happening. The following code >>> is an example exercise from K&R2 that yield said problem. >>> >>> #include >>> >>> int main() { >>> double nc; >>> >>> for (nc =3D 0; getchar() !=3D EOF; ++nc) { >>> ; /* syntactic null statement */ >>> } >>> >>> printf("%.0f\n", nc); >>> } >>> >>> $ ./a.out >>> 0D >>> $ >> >> I did a bit of experimenting with this issue. First, I cannot reproduce >> it on my Linux box. Second, this simpler program does the same thing: >> >> #include >> >> int main() { >> >> while (getchar() !=3D EOF) { >> ; /* syntactic null statement */ >> } >> >> printf("\n"); >> } >> >> In this case I get: >> >> % ./a.out >> ^D >> % >> >> However, if I remove that last printf statement, then no ^D is displayed= . >> >> Considering the inconsistent nature of when this ^D appears, I would >> prefer to call it a bug than a feature. But it must have been put there >> by design. >=20 > OK, that last printf is NOT responsible for the ^D. It is just that the > prompt wipes it out. Try this code: >=20 > #include > #include >=20 > int main() { >=20 > while (getchar() !=3D EOF) { > ; /* syntactic null statement */ > } > sleep(10); > } >=20 > Then the ^D shows until the prompt appears 10 seconds later. Even simpler - you don't even have to write a program: cat | sleep 10 From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 17 19:28:26 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9CA4DEE2 for ; Sat, 17 Jan 2015 19:28:26 +0000 (UTC) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 5DC97E65 for ; Sat, 17 Jan 2015 19:28:26 +0000 (UTC) Received: from critter.freebsd.dk (unknown [192.168.60.3]) by phk.freebsd.dk (Postfix) with ESMTP id 8B0DB3BD3D; Sat, 17 Jan 2015 19:28:18 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.9/8.14.9) with ESMTP id t0HJSHuD090953; Sat, 17 Jan 2015 19:28:17 GMT (envelope-from phk@phk.freebsd.dk) To: less xss Subject: Re: ctrl-d appends characters to output In-reply-to: From: "Poul-Henning Kamp" References: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <90951.1421522897.1@critter.freebsd.dk> Content-Transfer-Encoding: quoted-printable Date: Sat, 17 Jan 2015 19:28:17 +0000 Message-ID: <90952.1421522897@critter.freebsd.dk> X-Mailman-Approved-At: Sat, 17 Jan 2015 21:33:38 +0000 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jan 2015 19:28:26 -0000 -------- In message , less xss writes: > printf("%.0f\n", nc); Change to: > printf("\n%.0f\n", nc); Also, try: ./a.out | hexdump -C -- = Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe = Never attribute to malice what can adequately be explained by incompetence= .