Date: Thu, 31 Dec 2015 02:15:18 -0500 From: Allan Jude <allanjude@freebsd.org> To: Bruce Evans <brde@optusnet.com.au> Cc: "Jonathan T. Looney" <jtl@freebsd.org>, src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r292955 - head/lib/libmd Message-ID: <5684D606.3080609@freebsd.org> In-Reply-To: <20151231143314.Y1520@besplex.bde.org> References: <201512301804.tBUI4oGp065466@repo.freebsd.org> <20151231115651.R995@besplex.bde.org> <20151231143314.Y1520@besplex.bde.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--cefVxOIBD2vm4RApBE5O4tQ0lfigJF2Qm
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable
On 2015-12-30 23:10, Bruce Evans wrote:
> On Thu, 31 Dec 2015, Bruce Evans wrote:
>=20
>>
>> wc /proc/0/* works. md5 works like wc using a hack to avoid calling t=
his
>> broken function. E.g.,
>>
>> for i in $(ls /proc/0/*); do echo -n "$i: "; md5 <$i; done # gives=
>>
>> /proc/0/cmdline: 3c5896b1ac441f4998f052e2126e8d20
>> /proc/0/ctl: d41d8cd98f00b204e9800998ecf8427e
>> /proc/0/etype: 674441960ca1ba2de08ad4e50c9fde98
>> /proc/0/rlimit: 67d6ad67b412e1bceb7cb508a3492197
>> /proc/0/status: 3ccc3067b97c3232ea2dbcb64c458fd4
>=20
> Further examples:
>=20
> md5 # on terminal input
>=20
> works correctly by not using MDXFileChunk().
>=20
> md5 /dev/stdin # on the same terminal input
>=20
> produces the d41d8cd98f00b204e9800998ecf8427e garbage using
> MDXFileChunk(). truss shows that lseek() is broken too -- MDXFileChunk=
()
> tries it and it succeeds on the unseekable file /dev/stdin. Bugs in th=
is
> area are common. E.g., lseek() on named pipes was broken so that it
> succeeded, by rearranging the plumbing use fileops more or less and not=
> attaching lseek right.
>=20
> cat | md5 # on the same terminal input
>=20
> works correctly by not using MDXFileChunk().
>=20
> cat | md5 /dev/stdin # on the same terminal input
>=20
> doesn't work correctly, but it fails better than without the pipe. Now=
> a seek error occurs and is reported as "md5: /dev/stdin: Illegal seek"
> (I can't see where it is reported). Then md5 exits. Then cat waits to=
> read input. Then cat fails to write output and is killed by SIGPIPE.
> So md5 handled the seek error in a fail-safe though incorrect way. One=
> correct way is to fall back to the working code, but it is better to
> just use that without an lseek check.
>=20
> It is a bug that [l]stat() on /dev/stdin sees the device file and not t=
he
> actual stdin file. md5 can't work around this except by not using stat=
().
>=20
> Bruce
>=20
It seems these problems also slow things down, a lot:
# time md5 /media/md5test/bigdata
MD5 (/media/md5test/bigdata) =3D 6afad0bf5d8318093e943229be05be67
4.310u 3.476s 0:07.79 99.8% 20+167k 0+0io 0pf+0w
# time env LD_PRELOAD=3D/usr/obj/media/svn/md5/head/tmp/lib/libmd.so
/usr/obj/media/svn/md5/head/sbin/md5/md5 /media/md5test/bigdata
MD5 (/media/md5test/bigdata) =3D 6afad0bf5d8318093e943229be05be67
4.133u 0.354s 0:04.49 99.7% 20+167k 1+0io 0pf+0w
(file is fully cached in ZFS ARC, dd reads it at 11GB/s)
Will investigate more tomorrow.
char *
MDXFile(const char *filename, char *buf)
{
unsigned char buffer[16*1024];
MDX_CTX ctx;
int f, i;
MDXInit(&ctx);
f =3D open(filename, O_RDONLY);
if (f < 0)
return 0;
i =3D 0;
while (1) {
i =3D read(f, buffer, sizeof(buffer));
if (i <=3D 0)
break;
MDXUpdate(&ctx, buffer, i);
}
close(f);
return (MDXEnd(&ctx, buf));
}
--=20
Allan Jude
--cefVxOIBD2vm4RApBE5O4tQ0lfigJF2Qm
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (MingW32)
iQIcBAEBAgAGBQJWhNYKAAoJEBmVNT4SmAt+E8cP/3R383YXkUdn1jV4haIADU/r
SiVvOgU6eNOlrPv7koeL7A7qrkb23/Hs+D+ucXChaTx5cu8EBMdLgRrL4dKnu/tW
CyXMpq8k1i2h9PLdFiAM+SybVJ5FFkZdMwYvc17DxWkIGRyGuYteOE6QTB4IE2IW
V8FM7I372IpDGWzTyBzPhg7kO83WYrN3MfisC9PYuwQN2JoEg5TY5sjp84qiIJzl
OQjf6ZAQ2wBXqhF90L97Wz2dzP+IqJ8RovfP6yajGjC432aLQMoEEd/lb8OS6ftd
Fr68jJhmjGSCtSU0Ix+SFhofDe8p3AJuodU+XAaXIpMl/m0jNqW3TDsjyopHBY2L
+W4QjIbx1IZ4a9v8q33hPnFmY+WMhrhD7I/g6cqAFK81Ahfi9NfKhidDosjvHJVl
rFn/hxyv+qy7mvGhYFd/FkyNmXFKTfv3ZQC+C6x9hVE2jg4gC6zA4yCsNq1tSN40
50Nt2wCo2m5Zr+QyFx6o9JyRP0Z2FNW0DBy9hloUASg3+LgJ7dpCFKVvwsJR/h7h
/Odz1hYe3kHR9Zo2Zh+6PE21FMA2OzcrT6XLPbiLbRqLHhuGKBSt08XZEa2pm50m
yMG5SwYLKfB1kkqHKAScvZB65Cnq11t60xBHB1Al1HySGawwVib9tZSc5ehu8Ss9
yKFYIitpyPSRnpBV3WIh
=S8Jb
-----END PGP SIGNATURE-----
--cefVxOIBD2vm4RApBE5O4tQ0lfigJF2Qm--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5684D606.3080609>