From owner-svn-src-user@freebsd.org Tue Sep 3 14:07:26 2019 Return-Path: Delivered-To: svn-src-user@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0EBF1DDAD2 for ; Tue, 3 Sep 2019 14:07:26 +0000 (UTC) (envelope-from yuripv@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46N80Y3fyCz4QbM; Tue, 3 Sep 2019 14:07:25 +0000 (UTC) (envelope-from yuripv@freebsd.org) Received: by freefall.freebsd.org (Postfix, from userid 1452) id 0145C1B5FC; Tue, 3 Sep 2019 14:06:38 +0000 (UTC) X-Original-To: yuripv@localmail.freebsd.org Delivered-To: yuripv@localmail.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [96.47.72.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mx1.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by freefall.freebsd.org (Postfix) with ESMTPS id E4846D3BA; Tue, 23 Apr 2019 22:37:53 +0000 (UTC) (envelope-from owner-src-committers@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 487A48E49A; Tue, 23 Apr 2019 22:37:53 +0000 (UTC) (envelope-from owner-src-committers@freebsd.org) Received: by freefall.freebsd.org (Postfix, from userid 538) id 0AEC2D34E; Tue, 23 Apr 2019 22:37:53 +0000 (UTC) Delivered-To: src-committers@localmail.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mx1.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by freefall.freebsd.org (Postfix) with ESMTPS id 80B96D349 for ; Tue, 23 Apr 2019 22:37:50 +0000 (UTC) (envelope-from ngie@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 36D568E492; Tue, 23 Apr 2019 22:37:50 +0000 (UTC) (envelope-from ngie@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 112423128; Tue, 23 Apr 2019 22:37:50 +0000 (UTC) (envelope-from ngie@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x3NMbnvC071375; Tue, 23 Apr 2019 22:37:49 GMT (envelope-from ngie@FreeBSD.org) Received: (from ngie@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x3NMbn9a071373; Tue, 23 Apr 2019 22:37:49 GMT (envelope-from ngie@FreeBSD.org) Message-Id: <201904232237.x3NMbn9a071373@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: ngie set sender to ngie@FreeBSD.org using -f From: Enji Cooper To: src-committers@freebsd.org, svn-src-user@freebsd.org Subject: svn commit: r346610 - user/ngie/bug-237403/tests/sys/opencrypto X-SVN-Group: user X-SVN-Commit-Author: ngie X-SVN-Commit-Paths: user/ngie/bug-237403/tests/sys/opencrypto X-SVN-Commit-Revision: 346610 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk X-Loop: FreeBSD.org Sender: owner-src-committers@freebsd.org X-Rspamd-Queue-Id: 487A48E49A X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.96)[-0.963,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] Status: O X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.29 List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Date: Tue, 03 Sep 2019 14:07:27 -0000 X-Original-Date: Tue, 23 Apr 2019 22:37:49 +0000 (UTC) X-List-Received-Date: Tue, 03 Sep 2019 14:07:27 -0000 Author: ngie Date: Tue Apr 23 22:37:49 2019 New Revision: 346610 URL: https://svnweb.freebsd.org/changeset/base/346610 Log: KATParser: don't leak `self.fp` As noted by the python 3 interpreter, self.fp is leaked every time KATParser is called (it raises a `ResourceWarning`). Refactor KATParser in order to not leak `self.fp`: * Don't allocate `self.fp` in `__init__()`. Doing so requires adding `self.fp.close()`, and as noted before, makes for very fragile code when the interpreter is tearing down objects (it can result in crashes in some interpreter versions if triggered with the right conditions). Limit `self.fp` allocation/use to context suites ("the with statement") instead, as this is more likely to not cause non-deterministic weirdness issues with the interpreter. * Rename `__iter__` to `__next__` as it was providing `__next__` support and return the KATParser object instead with `__iter__`, as `__iter__` is meant to return an iterator and `__next__` is meant to help iterate over an iterator. Modified: user/ngie/bug-237403/tests/sys/opencrypto/cryptodev.py user/ngie/bug-237403/tests/sys/opencrypto/cryptotest.py Modified: user/ngie/bug-237403/tests/sys/opencrypto/cryptodev.py ============================================================================== --- user/ngie/bug-237403/tests/sys/opencrypto/cryptodev.py Tue Apr 23 22:34:32 2019 (r346609) +++ user/ngie/bug-237403/tests/sys/opencrypto/cryptodev.py Tue Apr 23 22:37:49 2019 (r346610) @@ -316,11 +316,23 @@ class MismatchError(Exception): class KATParser: def __init__(self, fname, fields): - self.fp = open(fname) self.fields = set(fields) self._pending = None + self.fname = fname + self.fp = None + def __enter__(self): + self.fp = open(self.fname) + return self + + def __exit__(self, exc_type, exc_value, exc_tb): + if self.fp is not None: + self.fp.close() + def __iter__(self): + return self + + def __next__(self): while True: didread = False if self._pending is not None: @@ -331,14 +343,15 @@ class KATParser: didread = True if didread and not i: - return + raise StopIteration - if (i and i[0] == '#') or not i.strip(): - continue - if i[0] == '[': - yield i[1:].split(']', 1)[0], self.fielditer() - else: - raise ValueError('unknown line: %r' % repr(i)) + if not i.startswith('#') and i.strip(): + break + + if i[0] == '[': + yield i[1:].split(']', 1)[0], self.fielditer() + else: + raise ValueError('unknown line: %r' % repr(i)) def eatblanks(self): while True: Modified: user/ngie/bug-237403/tests/sys/opencrypto/cryptotest.py ============================================================================== --- user/ngie/bug-237403/tests/sys/opencrypto/cryptotest.py Tue Apr 23 22:34:32 2019 (r346609) +++ user/ngie/bug-237403/tests/sys/opencrypto/cryptotest.py Tue Apr 23 22:37:49 2019 (r346610) @@ -94,8 +94,12 @@ def GenTestCase(cname): else: raise RuntimeError('unknown mode: %r' % repr(mode)) - for bogusmode, lines in cryptodev.KATParser(fname, - [ 'Count', 'Key', 'IV', 'CT', 'AAD', 'Tag', 'PT', ]): + columns = [ 'Count', 'Key', 'IV', 'CT', 'AAD', 'Tag', 'PT', ] + with cryptodev.KATParser(fname, columns) as parser: + self.runGCMWithParser(parser, mode) + + def runGCMWithParser(self, parser, mode): + for _, lines in next(parser): for data in lines: curcnt = int(data['Count']) cipherkey = binascii.unhexlify(data['Key']) @@ -155,9 +159,13 @@ def GenTestCase(cname): repr(data)) def runCBC(self, fname): + columns = [ 'COUNT', 'KEY', 'IV', 'PLAINTEXT', 'CIPHERTEXT', ] + with cryptodev.KATParser(fname, columns) as parser: + self.runCBCWithParser(parser) + + def runCBCWithParser(self, parser): curfun = None - for mode, lines in cryptodev.KATParser(fname, - [ 'COUNT', 'KEY', 'IV', 'PLAINTEXT', 'CIPHERTEXT', ]): + for mode, lines in next(parser): if mode == 'ENCRYPT': swapptct = False curfun = Crypto.encrypt @@ -182,10 +190,14 @@ def GenTestCase(cname): self.assertEqual(r, ct) def runXTS(self, fname, meth): + columns = [ 'COUNT', 'DataUnitLen', 'Key', 'DataUnitSeqNumber', 'PT', + 'CT'] + with cryptodev.KATParser(fname, columns) as parser: + self.runXTSWithParser(parser, meth) + + def runXTSWithParser(self, parser, meth): curfun = None - for mode, lines in cryptodev.KATParser(fname, - [ 'COUNT', 'DataUnitLen', 'Key', 'DataUnitSeqNumber', 'PT', - 'CT' ]): + for mode, lines in next(parser): if mode == 'ENCRYPT': swapptct = False curfun = Crypto.encrypt @@ -228,9 +240,13 @@ def GenTestCase(cname): self.runTDES(i) def runTDES(self, fname): + columns = [ 'COUNT', 'KEYs', 'IV', 'PLAINTEXT', 'CIPHERTEXT', ] + with cryptodev.KATParser(fname, columns) as parser: + self.runTDESWithParser(parser) + + def runTDESWithParser(self, parser): curfun = None - for mode, lines in cryptodev.KATParser(fname, - [ 'COUNT', 'KEYs', 'IV', 'PLAINTEXT', 'CIPHERTEXT', ]): + for mode, lines in next(parser): if mode == 'ENCRYPT': swapptct = False curfun = Crypto.encrypt @@ -271,8 +287,12 @@ def GenTestCase(cname): self.runSHA1HMAC(i) def runSHA1HMAC(self, fname): - for hashlength, lines in cryptodev.KATParser(fname, - [ 'Count', 'Klen', 'Tlen', 'Key', 'Msg', 'Mac' ]): + columns = [ 'Count', 'Klen', 'Tlen', 'Key', 'Msg', 'Mac' ] + with cryptodev.KATParser(fname, columns) as parser: + self.runSHA1HMACWithParser(parser) + + def runSHA1HMACWithParser(self, parser): + for hashlength, lines in next(parser): # E.g., hashlength will be "L=20" (bytes) hashlen = int(hashlength.split("=")[1])