From nobody Wed Jun 5 12:22:02 2024 X-Original-To: dev-commits-ports-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VvRRG6BtFz5MmVx; Wed, 05 Jun 2024 12:22:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VvRRG3fQvz4lmM; Wed, 5 Jun 2024 12:22:02 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1717590122; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=v2Zmw2iyq293EQsrHHixKguDt/1y+DyzRTNpZJI8E20=; b=x25uOXlxFHgOrxFduv6fAA+mYqV8ppjsqmjhbMn/09Kfb1d7attzomeZDBUmEQDje4dm2u fPAAavgHROG6h6EzCtoWEzI7z6R5EPdsGaQzXMU5Xn+ln0dsdhVSyIwOfHcIyI8Qlr7zom +cq4x9iIluk0enRT4weTV+Okk9FrpdcYE9DhhqUC9D2RWnPypsQP8fA+hRr4HykM5WfmxW Jq17gTzsv/+Sn6JVizjCQR6tLLMdZEpkkOPxeyOXTlo1YEXDVkA1VsewCLEtisOxkU4JjP jR7xo5NqLhl8mINeoJqsuvwQmh2IQBvCeo0ZJwy95mngRw29MQyt5gj4c4QcRQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1717590122; a=rsa-sha256; cv=none; b=fRShkYbyI2BxozizDUT3l7sv7DSInXBmA8qvThvzhpzPvOCJIbpMkXJAvqD+9mETFO80sE dSmXMuXgOPTfc6/Kp3ku5hL32lMgd0CQZY1puwxEi18z0tlu+NDGE3N+zXV7JLfHfSR3H6 CNo+2kBODJInXjDKUTV7V02WOwBorE828qa9348GuKq8htwaFd/0J+6GwjvNo+Gm2OfgLr ozp+T6aRyubnX5D4ABIWQcnSRrUr8h/Pb4aFFsPy9NQcnyX1tAfKKOpA6Tn0rs1dynmsEC 2ZOPkAOdLAhNnC2ayaszLSASWDABRznEHp5k/j6pJmdKveOFBewJlB5zKjUyKQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1717590122; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=v2Zmw2iyq293EQsrHHixKguDt/1y+DyzRTNpZJI8E20=; b=XSOOA/kSW/E/Wg0KwNd/8Qn3bFNTf2Sf2GnxxnaLnF5ZstkeUcPI2BF4RV8I9cVk+AbwgK bi7kJQRa2WnL9bc3uuQ9JRVIVC2bEUUKZzDUCYrvAVzEQDMoqmYb8kD7wxNCehomvC7zFz s9K0VA0xPoenLTghRuIAUTKEJBJKJKA8CHEN9JtZgO2svoCP6BycALJsq6aaBBzF2pNecL NenZOrhqBvtJk7szas5gXyBUKd7Y1msWCGwYjVyakTd5+wzKNKujXjlnw5Xe9bOiQvQowC ofehngl5zw9nWe7BB/p9pDNvjVLZM9TG/qXw525LLWXJLFNDRal7CdZTDWJJKg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4VvRRG3Fjkz12SJ; Wed, 5 Jun 2024 12:22:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 455CM2Sc031087; Wed, 5 Jun 2024 12:22:02 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 455CM2tL031084; Wed, 5 Jun 2024 12:22:02 GMT (envelope-from git) Date: Wed, 5 Jun 2024 12:22:02 GMT Message-Id: <202406051222.455CM2tL031084@gitrepo.freebsd.org> To: ports-committers@FreeBSD.org, dev-commits-ports-all@FreeBSD.org, dev-commits-ports-main@FreeBSD.org From: Hajimu UMEMOTO Subject: git: f770ff0c0dcc - main - mail/cyrus-imapd36: update to 3.6.5 List-Id: Commit messages for all branches of the ports repository List-Archive: https://lists.freebsd.org/archives/dev-commits-ports-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-ports-all@freebsd.org Sender: owner-dev-commits-ports-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: ume X-Git-Repository: ports X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: f770ff0c0dcc788d58a44176ff990068d96523aa Auto-Submitted: auto-generated The branch main has been updated by ume: URL: https://cgit.FreeBSD.org/ports/commit/?id=f770ff0c0dcc788d58a44176ff990068d96523aa commit f770ff0c0dcc788d58a44176ff990068d96523aa Author: Hajimu UMEMOTO AuthorDate: 2024-06-05 12:16:12 +0000 Commit: Hajimu UMEMOTO CommitDate: 2024-06-05 12:21:54 +0000 mail/cyrus-imapd36: update to 3.6.5 Relnotes: https://www.cyrusimap.org/3.6/imap/download/release-notes/3.6/x/3.6.5.html Security: CVE-2024-34055 --- mail/cyrus-imapd36/Makefile | 6 +- mail/cyrus-imapd36/distinfo | 6 +- mail/cyrus-imapd36/files/v36-CVE-2024-34055.patch | 5348 --------------------- 3 files changed, 5 insertions(+), 5355 deletions(-) diff --git a/mail/cyrus-imapd36/Makefile b/mail/cyrus-imapd36/Makefile index 8fed513b9d5a..5dca3baa72bc 100644 --- a/mail/cyrus-imapd36/Makefile +++ b/mail/cyrus-imapd36/Makefile @@ -1,6 +1,6 @@ PORTNAME= cyrus-imapd -PORTVERSION= 3.6.4 -PORTREVISION= 1 +PORTVERSION= 3.6.5 +PORTREVISION= 0 CATEGORIES= mail MASTER_SITES= https://github.com/cyrusimap/cyrus-imapd/releases/download/${PORTNAME}-${DISTVERSION}/ PKGNAMESUFFIX= ${CYRUS_IMAPD_VER} @@ -20,8 +20,6 @@ http_PKGNAMESUFFIX= ${CYRUS_IMAPD_VER}-http CYRUS_IMAPD_VER= 36 -EXTRA_PATCHES= ${FILESDIR}/v36-CVE-2024-34055.patch:-p1 - LIB_DEPENDS= libsasl2.so:security/cyrus-sasl2 \ libicuuc.so:devel/icu \ libjansson.so:devel/jansson \ diff --git a/mail/cyrus-imapd36/distinfo b/mail/cyrus-imapd36/distinfo index 4028c0592aef..d44a6e6a8767 100644 --- a/mail/cyrus-imapd36/distinfo +++ b/mail/cyrus-imapd36/distinfo @@ -1,3 +1,3 @@ -TIMESTAMP = 1710505927 -SHA256 (cyrus-imapd-3.6.4.tar.gz) = 4fa6ee42a08d842b9aab812fdace1bb4966e63f1ad9718345338e5896eb723e2 -SIZE (cyrus-imapd-3.6.4.tar.gz) = 13866715 +TIMESTAMP = 1717583649 +SHA256 (cyrus-imapd-3.6.5.tar.gz) = c807d6c566a8ff0986a5162eaa3077d57bb78e313d826e27140b5def3f7ac0b1 +SIZE (cyrus-imapd-3.6.5.tar.gz) = 13897004 diff --git a/mail/cyrus-imapd36/files/v36-CVE-2024-34055.patch b/mail/cyrus-imapd36/files/v36-CVE-2024-34055.patch deleted file mode 100644 index 8761b618599e..000000000000 --- a/mail/cyrus-imapd36/files/v36-CVE-2024-34055.patch +++ /dev/null @@ -1,5348 +0,0 @@ -From d95b0b211e8179b62fb3a814c827db5175968d01 Mon Sep 17 00:00:00 2001 -From: Robert Stepanek -Date: Wed, 3 Jan 2024 09:51:36 +0100 -Subject: [PATCH 01/16] SearchFuzzy.pm: do not use non-standard XSNIPPETS - command - -The XSNIPPETS and XCONVMULTISTANDARD commands in Cyrus got -deprecated, so don't keep our test using it. - -Signed-off-by: Robert Stepanek ---- - cassandane/Cassandane/Cyrus/SearchFuzzy.pm | 344 +++++++++------------ - 1 file changed, 146 insertions(+), 198 deletions(-) - -diff --git a/cassandane/Cassandane/Cyrus/SearchFuzzy.pm b/cassandane/Cassandane/Cyrus/SearchFuzzy.pm -index af0adaf64..5b3aad0e0 100644 ---- a/cassandane/Cassandane/Cyrus/SearchFuzzy.pm -+++ b/cassandane/Cassandane/Cyrus/SearchFuzzy.pm -@@ -43,6 +43,8 @@ use warnings; - use Cwd qw(abs_path); - use DateTime; - use Data::Dumper; -+use MIME::Base64 qw(encode_base64); -+use Encode qw(decode encode); - - use lib '.'; - use base qw(Cassandane::Cyrus::TestCase); -@@ -50,10 +52,19 @@ use Cassandane::Util::Log; - - sub new - { -+ - my ($class, @args) = @_; - my $config = Cassandane::Config->default()->clone(); -- $config->set(conversations => 'on'); -- return $class->SUPER::new({ config => $config }, @args); -+ $config->set( -+ conversations => 'on', -+ httpallowcompress => 'no', -+ httpmodules => 'jmap', -+ ); -+ return $class->SUPER::new({ -+ config => $config, -+ jmap => 1, -+ services => [ 'imap', 'http' ] -+ }, @args); - } - - sub set_up -@@ -134,6 +145,55 @@ sub create_testmessages - $self->{instance}->run_command({cyrus => 1}, 'squatter'); - } - -+sub get_snippets -+{ -+ # Previous versions of this test module used XSNIPPETS to -+ # assert snippets but this command got removed from Cyrus. -+ # Use JMAP instead. -+ -+ my ($self, $folder, $uids, $filter) = @_; -+ -+ my $imap = $self->{store}->get_client(); -+ my $jmap = $self->{jmap}; -+ -+ $self->assert_not_null($jmap); -+ -+ $imap->select($folder); -+ my $res = $imap->fetch($uids, ['emailid']); -+ my %emailIdToImapUid = map { $res->{$_}{emailid}[0] => $_ } keys %$res; -+ -+ $res = $jmap->CallMethods([ -+ ['SearchSnippet/get', { -+ filter => $filter, -+ emailIds => [ keys %emailIdToImapUid ], -+ }, 'R1'], -+ ]); -+ -+ my @snippets; -+ foreach (@{$res->[0][1]{list}}) { -+ if ($_->{subject}) { -+ push(@snippets, [ -+ 0, -+ $emailIdToImapUid{$_->{emailId}}, -+ 'SUBJECT', -+ $_->{subject}, -+ ]); -+ } -+ if ($_->{preview}) { -+ push(@snippets, [ -+ 0, -+ $emailIdToImapUid{$_->{emailId}}, -+ 'BODY', -+ $_->{preview}, -+ ]); -+ } -+ } -+ -+ return { -+ snippets => [ sort { $a->[1] <=> $b->[1] } @snippets ], -+ }; -+} -+ - sub test_copy_messages - :needs_search_xapian - { -@@ -151,12 +211,13 @@ sub test_copy_messages - } - - sub test_stem_verbs -- :min_version_3_0 :needs_search_xapian -+ :min_version_3_0 :needs_search_xapian :JMAPExtensions - { - my ($self) = @_; - $self->create_testmessages(); - - my $talk = $self->{store}->get_client(); -+ $self->assert_not_null($self->{jmap}); - - xlog $self, "Select INBOX"; - my $r = $talk->select("INBOX") || die; -@@ -175,11 +236,8 @@ sub test_stem_verbs - $r = $talk->search('fuzzy', ['subject', { Quote => "runs" }]) || die; - $self->assert_num_equals(3, scalar @$r); - -- xlog $self, 'XSNIPPETS for FUZZY subject "runs"'; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'subject', { Quote => 'runs' }] -- ) || die; -+ xlog $self, 'Get snippets for FUZZY subject "runs"'; -+ $r = $self->get_snippets('INBOX', $uids, { subject => 'runs' }); - $self->assert_num_equals(3, scalar @{$r->{snippets}}); - } - -@@ -250,12 +308,8 @@ sub test_snippet_wildcard - $talk->select("INBOX") || die; - my $uidvalidity = $talk->get_response_code('uidvalidity'); - -- xlog $self, "XSNIPPETS for $term"; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => "$term*" }] -- ) || die; -- xlog $self, Dumper($r); -+ xlog $self, "Get snippets for $term"; -+ $r = $self->get_snippets('INBOX', $uids, { 'text' => "$term*" }); - $self->assert_num_equals(2, scalar @{$r->{snippets}}); - } - -@@ -358,13 +412,17 @@ sub test_normalize_snippets - my ($self) = @_; - - # Set up test message with funny characters -- my $body = "foo gären советской diĝir naïve léger"; -- my @terms = split / /, $body; -+use utf8; -+ my @terms = ( "gären", "советской", "diĝir", "naïve", "léger" ); -+no utf8; -+ my $body = encode_base64(encode('UTF-8', join(' ', @terms))); -+ $body =~ s/\r?\n/\r\n/gs; - - xlog $self, "Generate and index test messages."; - my %params = ( - mime_charset => "utf-8", -- body => $body -+ mime_encoding => 'base64', -+ body => $body, - ); - $self->make_message("1", %params) || die; - -@@ -380,24 +438,20 @@ sub test_normalize_snippets - - # Assert that diacritics are matched and returned - foreach my $term (@terms) { -- xlog $self, "XSNIPPETS for FUZZY text \"$term\""; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => $term }] -- ) || die; -- $self->assert_num_not_equals(index($r->{snippets}[0][3], "$term"), -1); -+ $r = $self->get_snippets('INBOX', $uids, { text => $term }); -+ $self->assert_num_not_equals(index($r->{snippets}[0][3], "$term"), -1); - } - - # Assert that search without diacritics matches - if ($self->{skipdiacrit}) { - my $term = "naive"; -- xlog $self, "XSNIPPETS for FUZZY text \"$term\""; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => $term }] -- ) || die; -- $self->assert_num_not_equals(index($r->{snippets}[0][3], "naïve"), -1); -+ xlog $self, "Get snippets for FUZZY text \"$term\""; -+ $r = $self->get_snippets('INBOX', $uids, { 'text' => $term }); -+use utf8; -+ $self->assert_num_not_equals(index($r->{snippets}[0][3], "naïve"), -1); -+no utf8; - } -+ - } - - sub test_skipdiacrit -@@ -499,38 +553,23 @@ sub test_snippets_termcover - my $r = $talk->select("INBOX") || die; - my $uidvalidity = $talk->get_response_code('uidvalidity'); - my $uids = $talk->search('1:*', 'NOT', 'DELETED'); -- my $want = "favourite cereal"; -+ my $want = "favourite cereal"; - -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ -- 'fuzzy', 'text', 'favourite', -- 'fuzzy', 'text', 'cereal', -- 'fuzzy', 'text', { Quote => 'bogus gnarly' } -- ] -- ) || die; -+ $r = $self->get_snippets('INBOX', $uids, { -+ operator => 'AND', -+ conditions => [{ -+ text => 'favourite', -+ }, { -+ text => 'cereal', -+ }, { -+ text => '"bogus gnarly"' -+ }], -+ }); - $self->assert_num_not_equals(-1, index($r->{snippets}[0][3], $want)); - -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ -- 'fuzzy', 'text', 'favourite cereal' -- ] -- ) || die; -- $self->assert_num_not_equals(-1, index($r->{snippets}[0][3], $want)); -- -- # Regression - a phrase is treated as a loose term -- $r = $talk->xsnippets( [ [ 'INBOX', $uidvalidity, $uids ] ], -- 'utf-8', [ -- 'fuzzy', 'text', { Quote => 'favourite nope cereal' }, -- 'fuzzy', 'text', { Quote => 'bogus gnarly' } -- ] -- ) || die; -- $self->assert_num_not_equals(-1, index($r->{snippets}[0][3], $want)); -- -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ -- 'fuzzy', 'text', { Quote => 'favourite cereal' } -- ] -- ) || die; -+ $r = $self->get_snippets('INBOX', $uids, { -+ text => 'favourite cereal', -+ }); - $self->assert_num_not_equals(-1, index($r->{snippets}[0][3], $want)); - } - -@@ -542,18 +581,28 @@ sub test_cjk_words - - xlog $self, "Generate and index test messages."; - -+use utf8; - my $body = "明末時已經有香港地方的概念"; -+no utf8; -+ $body = encode_base64(encode('UTF-8', $body)); -+ $body =~ s/\r?\n/\r\n/gs; - my %params = ( - mime_charset => "utf-8", -- body => $body -+ mime_encoding => 'base64', -+ body => $body, - ); - $self->make_message("1", %params) || die; - - # Splits into the words: "み, 円, 月額, 申込 -+use utf8; - $body = "申込み！月額円"; -+no utf8; -+ $body = encode_base64(encode('UTF-8', $body)); -+ $body =~ s/\r?\n/\r\n/gs; - %params = ( - mime_charset => "utf-8", -- body => $body -+ mime_encoding => 'base64', -+ body => $body, - ); - $self->make_message("2", %params) || die; - -@@ -569,50 +618,45 @@ sub test_cjk_words - - my $term; - # Search for a two-character CJK word -+use utf8; - $term = "已經"; -- xlog $self, "XSNIPPETS for FUZZY text \"$term\""; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => $term }] -- ) || die; -- $self->assert_num_not_equals(index($r->{snippets}[0][3], "$term"), -1); -+no utf8; -+ xlog $self, "Get snippets for FUZZY text \"$term\""; -+ $r = $self->get_snippets('INBOX', $uids, { text => $term }); -+ $self->assert_num_not_equals(index($r->{snippets}[0][3], "$term"), -1); - - # Search for the CJK words 明末 and 時, note that the - # word order is reversed to the original message -+use utf8; - $term = "時明末"; -- xlog $self, "XSNIPPETS for FUZZY text \"$term\""; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => $term }] -- ) || die; -+no utf8; -+ xlog $self, "Get snippets for FUZZY text \"$term\""; -+ $r = $self->get_snippets('INBOX', $uids, { text => $term }); - $self->assert_num_equals(scalar @{$r->{snippets}}, 1); - - # Search for the partial CJK word 月 -+use utf8; - $term = "月"; -- xlog $self, "XSNIPPETS for FUZZY text \"$term\""; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => $term }] -- ) || die; -+no utf8; -+ xlog $self, "Get snippets for FUZZY text \"$term\""; -+ $r = $self->get_snippets('INBOX', $uids, { text => $term }); - $self->assert_num_equals(scalar @{$r->{snippets}}, 0); - - # Search for the interleaved, partial CJK word 額申 -+use utf8; - $term = "額申"; -- xlog $self, "XSNIPPETS for FUZZY text \"$term\""; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => $term }] -- ) || die; -+no utf8; -+ xlog $self, "Get snippets for FUZZY text \"$term\""; -+ $r = $self->get_snippets('INBOX', $uids, { text => $term }); - $self->assert_num_equals(scalar @{$r->{snippets}}, 0); - - # Search for three of four words: "み, 月額, 申込", - # in different order than the original. -+use utf8; - $term = "月額み申込"; -- xlog $self, "XSNIPPETS for FUZZY text \"$term\""; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => $term }] -- ) || die; -+no utf8; -+ xlog $self, "Get snippets for FUZZY text \"$term\""; -+ $r = $self->get_snippets('INBOX', $uids, { text => $term }); - $self->assert_num_equals(scalar @{$r->{snippets}}, 1); - } - -@@ -805,86 +849,6 @@ sub test_xattachmentname - } - - --sub test_xapianv2 -- :min_version_3_0 :needs_search_xapian --{ -- my ($self) = @_; -- -- my $talk = $self->{store}->get_client(); -- -- # This is a smallish regression test to check if we break something -- # obvious by moving Xapian indexing from folder:uid to message guids. -- # -- # Apart from the tests in this module, at least also the following -- # imodules are relevant: Metadata for SORT, Thread for THREAD. -- -- xlog $self, "Generate message"; -- my $r = $self->make_message("I run", body => "Run, Forrest! Run!" ) || die; -- my $uid = $r->{attrs}->{uid}; -- -- xlog $self, "Copy message into INBOX"; -- $talk->copy($uid, "INBOX"); -- -- xlog $self, "Run squatter"; -- $self->{instance}->run_command({cyrus => 1}, 'squatter'); -- -- $r = $talk->xconvmultisort( -- [ qw(reverse arrival) ], -- [ 'conversations', position => [1,10] ], -- 'utf-8', 'fuzzy', 'text', "run", -- ); -- $self->assert_num_equals(2, scalar @{$r->{sort}[0]} - 1); -- $self->assert_num_equals(1, scalar @{$r->{sort}}); -- -- xlog $self, "Create target mailbox"; -- $talk->create("INBOX.target"); -- -- xlog $self, "Copy message into INBOX.target"; -- $talk->copy($uid, "INBOX.target"); -- -- xlog $self, "Run squatter"; -- $self->{instance}->run_command({cyrus => 1}, 'squatter'); -- -- $r = $talk->xconvmultisort( -- [ qw(reverse arrival) ], -- [ 'conversations', position => [1,10] ], -- 'utf-8', 'fuzzy', 'text', "run", -- ); -- $self->assert_num_equals(3, scalar @{$r->{sort}[0]} - 1); -- $self->assert_num_equals(1, scalar @{$r->{sort}}); -- -- xlog $self, "Generate message"; -- $self->make_message("You run", body => "A running joke" ) || die; -- -- xlog $self, "Run squatter"; -- $self->{instance}->run_command({cyrus => 1}, 'squatter'); -- -- $r = $talk->xconvmultisort( -- [ qw(reverse arrival) ], -- [ 'conversations', position => [1,10] ], -- 'utf-8', 'fuzzy', 'text', "run", -- ); -- $self->assert_num_equals(2, scalar @{$r->{sort}}); -- -- xlog $self, "SEARCH FUZZY"; -- $r = $talk->search( -- "charset", "utf-8", "fuzzy", "text", "run", -- ) || die; -- $self->assert_num_equals(3, scalar @$r); -- -- xlog $self, "Select INBOX"; -- $r = $talk->select("INBOX") || die; -- my $uidvalidity = $talk->get_response_code('uidvalidity'); -- my $uids = $talk->search('1:*', 'NOT', 'DELETED'); -- -- xlog $self, "XSNIPPETS"; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'body', 'run'], -- ) || die; -- $self->assert_num_equals(3, scalar @{$r->{snippets}}); --} -- - sub test_snippets_escapehtml - :min_version_3_0 :needs_search_xapian - { -@@ -914,21 +878,15 @@ sub test_snippets_escapehtml - my $uids = $talk->search('1:*', 'NOT', 'DELETED'); - my %m; - -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ 'fuzzy', 'text', 'test1' ] -- ) || die; -- -+ $r = $self->get_snippets('INBOX', $uids, { 'text' => 'test1' }); - %m = map { lc($_->[2]) => $_->[3] } @{ $r->{snippets} }; -- $self->assert_str_equals("Test1 body with the same tag as snippets", $m{body}); -- $self->assert_str_equals("Test1 subject with an unescaped & in it", $m{subject}); -- -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ 'fuzzy', 'text', 'test2' ] -- ) || die; -+ $self->assert_str_equals("Test1 body with the same tag as snippets", $m{body}); -+ $self->assert_str_equals("Test1 subject with an unescaped & in it", $m{subject}); - -+ $r = $self->get_snippets('INBOX', $uids, { 'text' => 'test2' }); - %m = map { lc($_->[2]) => $_->[3] } @{ $r->{snippets} }; -- $self->assert_str_equals("Test2 body with a <tag/>, although it's plain text", $m{body}); -- $self->assert_str_equals("Test2 subject with a <tag> in it", $m{subject}); -+ $self->assert_str_equals("Test2 body with a <tag/>, although it's plain text", $m{body}); -+ $self->assert_str_equals("Test2 subject with a <tag> in it", $m{subject}); - } - - sub test_search_exactmatch -@@ -963,13 +921,10 @@ sub test_search_exactmatch - $self->assert_num_equals(1, scalar @$uids); - - my %m; -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ 'fuzzy', 'body', $query ] -- ) || die; -- -+ $r = $self->get_snippets('INBOX', $uids, { body => $query }); - %m = map { lc($_->[2]) => $_->[3] } @{ $r->{snippets} }; -- $self->assert(index($m{body}, "some text") != -1); -- $self->assert(index($m{body}, "some long text") == -1); -+ $self->assert(index($m{body}, "some text") != -1); -+ $self->assert(index($m{body}, "some long text") == -1); - } - - sub test_search_subjectsnippet -@@ -1004,10 +959,7 @@ sub test_search_subjectsnippet - $self->assert_num_equals(1, scalar @$uids); - - my %m; -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ 'fuzzy', 'text', $query ] -- ) || die; -- -+ $r = $self->get_snippets('INBOX', $uids, { text => $query }); - %m = map { lc($_->[2]) => $_->[3] } @{ $r->{snippets} }; - $self->assert_matches(qr/^\[plumbing\]/, $m{subject}); - } -@@ -1317,11 +1269,10 @@ sub test_detect_language - $self->assert_deep_equals([1], $uids); - - my $r = $talk->select("INBOX") || die; -- my $uidvalidity = $talk->get_response_code('uidvalidity'); -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ 'fuzzy', 'body', 'atmet' ] -- ) || die; -- $self->assert_num_not_equals(-1, index($r->{snippets}[0][3], ' Höhe atmeten.')); -+ $r = $self->get_snippets('INBOX', $uids, { body => 'atmet' }); -+use utf8; -+ $self->assert_num_not_equals(-1, index($r->{snippets}[0][3], ' Höhe atmeten.')); -+no utf8; - } - - sub test_detect_language_subject -@@ -1377,12 +1328,9 @@ sub test_detect_language_subject - $self->assert_deep_equals([1], $uids); - - my $r = $talk->select("INBOX") || die; -- my $uidvalidity = $talk->get_response_code('uidvalidity'); -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ 'fuzzy', 'subject', 'Landschaft' ] -- ) || die; -+ $r = $self->get_snippets('INBOX', $uids, { subject => 'Landschaft' }); - $self->assert_str_equals( -- 'A subject with the German word Landschaften', -+ 'A subject with the German word Landschaften', - $r->{snippets}[0][3] - ); - } --- -2.39.2 - - -From cce755f3a49f3768058c4f52b2c32763e890a6b8 Mon Sep 17 00:00:00 2001 -From: Ken Murchison -Date: Wed, 7 Feb 2024 14:00:00 -0500 -Subject: [PATCH 02/16] imapd.c: UIDVALIDITY should be uint32_t and parse it as - such - ---- - imap/imapd.c | 10 +++------- - imap/index.h | 2 +- - 2 files changed, 4 insertions(+), 8 deletions(-) - -diff --git a/imap/imapd.c b/imap/imapd.c -index 6f70820ca..8e087b731 100644 ---- a/imap/imapd.c -+++ b/imap/imapd.c -@@ -4279,15 +4279,11 @@ static void cmd_select(char *tag, char *cmd, char *name) - } - else if ((client_capa & CAPA_QRESYNC) && - !strcmp(arg.s, "QRESYNC")) { -- char *p; -- - if (c != ' ') goto badqresync; - c = prot_getc(imapd_in); - if (c != '(') goto badqresync; -- c = getastring(imapd_in, imapd_out, &arg); -- v->uidvalidity = strtoul(arg.s, &p, 10); -- if (*p || !v->uidvalidity || v->uidvalidity == ULONG_MAX) goto badqresync; -- if (c != ' ') goto badqresync; -+ c = getuint32(imapd_in, &v->uidvalidity); -+ if (c != ' ' || !v->uidvalidity) goto badqresync; - c = getmodseq(imapd_in, &v->modseq); - if (c == EOF) goto badqresync; - if (c == ' ') { -@@ -4427,7 +4423,7 @@ static void cmd_select(char *tag, char *cmd, char *name) - prot_printf(backend_current->out, "%s %s {" SIZE_T_FMT "+}\r\n%s", - tag, cmd, strlen(name), name); - if (v->uidvalidity) { -- prot_printf(backend_current->out, " (QRESYNC (%lu " MODSEQ_FMT, -+ prot_printf(backend_current->out, " (QRESYNC (%u " MODSEQ_FMT, - v->uidvalidity, v->modseq); - if (v->sequence) { - prot_printf(backend_current->out, " %s", v->sequence); -diff --git a/imap/index.h b/imap/index.h -index ce8fe36c2..df12e609a 100644 ---- a/imap/index.h -+++ b/imap/index.h -@@ -72,7 +72,7 @@ extern unsigned client_capa; - struct message; - - struct vanished_params { -- unsigned long uidvalidity; -+ uint32_t uidvalidity; - modseq_t modseq; - const char *match_seq; - const char *match_uid; --- -2.39.2 - - -From f37421f2687b811c4e4bd8c0fc5c66d368cc3ffe Mon Sep 17 00:00:00 2001 -From: Ken Murchison -Date: Fri, 9 Feb 2024 08:13:05 -0500 -Subject: [PATCH 03/16] imapd.c: consolidate ID field-value parse error - response - ---- - imap/imapd.c | 17 +++++------------ - 1 file changed, 5 insertions(+), 12 deletions(-) - -diff --git a/imap/imapd.c b/imap/imapd.c -index 8e087b731..8773621e0 100644 ---- a/imap/imapd.c -+++ b/imap/imapd.c -@@ -3104,19 +3104,12 @@ static void cmd_id(char *tag) - - /* get field name */ - c = getstring(imapd_in, imapd_out, &field); -- if (c != ' ') { -+ if (c != ' ' || -+ /* get field value */ -+ (c = getnstring(imapd_in, imapd_out, &arg)) == EOF || -+ (c != ' ' && c != ')')) { - prot_printf(imapd_out, -- "%s BAD Invalid/missing field name in Id\r\n", -- tag); -- eatline(imapd_in, c); -- return; -- } -- -- /* get field value */ -- c = getnstring(imapd_in, imapd_out, &arg); -- if (c != ' ' && c != ')') { -- prot_printf(imapd_out, -- "%s BAD Invalid/missing value in Id\r\n", -+ "%s BAD Invalid field-value pair in Id\r\n", - tag); - eatline(imapd_in, c); - return; --- -2.39.2 - - -From 0ed0466636f43ea32cb765c33fa6a8109d9ea69e Mon Sep 17 00:00:00 2001 -From: Ken Murchison -Date: Wed, 7 Feb 2024 14:12:41 -0500 -Subject: [PATCH 04/16] imapd.c: response code in fatal() string MUST - immediately follow "* BYE" - ---- - imap/imapd.c | 3 ++- - 1 file changed, 2 insertions(+), 1 deletion(-) - -diff --git a/imap/imapd.c b/imap/imapd.c -index 8773621e0..464d9d7ad 100644 ---- a/imap/imapd.c -+++ b/imap/imapd.c -@@ -1186,7 +1186,8 @@ EXPORTED void fatal(const char *s, int code) - } - recurse_code = code; - if (imapd_out) { -- prot_printf(imapd_out, "* BYE Fatal error: %s\r\n", s); -+ prot_printf(imapd_out, "* BYE %s%s\r\n", -+ *s == '[' /* resp-text-code */ ? "" : "Fatal error: ", s); - prot_flush(imapd_out); - } - if (stages.count) { --- -2.39.2 - - -From fb0eee5f3c4d67c0ee84d8e33bd835d9f0044e4a Mon Sep 17 00:00:00 2001 -From: Ken Murchison -Date: Fri, 23 Feb 2024 11:00:19 -0500 -Subject: [PATCH 05/16] imapparse.c: include [TOOBIG] response code for - oversized word/qstring - ---- - imap/imapparse.c | 7 +++++-- - 1 file changed, 5 insertions(+), 2 deletions(-) - -diff --git a/imap/imapparse.c b/imap/imapparse.c -index 02cc1ed92..d096584e3 100644 ---- a/imap/imapparse.c -+++ b/imap/imapparse.c -@@ -74,7 +74,7 @@ EXPORTED int getword(struct protstream *in, struct buf *buf) - } - buf_putc(buf, c); - if (config_maxword && buf_len(buf) > config_maxword) { -- fatal("word too long", EX_IOERR); -+ fatal("[TOOBIG] Word too long", EX_IOERR); - } - } - } -@@ -138,7 +138,7 @@ EXPORTED int getxstring(struct protstream *pin, struct protstream *pout, - } - buf_putc(buf, c); - if (config_maxquoted && buf_len(buf) > config_maxquoted) { -- fatal("quoted value too long", EX_IOERR); -+ fatal("[TOOBIG] Quoted value too long", EX_IOERR); - } - } - -@@ -212,6 +212,9 @@ EXPORTED int getxstring(struct protstream *pin, struct protstream *pout, - return c; - } - buf_putc(buf, c); -+ if (config_maxword && buf_len(buf) > config_maxword) { -+ fatal("[TOOBIG] Word too long", EX_IOERR); -+ } - c = prot_getc(pin); - } - /* never gets here */ --- -2.39.2 - - -From dc9846028db0022372b4cfe4d5ea92ab22eb2ed0 Mon Sep 17 00:00:00 2001 -From: Ken Murchison -Date: Fri, 9 Feb 2024 13:31:11 -0500 -Subject: [PATCH 06/16] imapparse.c: fatal() when a client violates LITERAL- - limit - ---- - imap/imap_err.et | 3 +++ - imap/imapparse.c | 7 +++++-- - 2 files changed, 8 insertions(+), 2 deletions(-) - -diff --git a/imap/imap_err.et b/imap/imap_err.et -index a98ec0e1b..072078f94 100644 ---- a/imap/imap_err.et -+++ b/imap/imap_err.et -@@ -65,6 +65,9 @@ ec IMAP_QUOTA_EXCEEDED, - ec IMAP_MESSAGE_TOO_LARGE, - "Message size exceeds fixed limit" - -+ec IMAP_LITERAL_MINUS_TOO_LARGE, -+ "[TOOBIG] Non-synchronizing literal size exceeds 4K" -+ - ec IMAP_USERFLAG_EXHAUSTED, - "Too many user flags in mailbox" - -diff --git a/imap/imapparse.c b/imap/imapparse.c -index d096584e3..7a4794600 100644 ---- a/imap/imapparse.c -+++ b/imap/imapparse.c -@@ -153,8 +153,11 @@ EXPORTED int getxstring(struct protstream *pin, struct protstream *pout, - buf_reset(buf); - c = getint32(pin, &len); - if (c == '+') { -- // LITERAL- says maximum size is 4096! -- if (lminus && len > 4096) return EOF; -+ /* LITERAL- says maximum size is 4096! */ -+ if (lminus && len > 4096) { -+ /* Fail per RFC 7888, Section 4, choice 2 */ -+ fatal(error_message(IMAP_LITERAL_MINUS_TOO_LARGE), EX_IOERR); -+ } - isnowait++; - c = prot_getc(pin); - } --- -2.39.2 - - -From 8414e71a9d1fbc56d03d6fddb096484364e3406c Mon Sep 17 00:00:00 2001 -From: Ken Murchison -Date: Mon, 12 Feb 2024 10:54:03 -0500 -Subject: [PATCH 07/16] Cleanup and document the use of prot_setisclient() - -Only IMAP(-like) clients send LITERAL+ syntax ---- - backup/backupd.c | 3 +-- - backup/lcb.c | 1 - - backup/lcb_read.c | 2 -- - backup/lcb_verify.c | 2 -- - cunit/getxstring.testc | 4 ---- - imap/append.c | 1 - - imap/backend.c | 2 -- - imap/cyr_dbtool.c | 1 - - imap/dlist.c | 5 ++++- - imap/imapd.c | 4 ++++ - imap/imapparse.c | 5 +++-- - imap/message.c | 3 --- - imap/mupdate.c | 3 +++ - imap/sync_server.c | 3 +-- - imap/sync_support.c | 4 ---- - lib/prot.h | 2 +- - 16 files changed, 17 insertions(+), 28 deletions(-) - -diff --git a/backup/backupd.c b/backup/backupd.c -index 9f8fa7b15..d42ddbb70 100644 ---- a/backup/backupd.c -+++ b/backup/backupd.c -@@ -229,9 +229,8 @@ EXPORTED int service_main(int argc __attribute__((unused)), - backupd_in = prot_new(0, 0); - backupd_out = prot_new(1, 1); - -- /* Force use of LITERAL+ so we don't need two way communications */ -+ /* Allow use of LITERAL+ */ - prot_setisclient(backupd_in, 1); -- prot_setisclient(backupd_out, 1); - - /* Find out name of client host */ - backupd_clienthost = get_clienthost(0, &localip, &remoteip); -diff --git a/backup/lcb.c b/backup/lcb.c -index dbba85ca7..3f68b1aaa 100644 ---- a/backup/lcb.c -+++ b/backup/lcb.c -@@ -606,7 +606,6 @@ EXPORTED int backup_reindex(const char *name, - fprintf(out, "\nfound chunk at offset " OFF_T_FMT "\n\n", member_offset); - - struct protstream *member = prot_readcb(_prot_fill_cb, gzuc); -- prot_setisclient(member, 1); /* don't sync literals */ - - // FIXME stricter timestamp sequence checks - time_t member_start_ts = -1; -diff --git a/backup/lcb_read.c b/backup/lcb_read.c -index f597c97c5..f2342d69d 100644 ---- a/backup/lcb_read.c -+++ b/backup/lcb_read.c -@@ -113,7 +113,6 @@ EXPORTED int backup_read_message_data(struct backup *backup, - if (r) return r; - - struct protstream *ps = prot_readcb(_prot_fill_cb, gzuc); -- prot_setisclient(ps, 1); /* don't sync literals */ - r = parse_backup_line(ps, NULL, NULL, &dl); - prot_free(ps); - -@@ -203,7 +202,6 @@ EXPORTED int backup_prepare_message_upload(struct backup *backup, - if (!r) { - struct protstream *ps = prot_readcb(_prot_fill_cb, gzuc); - int c; -- prot_setisclient(ps, 1); /* don't sync literals */ - c = parse_backup_line(ps, NULL, NULL, &dl); - prot_free(ps); - ps = NULL; -diff --git a/backup/lcb_verify.c b/backup/lcb_verify.c -index fb0477a8b..88e748c51 100644 ---- a/backup/lcb_verify.c -+++ b/backup/lcb_verify.c -@@ -228,7 +228,6 @@ static int _verify_message_cb(const struct backup_message *message, void *rock) - if (r) return r; - - struct protstream *ps = prot_readcb(_prot_fill_cb, vmrock->gzuc); -- prot_setisclient(ps, 1); /* don't sync literals */ - r = parse_backup_line(ps, NULL, NULL, &dl); - - if (r == EOF) { -@@ -528,7 +527,6 @@ static int verify_chunk_mailbox_links(struct backup *backup, struct backup_chunk - goto done; - } - struct protstream *ps = prot_readcb(_prot_fill_cb, gzuc); -- prot_setisclient(ps, 1); /* don't sync literals */ - - struct buf cmd = BUF_INITIALIZER; - while (1) { -diff --git a/cunit/getxstring.testc b/cunit/getxstring.testc -index f5a5989a3..12efe86aa 100644 ---- a/cunit/getxstring.testc -+++ b/cunit/getxstring.testc -@@ -72,9 +72,6 @@ static int tear_down(void) - - /* - * Run a single testcase. -- * -- * Note: prot_setisclient() turns off off literal synchronising so -- * we don't have to futz around with testing that. - */ - #define _TESTCASE_PRE(fut, input, retval, consumed) \ - do { \ -@@ -84,7 +81,6 @@ static int tear_down(void) - long long _consumed = (consumed); \ - p = prot_readmap(input, sizeof(input)-1); \ - CU_ASSERT_PTR_NOT_NULL_FATAL(p); \ -- prot_setisclient(p, 1); \ - c = fut(p, NULL, &b); \ - CU_ASSERT_EQUAL(c, retval); \ - if (_consumed >= 0) { \ -diff --git a/imap/append.c b/imap/append.c -index f7cf7e770..b8bc7963a 100644 ---- a/imap/append.c -+++ b/imap/append.c -@@ -436,7 +436,6 @@ static int callout_receive_reply(const char *callout, - } - - p = prot_new(fd, /*write*/0); -- prot_setisclient(p, 1); - - /* read and parse the reply as a dlist */ - c = dlist_parse(results, /*parsekeys*/0, /*isbackup*/0, p); -diff --git a/imap/backend.c b/imap/backend.c -index 36b83b4d9..546f41d25 100644 ---- a/imap/backend.c -+++ b/imap/backend.c -@@ -955,7 +955,6 @@ EXPORTED struct backend *backend_connect_pipe(int infd, int outfd, - ret->prot = prot; - - /* use literal+ to send literals */ -- prot_setisclient(ret->in, 1); - prot_setisclient(ret->out, 1); - - /* Start TLS if required */ -@@ -1153,7 +1152,6 @@ EXPORTED struct backend *backend_connect(struct backend *ret_backend, const char - ret->prot = prot; - - /* use literal+ to send literals */ -- prot_setisclient(ret->in, 1); - prot_setisclient(ret->out, 1); - - /* Start TLS if required */ -diff --git a/imap/cyr_dbtool.c b/imap/cyr_dbtool.c -index fec57cede..c6d56ec77 100644 ---- a/imap/cyr_dbtool.c -+++ b/imap/cyr_dbtool.c -@@ -155,7 +155,6 @@ static void batch_commands(struct db *db) - int r = 0; - - prot_setisclient(in, 1); -- prot_setisclient(out, 1); - - while (1) { - buf_reset(&cmd); -diff --git a/imap/dlist.c b/imap/dlist.c -index a2f876cd4..2b73ad5d0 100644 ---- a/imap/dlist.c -+++ b/imap/dlist.c -@@ -1225,7 +1225,10 @@ EXPORTED int dlist_parsemap(struct dlist **dlp, int parsekey, int isbackup, - struct dlist *dl = NULL; - - stream = prot_readmap(base, len); *** 4421 LINES SKIPPED ***