From nobody Wed Jun 5 12:22:03 2024 X-Original-To: dev-commits-ports-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VvRRH5l3zz5Mmgn; Wed, 05 Jun 2024 12:22:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VvRRH4DSrz4lvf; Wed, 5 Jun 2024 12:22:03 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1717590123; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=dO/y0IqhDOk+W0ERHAgVymLH+D4FHTeQ6MaLMix8B+c=; b=HSouiz46+0W9fbBBuRlnrl+JIhwMpEi6AEyueVyGlBSBnygSf3qRHK5BFbdmZimy8KZLkK yANARgFqlyQeGw7X5audm0fVNIm9itxmuoXXd0zeEGD43YMzTjV3FXEIAeqU0gS//XaSZ2 fGCtQCzYWaqrceMOJraOeWOOrpnnn2drMxw+wZKW9JaMI3qZFaBs2EnPGESjdJPDgYulAM lwMiydxoJPZ5GyTLJSN8rGSVcF62+VE88CQOZL+Xhna3+nLhvC6fvvd1jPIAi9KA5iQfo3 OrCpN2rVx73FcLmiFfZcK975HoQFm/W8fq33cDzkUEVdHN/6ztaLVJwqIEiiDQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1717590123; a=rsa-sha256; cv=none; b=ZoyKMFczO2i8SDY4cFJG2hcRCtsUlI6rXkAUY7l3ajcDlfHQAAk0Qx4bDAGOS+QTlyrMoU BXA/QFTt2BFq++pl9s2LAAr4YG0IJU4yrg7pl1CF+v2BFTV4+jgNL98q4J/WB8fSvMO12Q 0OazBSALqKCN/jYBPjUwFkzcj+d9jBBN6yr3AsVacHn7pS5WVlFiMvbYbzPDzuMqadNZip M5M0QvifwLrF4eBRh/0JBRAjFIp1f+9VMWoNBDFB7p0enfCMTEK1xBYyB5j/rSkvVumyb0 eOuSP0CdisjYQ0V/1WRa2BrchCW6mulAMLrU0oGdqlY+KRHMbB04CJ/Pply5WA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1717590123; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=dO/y0IqhDOk+W0ERHAgVymLH+D4FHTeQ6MaLMix8B+c=; b=GFVcp9iQybV+Z7Fhq5llYffKnllvgOR3tymcM5N1HeSgvZ70qkPLFWQDxSBUy96plVqH55 +h5dD9DbgbiQJQLtJT7trW9y3nfkGAEqveY+8vUiARi836ADzRyxA3DqweQO7NqKkv4uPa dWxWnEV0nEUmPWF+Sp8pP30hshjPs/E4fMg5JJ7i52CvXuk2H0DkF0/kxqLaZsWXklUpZE TIYHyqTF4z74J3qsUd9AxgBbxlr8241vohowqN8nlm9RJ3QSctqtTMHdQHWm4+TAgPN+Sb 452v1/YdCuamrkCd7j1YDZWoulXyERMuS4oP18U4ZdGpt+MHRrzywWc3s2fEGQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4VvRRH3r1zz12SK; Wed, 5 Jun 2024 12:22:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 455CM3KO031126; Wed, 5 Jun 2024 12:22:03 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 455CM39e031123; Wed, 5 Jun 2024 12:22:03 GMT (envelope-from git) Date: Wed, 5 Jun 2024 12:22:03 GMT Message-Id: <202406051222.455CM39e031123@gitrepo.freebsd.org> To: ports-committers@FreeBSD.org, dev-commits-ports-all@FreeBSD.org, dev-commits-ports-main@FreeBSD.org From: Hajimu UMEMOTO Subject: git: 09c8412feede - main - mail/cyrus-imapd34: update to 3.4.8 List-Id: Commit messages for all branches of the ports repository List-Archive: https://lists.freebsd.org/archives/dev-commits-ports-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-ports-all@freebsd.org Sender: owner-dev-commits-ports-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: ume X-Git-Repository: ports X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 09c8412feede7a96b9fc14df16f8dd8730c61c84 Auto-Submitted: auto-generated The branch main has been updated by ume: URL: https://cgit.FreeBSD.org/ports/commit/?id=09c8412feede7a96b9fc14df16f8dd8730c61c84 commit 09c8412feede7a96b9fc14df16f8dd8730c61c84 Author: Hajimu UMEMOTO AuthorDate: 2024-06-05 12:18:30 +0000 Commit: Hajimu UMEMOTO CommitDate: 2024-06-05 12:21:54 +0000 mail/cyrus-imapd34: update to 3.4.8 Relnotes: https://www.cyrusimap.org/3.4/imap/download/release-notes/3.4/x/3.4.8.html Security: CVE-2024-34055 --- mail/cyrus-imapd34/Makefile | 6 +- mail/cyrus-imapd34/distinfo | 6 +- mail/cyrus-imapd34/files/v34-CVE-2024-34055.patch | 5815 --------------------- 3 files changed, 5 insertions(+), 5822 deletions(-) diff --git a/mail/cyrus-imapd34/Makefile b/mail/cyrus-imapd34/Makefile index 35a8252929ee..8b82bfdc0088 100644 --- a/mail/cyrus-imapd34/Makefile +++ b/mail/cyrus-imapd34/Makefile @@ -1,6 +1,6 @@ PORTNAME= cyrus-imapd -PORTVERSION= 3.4.7 -PORTREVISION= 1 +PORTVERSION= 3.4.8 +PORTREVISION= 0 CATEGORIES= mail MASTER_SITES= https://github.com/cyrusimap/cyrus-imapd/releases/download/${PORTNAME}-${PORTVERSION}/ PKGNAMESUFFIX= ${CYRUS_IMAPD_VER} @@ -20,8 +20,6 @@ http_PKGNAMESUFFIX= ${CYRUS_IMAPD_VER}-http CYRUS_IMAPD_VER= 34 -EXTRA_PATCHES= ${FILESDIR}/v34-CVE-2024-34055.patch:-p1 - LIB_DEPENDS= libsasl2.so:security/cyrus-sasl2 \ libicuuc.so:devel/icu \ libjansson.so:devel/jansson \ diff --git a/mail/cyrus-imapd34/distinfo b/mail/cyrus-imapd34/distinfo index 986ce4ee823f..9e3efd0bb118 100644 --- a/mail/cyrus-imapd34/distinfo +++ b/mail/cyrus-imapd34/distinfo @@ -1,3 +1,3 @@ -TIMESTAMP = 1710506943 -SHA256 (cyrus-imapd-3.4.7.tar.gz) = 8be5abdc8392de9e217a6c4c0b24132d16cf9f68e42c071ec9d4b9fdca44da38 -SIZE (cyrus-imapd-3.4.7.tar.gz) = 13411396 +TIMESTAMP = 1717583784 +SHA256 (cyrus-imapd-3.4.8.tar.gz) = 791258fae0bbfe6d39101a287910ec37368f454982d473b2ff93ab3ea91bf55a +SIZE (cyrus-imapd-3.4.8.tar.gz) = 13428824 diff --git a/mail/cyrus-imapd34/files/v34-CVE-2024-34055.patch b/mail/cyrus-imapd34/files/v34-CVE-2024-34055.patch deleted file mode 100644 index c1719ea49b28..000000000000 --- a/mail/cyrus-imapd34/files/v34-CVE-2024-34055.patch +++ /dev/null @@ -1,5815 +0,0 @@ -From b6682068bf8c754a87f98ee59d2616d48ed756c7 Mon Sep 17 00:00:00 2001 -From: Robert Stepanek -Date: Wed, 3 Jan 2024 09:51:36 +0100 -Subject: [PATCH 01/22] SearchFuzzy.pm: do not use non-standard XSNIPPETS - command - -The XSNIPPETS and XCONVMULTISTANDARD commands in Cyrus got -deprecated, so don't keep our test using it. - -Signed-off-by: Robert Stepanek ---- - cassandane/Cassandane/Cyrus/SearchFuzzy.pm | 344 +++++++++------------ - 1 file changed, 146 insertions(+), 198 deletions(-) - -diff --git a/cassandane/Cassandane/Cyrus/SearchFuzzy.pm b/cassandane/Cassandane/Cyrus/SearchFuzzy.pm -index 1ac00dc49..dd1a369bd 100644 ---- a/cassandane/Cassandane/Cyrus/SearchFuzzy.pm -+++ b/cassandane/Cassandane/Cyrus/SearchFuzzy.pm -@@ -43,6 +43,8 @@ use warnings; - use Cwd qw(abs_path); - use DateTime; - use Data::Dumper; -+use MIME::Base64 qw(encode_base64); -+use Encode qw(decode encode); - - use lib '.'; - use base qw(Cassandane::Cyrus::TestCase); -@@ -50,10 +52,19 @@ use Cassandane::Util::Log; - - sub new - { -+ - my ($class, @args) = @_; - my $config = Cassandane::Config->default()->clone(); -- $config->set(conversations => 'on'); -- return $class->SUPER::new({ config => $config }, @args); -+ $config->set( -+ conversations => 'on', -+ httpallowcompress => 'no', -+ httpmodules => 'jmap', -+ ); -+ return $class->SUPER::new({ -+ config => $config, -+ jmap => 1, -+ services => [ 'imap', 'http' ] -+ }, @args); - } - - sub set_up -@@ -134,6 +145,55 @@ sub create_testmessages - $self->{instance}->run_command({cyrus => 1}, 'squatter'); - } - -+sub get_snippets -+{ -+ # Previous versions of this test module used XSNIPPETS to -+ # assert snippets but this command got removed from Cyrus. -+ # Use JMAP instead. -+ -+ my ($self, $folder, $uids, $filter) = @_; -+ -+ my $imap = $self->{store}->get_client(); -+ my $jmap = $self->{jmap}; -+ -+ $self->assert_not_null($jmap); -+ -+ $imap->select($folder); -+ my $res = $imap->fetch($uids, ['emailid']); -+ my %emailIdToImapUid = map { $res->{$_}{emailid}[0] => $_ } keys %$res; -+ -+ $res = $jmap->CallMethods([ -+ ['SearchSnippet/get', { -+ filter => $filter, -+ emailIds => [ keys %emailIdToImapUid ], -+ }, 'R1'], -+ ]); -+ -+ my @snippets; -+ foreach (@{$res->[0][1]{list}}) { -+ if ($_->{subject}) { -+ push(@snippets, [ -+ 0, -+ $emailIdToImapUid{$_->{emailId}}, -+ 'SUBJECT', -+ $_->{subject}, -+ ]); -+ } -+ if ($_->{preview}) { -+ push(@snippets, [ -+ 0, -+ $emailIdToImapUid{$_->{emailId}}, -+ 'BODY', -+ $_->{preview}, -+ ]); -+ } -+ } -+ -+ return { -+ snippets => [ sort { $a->[1] <=> $b->[1] } @snippets ], -+ }; -+} -+ - sub test_copy_messages - :needs_search_xapian - { -@@ -151,12 +211,13 @@ sub test_copy_messages - } - - sub test_stem_verbs -- :min_version_3_0 :needs_search_xapian -+ :min_version_3_0 :needs_search_xapian :JMAPExtensions - { - my ($self) = @_; - $self->create_testmessages(); - - my $talk = $self->{store}->get_client(); -+ $self->assert_not_null($self->{jmap}); - - xlog $self, "Select INBOX"; - my $r = $talk->select("INBOX") || die; -@@ -175,11 +236,8 @@ sub test_stem_verbs - $r = $talk->search('fuzzy', ['subject', { Quote => "runs" }]) || die; - $self->assert_num_equals(3, scalar @$r); - -- xlog $self, 'XSNIPPETS for FUZZY subject "runs"'; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'subject', { Quote => 'runs' }] -- ) || die; -+ xlog $self, 'Get snippets for FUZZY subject "runs"'; -+ $r = $self->get_snippets('INBOX', $uids, { subject => 'runs' }); - $self->assert_num_equals(3, scalar @{$r->{snippets}}); - } - -@@ -250,12 +308,8 @@ sub test_snippet_wildcard - $talk->select("INBOX") || die; - my $uidvalidity = $talk->get_response_code('uidvalidity'); - -- xlog $self, "XSNIPPETS for $term"; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => "$term*" }] -- ) || die; -- xlog $self, Dumper($r); -+ xlog $self, "Get snippets for $term"; -+ $r = $self->get_snippets('INBOX', $uids, { 'text' => "$term*" }); - $self->assert_num_equals(2, scalar @{$r->{snippets}}); - } - -@@ -358,13 +412,17 @@ sub test_normalize_snippets - my ($self) = @_; - - # Set up test message with funny characters -- my $body = "foo gären советской diĝir naïve léger"; -- my @terms = split / /, $body; -+use utf8; -+ my @terms = ( "gären", "советской", "diĝir", "naïve", "léger" ); -+no utf8; -+ my $body = encode_base64(encode('UTF-8', join(' ', @terms))); -+ $body =~ s/\r?\n/\r\n/gs; - - xlog $self, "Generate and index test messages."; - my %params = ( - mime_charset => "utf-8", -- body => $body -+ mime_encoding => 'base64', -+ body => $body, - ); - $self->make_message("1", %params) || die; - -@@ -380,24 +438,20 @@ sub test_normalize_snippets - - # Assert that diacritics are matched and returned - foreach my $term (@terms) { -- xlog $self, "XSNIPPETS for FUZZY text \"$term\""; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => $term }] -- ) || die; -- $self->assert_num_not_equals(index($r->{snippets}[0][3], "$term"), -1); -+ $r = $self->get_snippets('INBOX', $uids, { text => $term }); -+ $self->assert_num_not_equals(index($r->{snippets}[0][3], "$term"), -1); - } - - # Assert that search without diacritics matches - if ($self->{skipdiacrit}) { - my $term = "naive"; -- xlog $self, "XSNIPPETS for FUZZY text \"$term\""; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => $term }] -- ) || die; -- $self->assert_num_not_equals(index($r->{snippets}[0][3], "naïve"), -1); -+ xlog $self, "Get snippets for FUZZY text \"$term\""; -+ $r = $self->get_snippets('INBOX', $uids, { 'text' => $term }); -+use utf8; -+ $self->assert_num_not_equals(index($r->{snippets}[0][3], "naïve"), -1); -+no utf8; - } -+ - } - - sub test_skipdiacrit -@@ -499,38 +553,23 @@ sub test_snippets_termcover - my $r = $talk->select("INBOX") || die; - my $uidvalidity = $talk->get_response_code('uidvalidity'); - my $uids = $talk->search('1:*', 'NOT', 'DELETED'); -- my $want = "favourite cereal"; -+ my $want = "favourite cereal"; - -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ -- 'fuzzy', 'text', 'favourite', -- 'fuzzy', 'text', 'cereal', -- 'fuzzy', 'text', { Quote => 'bogus gnarly' } -- ] -- ) || die; -+ $r = $self->get_snippets('INBOX', $uids, { -+ operator => 'AND', -+ conditions => [{ -+ text => 'favourite', -+ }, { -+ text => 'cereal', -+ }, { -+ text => '"bogus gnarly"' -+ }], -+ }); - $self->assert_num_not_equals(-1, index($r->{snippets}[0][3], $want)); - -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ -- 'fuzzy', 'text', 'favourite cereal' -- ] -- ) || die; -- $self->assert_num_not_equals(-1, index($r->{snippets}[0][3], $want)); -- -- # Regression - a phrase is treated as a loose term -- $r = $talk->xsnippets( [ [ 'INBOX', $uidvalidity, $uids ] ], -- 'utf-8', [ -- 'fuzzy', 'text', { Quote => 'favourite nope cereal' }, -- 'fuzzy', 'text', { Quote => 'bogus gnarly' } -- ] -- ) || die; -- $self->assert_num_not_equals(-1, index($r->{snippets}[0][3], $want)); -- -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ -- 'fuzzy', 'text', { Quote => 'favourite cereal' } -- ] -- ) || die; -+ $r = $self->get_snippets('INBOX', $uids, { -+ text => 'favourite cereal', -+ }); - $self->assert_num_not_equals(-1, index($r->{snippets}[0][3], $want)); - } - -@@ -542,18 +581,28 @@ sub test_cjk_words - - xlog $self, "Generate and index test messages."; - -+use utf8; - my $body = "明末時已經有香港地方的概念"; -+no utf8; -+ $body = encode_base64(encode('UTF-8', $body)); -+ $body =~ s/\r?\n/\r\n/gs; - my %params = ( - mime_charset => "utf-8", -- body => $body -+ mime_encoding => 'base64', -+ body => $body, - ); - $self->make_message("1", %params) || die; - - # Splits into the words: "み, 円, 月額, 申込 -+use utf8; - $body = "申込み！月額円"; -+no utf8; -+ $body = encode_base64(encode('UTF-8', $body)); -+ $body =~ s/\r?\n/\r\n/gs; - %params = ( - mime_charset => "utf-8", -- body => $body -+ mime_encoding => 'base64', -+ body => $body, - ); - $self->make_message("2", %params) || die; - -@@ -569,50 +618,45 @@ sub test_cjk_words - - my $term; - # Search for a two-character CJK word -+use utf8; - $term = "已經"; -- xlog $self, "XSNIPPETS for FUZZY text \"$term\""; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => $term }] -- ) || die; -- $self->assert_num_not_equals(index($r->{snippets}[0][3], "$term"), -1); -+no utf8; -+ xlog $self, "Get snippets for FUZZY text \"$term\""; -+ $r = $self->get_snippets('INBOX', $uids, { text => $term }); -+ $self->assert_num_not_equals(index($r->{snippets}[0][3], "$term"), -1); - - # Search for the CJK words 明末 and 時, note that the - # word order is reversed to the original message -+use utf8; - $term = "時明末"; -- xlog $self, "XSNIPPETS for FUZZY text \"$term\""; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => $term }] -- ) || die; -+no utf8; -+ xlog $self, "Get snippets for FUZZY text \"$term\""; -+ $r = $self->get_snippets('INBOX', $uids, { text => $term }); - $self->assert_num_equals(scalar @{$r->{snippets}}, 1); - - # Search for the partial CJK word 月 -+use utf8; - $term = "月"; -- xlog $self, "XSNIPPETS for FUZZY text \"$term\""; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => $term }] -- ) || die; -+no utf8; -+ xlog $self, "Get snippets for FUZZY text \"$term\""; -+ $r = $self->get_snippets('INBOX', $uids, { text => $term }); - $self->assert_num_equals(scalar @{$r->{snippets}}, 0); - - # Search for the interleaved, partial CJK word 額申 -+use utf8; - $term = "額申"; -- xlog $self, "XSNIPPETS for FUZZY text \"$term\""; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => $term }] -- ) || die; -+no utf8; -+ xlog $self, "Get snippets for FUZZY text \"$term\""; -+ $r = $self->get_snippets('INBOX', $uids, { text => $term }); - $self->assert_num_equals(scalar @{$r->{snippets}}, 0); - - # Search for three of four words: "み, 月額, 申込", - # in different order than the original. -+use utf8; - $term = "月額み申込"; -- xlog $self, "XSNIPPETS for FUZZY text \"$term\""; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'text', { Quote => $term }] -- ) || die; -+no utf8; -+ xlog $self, "Get snippets for FUZZY text \"$term\""; -+ $r = $self->get_snippets('INBOX', $uids, { text => $term }); - $self->assert_num_equals(scalar @{$r->{snippets}}, 1); - } - -@@ -805,86 +849,6 @@ sub test_xattachmentname - } - - --sub test_xapianv2 -- :min_version_3_0 :needs_search_xapian --{ -- my ($self) = @_; -- -- my $talk = $self->{store}->get_client(); -- -- # This is a smallish regression test to check if we break something -- # obvious by moving Xapian indexing from folder:uid to message guids. -- # -- # Apart from the tests in this module, at least also the following -- # imodules are relevant: Metadata for SORT, Thread for THREAD. -- -- xlog $self, "Generate message"; -- my $r = $self->make_message("I run", body => "Run, Forrest! Run!" ) || die; -- my $uid = $r->{attrs}->{uid}; -- -- xlog $self, "Copy message into INBOX"; -- $talk->copy($uid, "INBOX"); -- -- xlog $self, "Run squatter"; -- $self->{instance}->run_command({cyrus => 1}, 'squatter'); -- -- $r = $talk->xconvmultisort( -- [ qw(reverse arrival) ], -- [ 'conversations', position => [1,10] ], -- 'utf-8', 'fuzzy', 'text', "run", -- ); -- $self->assert_num_equals(2, scalar @{$r->{sort}[0]} - 1); -- $self->assert_num_equals(1, scalar @{$r->{sort}}); -- -- xlog $self, "Create target mailbox"; -- $talk->create("INBOX.target"); -- -- xlog $self, "Copy message into INBOX.target"; -- $talk->copy($uid, "INBOX.target"); -- -- xlog $self, "Run squatter"; -- $self->{instance}->run_command({cyrus => 1}, 'squatter'); -- -- $r = $talk->xconvmultisort( -- [ qw(reverse arrival) ], -- [ 'conversations', position => [1,10] ], -- 'utf-8', 'fuzzy', 'text', "run", -- ); -- $self->assert_num_equals(3, scalar @{$r->{sort}[0]} - 1); -- $self->assert_num_equals(1, scalar @{$r->{sort}}); -- -- xlog $self, "Generate message"; -- $self->make_message("You run", body => "A running joke" ) || die; -- -- xlog $self, "Run squatter"; -- $self->{instance}->run_command({cyrus => 1}, 'squatter'); -- -- $r = $talk->xconvmultisort( -- [ qw(reverse arrival) ], -- [ 'conversations', position => [1,10] ], -- 'utf-8', 'fuzzy', 'text', "run", -- ); -- $self->assert_num_equals(2, scalar @{$r->{sort}}); -- -- xlog $self, "SEARCH FUZZY"; -- $r = $talk->search( -- "charset", "utf-8", "fuzzy", "text", "run", -- ) || die; -- $self->assert_num_equals(3, scalar @$r); -- -- xlog $self, "Select INBOX"; -- $r = $talk->select("INBOX") || die; -- my $uidvalidity = $talk->get_response_code('uidvalidity'); -- my $uids = $talk->search('1:*', 'NOT', 'DELETED'); -- -- xlog $self, "XSNIPPETS"; -- $r = $talk->xsnippets( -- [['INBOX', $uidvalidity, $uids]], 'utf-8', -- ['fuzzy', 'body', 'run'], -- ) || die; -- $self->assert_num_equals(3, scalar @{$r->{snippets}}); --} -- - sub test_snippets_escapehtml - :min_version_3_0 :needs_search_xapian - { -@@ -914,21 +878,15 @@ sub test_snippets_escapehtml - my $uids = $talk->search('1:*', 'NOT', 'DELETED'); - my %m; - -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ 'fuzzy', 'text', 'test1' ] -- ) || die; -- -+ $r = $self->get_snippets('INBOX', $uids, { 'text' => 'test1' }); - %m = map { lc($_->[2]) => $_->[3] } @{ $r->{snippets} }; -- $self->assert_str_equals("Test1 body with the same tag as snippets", $m{body}); -- $self->assert_str_equals("Test1 subject with an unescaped & in it", $m{subject}); -- -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ 'fuzzy', 'text', 'test2' ] -- ) || die; -+ $self->assert_str_equals("Test1 body with the same tag as snippets", $m{body}); -+ $self->assert_str_equals("Test1 subject with an unescaped & in it", $m{subject}); - -+ $r = $self->get_snippets('INBOX', $uids, { 'text' => 'test2' }); - %m = map { lc($_->[2]) => $_->[3] } @{ $r->{snippets} }; -- $self->assert_str_equals("Test2 body with a <tag/>, although it's plain text", $m{body}); -- $self->assert_str_equals("Test2 subject with a <tag> in it", $m{subject}); -+ $self->assert_str_equals("Test2 body with a <tag/>, although it's plain text", $m{body}); -+ $self->assert_str_equals("Test2 subject with a <tag> in it", $m{subject}); - } - - sub test_search_exactmatch -@@ -963,13 +921,10 @@ sub test_search_exactmatch - $self->assert_num_equals(1, scalar @$uids); - - my %m; -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ 'fuzzy', 'body', $query ] -- ) || die; -- -+ $r = $self->get_snippets('INBOX', $uids, { body => $query }); - %m = map { lc($_->[2]) => $_->[3] } @{ $r->{snippets} }; -- $self->assert(index($m{body}, "some text") != -1); -- $self->assert(index($m{body}, "some long text") == -1); -+ $self->assert(index($m{body}, "some text") != -1); -+ $self->assert(index($m{body}, "some long text") == -1); - } - - sub test_search_subjectsnippet -@@ -1004,10 +959,7 @@ sub test_search_subjectsnippet - $self->assert_num_equals(1, scalar @$uids); - - my %m; -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ 'fuzzy', 'text', $query ] -- ) || die; -- -+ $r = $self->get_snippets('INBOX', $uids, { text => $query }); - %m = map { lc($_->[2]) => $_->[3] } @{ $r->{snippets} }; - $self->assert_matches(qr/^\[plumbing\]/, $m{subject}); - } -@@ -1317,11 +1269,10 @@ sub test_detect_language - $self->assert_deep_equals([1], $uids); - - my $r = $talk->select("INBOX") || die; -- my $uidvalidity = $talk->get_response_code('uidvalidity'); -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ 'fuzzy', 'body', 'atmet' ] -- ) || die; -- $self->assert_num_not_equals(-1, index($r->{snippets}[0][3], ' Höhe atmeten.')); -+ $r = $self->get_snippets('INBOX', $uids, { body => 'atmet' }); -+use utf8; -+ $self->assert_num_not_equals(-1, index($r->{snippets}[0][3], ' Höhe atmeten.')); -+no utf8; - } - - sub test_detect_language_subject -@@ -1377,12 +1328,9 @@ sub test_detect_language_subject - $self->assert_deep_equals([1], $uids); - - my $r = $talk->select("INBOX") || die; -- my $uidvalidity = $talk->get_response_code('uidvalidity'); -- $r = $talk->xsnippets( [ [ 'inbox', $uidvalidity, $uids ] ], -- 'utf-8', [ 'fuzzy', 'subject', 'Landschaft' ] -- ) || die; -+ $r = $self->get_snippets('INBOX', $uids, { subject => 'Landschaft' }); - $self->assert_str_equals( -- 'A subject with the German word Landschaften', -+ 'A subject with the German word Landschaften', - $r->{snippets}[0][3] - ); - } --- -2.39.2 - - -From 00aafb0fd51aaac1badc3370a250605cff4313b0 Mon Sep 17 00:00:00 2001 -From: Bron Gondwana -Date: Fri, 20 Nov 2020 11:24:58 +1100 -Subject: [PATCH 02/22] imapd: maxsize for appends - ---- - imap/imapd.c | 4 ++++ - lib/imapoptions | 4 ++++ - 2 files changed, 8 insertions(+) - -diff --git a/imap/imapd.c b/imap/imapd.c -index a617ff80c..48055ccce 100644 ---- a/imap/imapd.c -+++ b/imap/imapd.c -@@ -3829,6 +3829,8 @@ static void cmd_append(char *tag, char *name, const char *cur_name) - const char *parseerr = NULL, *url = NULL; - struct appendstage *curstage; - mbentry_t *mbentry = NULL; -+ size_t maxsize = config_getint(IMAPOPT_APPEND_MAXSIZE) * 1024; -+ if (!maxsize) maxsize = UINT32_MAX; - - memset(&appendstate, 0, sizeof(struct appendstate)); - -@@ -4004,12 +4006,14 @@ static void cmd_append(char *tag, char *name, const char *cur_name) - size = 0; - r = append_catenate(curstage->f, cur_name, &size, - &(curstage->binary), &parseerr, &url); -+ if (!r && size > maxsize) r = IMAP_MESSAGE_TOO_LARGE; - if (r) goto done; - } - else { - /* Read size from literal */ - r = getliteralsize(arg.s, c, &size, &(curstage->binary), &parseerr); - if (!r && size == 0) r = IMAP_ZERO_LENGTH_LITERAL; -+ if (!r && size > maxsize) r = IMAP_MESSAGE_TOO_LARGE; - if (r) goto done; - - /* Copy message to stage */ -diff --git a/lib/imapoptions b/lib/imapoptions -index 5cb8ef7b8..786b288fe 100644 ---- a/lib/imapoptions -+++ b/lib/imapoptions -@@ -296,6 +296,10 @@ Blank lines and lines beginning with ``#'' are ignored. - but might be useful in the meantime for supporting old clients that - do not implement the RFC 5464 IMAP METADATA extension. */ - -+{ "append_maxsize", 0, INT, "3.3.2" } -+/* The size in kilobytes of the largest message that can be appended -+ via IMAP. If zero, no limit (i.e UINT32_MAX) */ -+ - { "aps_topic", NULL, STRING, "3.0.0" } - /* Topic for Apple Push Service registration. */ - { "aps_topic_caldav", NULL, STRING, "3.0.0" } --- -2.39.2 - - -From 02f158782578d4d99e0915c317ffe9d339180cca Mon Sep 17 00:00:00 2001 -From: Bron Gondwana -Date: Fri, 20 Nov 2020 12:54:58 +1100 -Subject: [PATCH 03/22] imapd: push the maxsize down into each parser to avoid - spooling - ---- - imap/imap_proxy.c | 7 ++++++- - imap/imap_proxy.h | 2 +- - imap/imapd.c | 42 ++++++++++++++++++------------------------ - imap/index.c | 7 ++++++- - imap/index.h | 2 +- - 5 files changed, 32 insertions(+), 28 deletions(-) - -diff --git a/imap/imap_proxy.c b/imap/imap_proxy.c -index fb585e680..2dac80455 100644 ---- a/imap/imap_proxy.c -+++ b/imap/imap_proxy.c -@@ -1207,7 +1207,7 @@ void proxy_copy(const char *tag, char *sequence, char *name, int myrights, - /* xxx end of separate proxy-only code */ - - int proxy_catenate_url(struct backend *s, struct imapurl *url, FILE *f, -- unsigned long *size, const char **parseerr) -+ size_t maxsize, unsigned long *size, const char **parseerr) - { - char mytag[128]; - int c, r = 0, found = 0; -@@ -1309,6 +1309,11 @@ int proxy_catenate_url(struct backend *s, struct imapurl *url, FILE *f, - if (c == '}') c = prot_getc(s->in); - if (c == '\r') c = prot_getc(s->in); - if (c != '\n') c = EOF; -+ if (sz > maxsize) { -+ r = IMAP_MESSAGE_TOO_LARGE; -+ eatline(s->in, c); -+ goto next_resp; -+ } - } - else if (c == 'n' || c == 'N') { - c = chomp(s->in, "il"); -diff --git a/imap/imap_proxy.h b/imap/imap_proxy.h -index aa2170960..89cb02002 100644 ---- a/imap/imap_proxy.h -+++ b/imap/imap_proxy.h -@@ -86,7 +86,7 @@ void proxy_copy(const char *tag, char *sequence, char *name, int myrights, - int usinguid, struct backend *s); - - int proxy_catenate_url(struct backend *s, struct imapurl *url, FILE *f, -- unsigned long *size, const char **parseerr); -+ size_t maxsize, unsigned long *size, const char **parseerr); - - int annotate_fetch_proxy(const char *server, const char *mbox_pat, - const strarray_t *entry_pat, -diff --git a/imap/imapd.c b/imap/imapd.c -index 48055ccce..2e55a6285 100644 ---- a/imap/imapd.c -+++ b/imap/imapd.c -@@ -3534,7 +3534,7 @@ static int isokflag(char *s, int *isseen) - } - } - --static int getliteralsize(const char *p, int c, -+static int getliteralsize(const char *p, int c, size_t maxsize, - unsigned *size, int *binary, const char **parseerr) - - { -@@ -3573,6 +3573,9 @@ static int getliteralsize(const char *p, int c, - return IMAP_PROTOCOL_ERROR; - } - -+ if (num > maxsize) -+ return IMAP_MESSAGE_TOO_LARGE; -+ - if (!isnowait) { - /* Tell client to send the message */ - prot_printf(imapd_out, "+ go ahead\r\n"); -@@ -3584,7 +3587,7 @@ static int getliteralsize(const char *p, int c, - return 0; - } - --static int catenate_text(FILE *f, unsigned *totalsize, int *binary, -+static int catenate_text(FILE *f, size_t maxsize, unsigned *totalsize, int *binary, - const char **parseerr) - { - int c; -@@ -3597,11 +3600,9 @@ static int catenate_text(FILE *f, unsigned *totalsize, int *binary, - c = getword(imapd_in, &arg); - - /* Read size from literal */ -- r = getliteralsize(arg.s, c, &size, binary, parseerr); -+ r = getliteralsize(arg.s, c, maxsize - *totalsize, &size, binary, parseerr); - if (r) return r; - -- if (*totalsize > UINT_MAX - size) r = IMAP_MESSAGE_TOO_LARGE; -- - /* Catenate message part to stage */ - while (size) { - n = prot_read(imapd_in, buf, size > 4096 ? 4096 : size); -@@ -3629,7 +3630,7 @@ static int catenate_text(FILE *f, unsigned *totalsize, int *binary, - } - - static int catenate_url(const char *s, const char *cur_name, FILE *f, -- unsigned *totalsize, const char **parseerr) -+ size_t maxsize, unsigned *totalsize, const char **parseerr) - { - struct imapurl url; - struct index_state *state; -@@ -3668,11 +3669,8 @@ static int catenate_url(const char *s, const char *cur_name, FILE *f, - proxy_userid, &backend_cached, - &backend_current, &backend_inbox, imapd_in); - if (be) { -- r = proxy_catenate_url(be, &url, f, &size, parseerr); -- if (*totalsize > UINT_MAX - size) -- r = IMAP_MESSAGE_TOO_LARGE; -- else -- *totalsize += size; -+ r = proxy_catenate_url(be, &url, f, maxsize - *totalsize, &size, parseerr); -+ *totalsize += size; - } - else - r = IMAP_SERVER_UNAVAILABLE; -@@ -3727,14 +3725,12 @@ static int catenate_url(const char *s, const char *cur_name, FILE *f, - struct protstream *s = prot_new(fileno(f), 1); - - r = index_urlfetch(state, msgno, 0, url.section, -- url.start_octet, url.octet_count, s, &size); -+ url.start_octet, url.octet_count, s, -+ maxsize - *totalsize, &size); - if (r == IMAP_BADURL) - *parseerr = "No such message part"; - else if (!r) { -- if (*totalsize > UINT_MAX - size) -- r = IMAP_MESSAGE_TOO_LARGE; -- else -- *totalsize += size; -+ *totalsize += size; - } - - prot_flush(s); -@@ -3751,7 +3747,7 @@ static int catenate_url(const char *s, const char *cur_name, FILE *f, - return r; - } - --static int append_catenate(FILE *f, const char *cur_name, unsigned *totalsize, -+static int append_catenate(FILE *f, const char *cur_name, size_t maxsize, unsigned *totalsize, - int *binary, const char **parseerr, const char **url) - { - int c, r = 0; -@@ -3765,7 +3761,7 @@ static int append_catenate(FILE *f, const char *cur_name, unsigned *totalsize, - } - - if (!strcasecmp(arg.s, "TEXT")) { -- int r1 = catenate_text(f, totalsize, binary, parseerr); -+ int r1 = catenate_text(f, maxsize, totalsize, binary, parseerr); - if (r1) return r1; - - /* if we see a SP, we're trying to catenate more than one part */ -@@ -3781,7 +3777,7 @@ static int append_catenate(FILE *f, const char *cur_name, unsigned *totalsize, - } - - if (!r) { -- r = catenate_url(arg.s, cur_name, f, totalsize, parseerr); -+ r = catenate_url(arg.s, cur_name, f, maxsize, totalsize, parseerr); - if (r) { - *url = arg.s; - return r; -@@ -4004,16 +4000,14 @@ static void cmd_append(char *tag, char *name, const char *cur_name) - - /* Catenate the message part(s) to stage */ - size = 0; -- r = append_catenate(curstage->f, cur_name, &size, -+ r = append_catenate(curstage->f, cur_name, maxsize, &size, - &(curstage->binary), &parseerr, &url); -- if (!r && size > maxsize) r = IMAP_MESSAGE_TOO_LARGE; - if (r) goto done; - } - else { - /* Read size from literal */ -- r = getliteralsize(arg.s, c, &size, &(curstage->binary), &parseerr); -+ r = getliteralsize(arg.s, c, maxsize, &size, &(curstage->binary), &parseerr); - if (!r && size == 0) r = IMAP_ZERO_LENGTH_LITERAL; -- if (!r && size > maxsize) r = IMAP_MESSAGE_TOO_LARGE; - if (r) goto done; - - /* Copy message to stage */ -@@ -14010,7 +14004,7 @@ static void cmd_urlfetch(char *tag) - } else { - r = index_urlfetch(state, msgno, params, url.section, - url.start_octet, url.octet_count, -- imapd_out, NULL); -+ imapd_out, UINT32_MAX, NULL); - } - - err: -diff --git a/imap/index.c b/imap/index.c -index ef537aa55..35ca866aa 100644 ---- a/imap/index.c -+++ b/imap/index.c -@@ -4550,7 +4550,7 @@ static int index_fetchreply(struct index_state *state, uint32_t msgno, - EXPORTED int index_urlfetch(struct index_state *state, uint32_t msgno, - unsigned params, const char *section, - unsigned long start_octet, unsigned long octet_count, -- struct protstream *pout, unsigned long *outsize) -+ struct protstream *pout, size_t maxsize, unsigned long *outsize) - { - /* dumbass eM_Client sends this: - * A4 APPEND "INBOX.Junk Mail" () "14-Jul-2013 17:01:02 +0000" -@@ -4723,6 +4723,11 @@ EXPORTED int index_urlfetch(struct index_state *state, uint32_t msgno, - n = size - start_octet; - } - -+ if (n > maxsize) { -+ r = IMAP_MESSAGE_TOO_LARGE; -+ goto done; -+ } -+ - if (outsize) { - /* Return size (CATENATE) */ - *outsize = n; -diff --git a/imap/index.h b/imap/index.h -index 196607f3f..bf8006d9b 100644 ---- a/imap/index.h -+++ b/imap/index.h -@@ -303,7 +303,7 @@ extern struct seqset *index_vanished(struct index_state *state, - extern int index_urlfetch(struct index_state *state, uint32_t msgno, - unsigned params, const char *section, - unsigned long start_octet, unsigned long octet_count, -- struct protstream *pout, unsigned long *size); -+ struct protstream *pout, size_t maxsize, unsigned long *size); - extern char *index_get_msgid(struct index_state *state, uint32_t msgno); - extern struct nntp_overview *index_overview(struct index_state *state, - uint32_t msgno); --- -2.39.2 - - -From 133a11ebfd9e3f659da3081d8e7c9f416c8ead3b Mon Sep 17 00:00:00 2001 -From: Bron Gondwana -Date: Tue, 1 Dec 2020 08:11:31 +1100 -Subject: [PATCH 04/22] use maxmessagesize rather than our own config option - ---- - imap/imapd.c | 2 +- - lib/imapoptions | 4 ---- - 2 files changed, 1 insertion(+), 5 deletions(-) - -diff --git a/imap/imapd.c b/imap/imapd.c -index 2e55a6285..d9a9dd776 100644 ---- a/imap/imapd.c -+++ b/imap/imapd.c -@@ -3825,7 +3825,7 @@ static void cmd_append(char *tag, char *name, const char *cur_name) - const char *parseerr = NULL, *url = NULL; - struct appendstage *curstage; - mbentry_t *mbentry = NULL; -- size_t maxsize = config_getint(IMAPOPT_APPEND_MAXSIZE) * 1024; -+ size_t maxsize = config_getint(IMAPOPT_MAXMESSAGESIZE) * 1024; - if (!maxsize) maxsize = UINT32_MAX; - - memset(&appendstate, 0, sizeof(struct appendstate)); -diff --git a/lib/imapoptions b/lib/imapoptions -index 786b288fe..5cb8ef7b8 100644 ---- a/lib/imapoptions -+++ b/lib/imapoptions -@@ -296,10 +296,6 @@ Blank lines and lines beginning with ``#'' are ignored. - but might be useful in the meantime for supporting old clients that - do not implement the RFC 5464 IMAP METADATA extension. */ - --{ "append_maxsize", 0, INT, "3.3.2" } --/* The size in kilobytes of the largest message that can be appended -- via IMAP. If zero, no limit (i.e UINT32_MAX) */ -- - { "aps_topic", NULL, STRING, "3.0.0" } - /* Topic for Apple Push Service registration. */ - { "aps_topic_caldav", NULL, STRING, "3.0.0" } --- -2.39.2 - - -From ddc431769b61eef06550da624c1c99a2fd620dbb Mon Sep 17 00:00:00 2001 -From: ellie timoney -Date: Wed, 27 Mar 2024 11:31:58 +1100 -Subject: [PATCH 05/22] imapd: read maxmsgsize once at startup - -Based on: -40793dfde8c96797d86f80e9f461bea61bca3bc9 imapd.c: Advertise APPENDLIMIT= capability - -but without introducing the APPENDLIMIT= capability ---- - imap/imapd.c | 6 ++++-- - 1 file changed, 4 insertions(+), 2 deletions(-) - -diff --git a/imap/imapd.c b/imap/imapd.c -index d9a9dd776..e7cf600c7 100644 ---- a/imap/imapd.c -+++ b/imap/imapd.c -@@ -135,6 +135,7 @@ static int imaps = 0; - static sasl_ssf_t extprops_ssf = 0; - static int nosaslpasswdcheck = 0; - static int apns_enabled = 0; -+static size_t maxsize = 0; - - /* PROXY STUFF */ - /* we want a list of our outgoing connections here and which one we're -@@ -908,6 +909,9 @@ int service_init(int argc, char **argv, char **envp) - - prometheus_increment(CYRUS_IMAP_READY_LISTENERS); - -+ maxsize = config_getint(IMAPOPT_MAXMESSAGESIZE) * 1024; -+ if (!maxsize) maxsize = UINT32_MAX; -+ - return 0; - } - -@@ -3825,8 +3829,6 @@ static void cmd_append(char *tag, char *name, const char *cur_name) - const char *parseerr = NULL, *url = NULL; - struct appendstage *curstage; - mbentry_t *mbentry = NULL; -- size_t maxsize = config_getint(IMAPOPT_MAXMESSAGESIZE) * 1024; -- if (!maxsize) maxsize = UINT32_MAX; - - memset(&appendstate, 0, sizeof(struct appendstate)); - --- -2.39.2 - - -From a32fe042bc503a36393e7d888b26b6c1759cf6b0 Mon Sep 17 00:00:00 2001 -From: Matthew Horsfall -Date: Wed, 15 Jun 2022 14:57:02 -0400 -Subject: [PATCH 06/22] imap/imapd.c: IMAPOPT_MAXMESSAGESIZE is bytes, not - kilobytes - -I think this was a mistake added in bf28aa3fb6 when replacing *** 4888 LINES SKIPPED ***