From nobody Tue May 3 07:48:35 2022 X-Original-To: freebsd-git@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 52D171AB2FD3 for ; Tue, 3 May 2022 07:49:05 +0000 (UTC) (envelope-from dch@skunkwerks.at) Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4KssXv6lxYz3nZ2 for ; Tue, 3 May 2022 07:49:03 +0000 (UTC) (envelope-from dch@skunkwerks.at) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 5AA4232009E8; Tue, 3 May 2022 03:48:56 -0400 (EDT) Received: from imap44 ([10.202.2.94]) by compute2.internal (MEProxy); Tue, 03 May 2022 03:48:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=skunkwerks.at; h=cc:content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm2; t=1651564135; x=1651650535; bh=Q1rckjiy08 M9+bfHkzORBExZ/sctr+sFkx7+DDrwyt8=; b=RojQFhjlWcuaeXKD7pr2oY2/D2 ytrcfBkt9QXEFvm92+9zSkbRSfcuWHB+KnJI+QAX4toz2nnwmEeQdp8hyhB/N5va AKC3XyIbo88qFqlpiCBMglyBR8W/8bdHXOtunaR64rk4JZuaJ7FhiN4vSyytuUb7 Etd2J2KPuekKGIF4AgQcnp9zp/s4GF1hvwNwjpLBLkbzbRfJkjhZDW8c15Q0gC0W O21LOidxwV3+mThGqPPQYLmD9HNvRSrHJKeKhkDFUxiF+SmftYiZgzIoT0jX1ZOK J+hCzNtVAYbKJs5VfCDjUJPX4ZK4ffM+nQDU9/89iCCYkmLZ1+I4XP9d+adA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1651564135; x= 1651650535; bh=Q1rckjiy08M9+bfHkzORBExZ/sctr+sFkx7+DDrwyt8=; b=J 2pWEqruYMuMHy3RAXiIlfI9AGkNI+OkdzccWEOIn0S/3Kh84T9YYVeLflusbJIxR jXHQ+tL/OTb8Zhhj5rtzjFazpe/QlnF/o1Xo+AnEDC25qC6MuIvwLFtNJYKe2D4Z PoVYoSawWFvK61QTYZiZjceIgFdzeewxnkoppHTgsloRlC/0xMAdYQYPlSdUUzGi 5XCB5hPH/YTDHtG1q3pOdkuVXIZawkGepdLw5MX7IunSnYiODIsePzRORVKiVHGg uLIayFmMsk4UthXjnRw5uQol0R+oE2Hjar07iYr7Ge1oob8CjHvtQqfxKhzZjpKS EAF5WmrzdNZHnenAw9HVw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdeigdduvdegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpedfffgrvhgvucevohhtthhlvghhuhgsvghrfdcuoegutghh sehskhhunhhkfigvrhhkshdrrghtqeenucggtffrrghtthgvrhhnpeejvdegvddvueeiue fgleegffeuuedugfekvedvkeegvdeivdegtdeluedufeevieenucffohhmrghinhepfhhr vggvsghsugdrohhrghdpghhithdqshgtmhdrtghomhenucevlhhushhtvghrufhiiigvpe dtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegutghhsehskhhunhhkfigvrhhkshdrrght X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 5F90F36A005C; Tue, 3 May 2022 03:48:55 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.7.0-alpha0-591-gfe6c3a2700-fm-20220427.001-gfe6c3a27 List-Id: Discussion of git use in the FreeBSD project List-Archive: https://lists.freebsd.org/archives/freebsd-git List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-git@freebsd.org Mime-Version: 1.0 Message-Id: In-Reply-To: <1149BE63-059F-4093-B651-4DF2C413E012@freebsd.org> References: <7dc2545d68f3d4ebbedef4d29c6161f5@bsdforge.com> <81AFA1F5-E3EE-4E33-A2D7-58DB775674C3@freebsd.org> <26dc45b81218bd3b670145340035ad66@bsdforge.com> <1149BE63-059F-4093-B651-4DF2C413E012@freebsd.org> Date: Tue, 03 May 2022 07:48:35 +0000 From: "Dave Cottlehuber" To: freebsd-git@freebsd.org, bsd-lists@bsdforge.com Subject: Re: When are the git servers available to obtain the ports tree? Content-Type: text/plain X-Rspamd-Queue-Id: 4KssXv6lxYz3nZ2 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=skunkwerks.at header.s=fm2 header.b=RojQFhjl; dkim=pass header.d=messagingengine.com header.s=fm1 header.b="J 2pWEqr"; dmarc=none; spf=pass (mx1.freebsd.org: domain of dch@skunkwerks.at designates 64.147.123.20 as permitted sender) smtp.mailfrom=dch@skunkwerks.at X-Spamd-Result: default: False [-2.59 / 15.00]; XM_UA_NO_VERSION(0.01)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.20]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[skunkwerks.at:+,messagingengine.com:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.20:from]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[skunkwerks.at:s=fm2,messagingengine.com:s=fm1]; FREEFALL_USER(0.00)[dch]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[skunkwerks.at]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-git]; MID_RHS_WWW(0.50)[] X-ThisMailContainsUnwantedMimeParts: N On Tue, 3 May 2022, at 04:04, Philip Paeps wrote: >>>> I'm a maintainer for well over 100 ports. But more often than >>>> not, I am not permitted to obtain the ports tree from any of >>>> the FreeBSD git servers: >>>> >>>> # git clone -o freebsd --config >>>> remote.freebsd.fetch='+refs/notes/*:refs/notes/*' >>>> https://git.freebsd.org/ports.git PORTS-20220502 The initial full clone from github, of ports and/or src, often fails with gateway errors. I've bugged them in the past and they have amended their "limits" once, and given up the next time. I'm curious about the "shallow-info" appearing in your logs though. https://git-scm.com/book/en/v2/Git-Internals-Environment-Variables There's extensive debugging for git, and for HTTP, you could try `GIT_CURL_VERBOSE=1` and see if that helps identify problems. I've previously seen gateway errors when the remote git server takes too long to prepare whatever packfile it needs to send me. I'm murky on git internals so I don't really understand what it's failing on. Generally, I only cloned a single branch, and fetch main, which seems to have removed issues for me, but I do this at least 1x/week. My freebsd remote is called upstream. # fetch a single branch instead of all branches git fetch upstream main # my snippet from /usr/ports/.git/config [remote "upstream"] url = https://git.freebsd.org/ports.git fetch = +refs/heads/main:refs/remotes/upstream/main fetch = +refs/notes/*:refs/notes/* pushurl = ssh://git@gitrepo.freebsd.org/ports.git [branch "upstream"] remote = upstream merge = refs/heads/main If, for some reason, you really need to do a from-scratch clone every time, it might be simpler to have a personal mirror of just the /main/ branch on github or similar, and avoid pulling the whole lot from the mirrors each time. You can also put `git fetch ` into a crontab and let it catch up daily and with less to fetch. A+ Dave