From nobody Wed Jun 8 21:51:48 2022 X-Original-To: ports-bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 30BDA838AD4 for ; Wed, 8 Jun 2022 21:51:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LJLXg6SH1z4SZ2 for ; Wed, 8 Jun 2022 21:51:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BE11117C66 for ; Wed, 8 Jun 2022 21:51:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 258Lplio030101 for ; Wed, 8 Jun 2022 21:51:47 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 258LplOv030100 for ports-bugs@FreeBSD.org; Wed, 8 Jun 2022 21:51:47 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: ports-bugs@FreeBSD.org Subject: [Bug 264558] www/gitlab sidekiq crashes shortly after starting Date: Wed, 08 Jun 2022 21:51:48 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: gwright@antiope.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Ports bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-ports-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports-bugs@freebsd.org X-BeenThere: freebsd-ports-bugs@freebsd.org MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1654725107; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=rnBIvTnqzqjMlp94DB1NHX4aths0YvTkfjppgkdi//A=; b=I78vxb2mzoI2C6NQZZUPUcLEJyFhflPg08azxgHLrRAZZmHjUIPLdLqwhsavYP83mV6+ID 65u0XymCdtz77jPIXoPl1C8nYJcnp5lX8/cVwIjZgLStI3R6KOsGVUBajgtsv5usMHyy7V 6rlJi58MJFCHQzL9/TnVm4SFw6tJRkeiRhZZlBjvmXDfar35zK+Ru4XYBuLdSbh/RQaIWz xO2Uj+kcO4CBdhg6HEft5v3V9PTBl9vRz8QqYKB+32uXN/CSBgNOFEwBiYfqAXTIaPlMN6 P/HEVG+355VJAVtn6hXxZ9+n/WSoekDldOL8V8PagBErHiolN1+vH92nPPiSXw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1654725107; a=rsa-sha256; cv=none; b=jLg75X6zDwFswFO+wjNUbwZwM1UgKpZdAwCRBMcAX0KVmK2XYLIEXY5rbsOC8H551XTzSt 2wOrQ7ASmbjLm1AP8CSptCL5ESiTOxHYy3dpZ4eB01ZSxRabNZv7Z8MHE23/MT2/5ZsICL PQscwOUR6ZZZ0mzzcJL8a1pjEGqgI0gM4ewhvWQKwjIZHUazFlbMAVA+07wA18jEBVqEbp 4/kRebsfq+IP6RWit703LAPIar9vWn5bO7ab/Zx0hm3M7s7Ag4Zfd2SHOh+tt78h4lJdfn 7aiav6gGdAbhY7TOaAJgEZ+wVsOlyVKAlnQC8M4ZC9vjDHWeWmAkvjIbL4pDCQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D264558 Bug ID: 264558 Summary: www/gitlab sidekiq crashes shortly after starting Product: Ports & Packages Version: Latest Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: Individual Port(s) Assignee: ports-bugs@FreeBSD.org Reporter: gwright@antiope.com On FreeBSD 13.1-amd64, the sidekiq job dispatcher (a dependency of gitlab-c= e) crashes shortly after starting. This happens with either gitlab-ce version 15.0.0 or version 15.0.1. An initial restart seems to show everything is running, but after less than= a minute, sidekiq is dead: ``` root@gitlab:/usr/local/www/gitlab-ce/log # service gitlab restart Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will break this application for all non-root users on this machine. Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will break this application for all non-root users on this machine. Removing stale Sidekiq job dispatcher pid. This is most likely caused by Sidekiq crashing the last time it ran. Shutting down GitLab web server 61650 Shutting down GitLab Workhorse Shutting down Gitaly ........... GitLab is not running. Starting GitLab web server (puma) Starting GitLab Sidekiq Starting GitLab Workhorse Starting Gitaly .No pidfile found at /usr/local/www/gitlab-ce/tmp/pids/sidekiq-cluster.pid;= is Sidekiq running? .{"timestamp":"2022-06-08T21:26:41.366Z","pid":35695,"message":"Puma starti= ng in cluster mode..."} {"timestamp":"2022-06-08T21:26:41.366Z","pid":35695,"message":"* Puma versi= on: 5.6.4 (ruby 3.0.4-p208) (\"Birdie's Version\")"} {"timestamp":"2022-06-08T21:26:41.366Z","pid":35695,"message":"* Min threa= ds: 1"} {"timestamp":"2022-06-08T21:26:41.366Z","pid":35695,"message":"* Max threa= ds: 16"} {"timestamp":"2022-06-08T21:26:41.367Z","pid":35695,"message":"* Environme= nt: production"} {"timestamp":"2022-06-08T21:26:41.367Z","pid":35695,"message":"* Master P= ID: 35695"} {"timestamp":"2022-06-08T21:26:41.367Z","pid":35695,"message":"* Worke= rs: 8"} {"timestamp":"2022-06-08T21:26:41.367Z","pid":35695,"message":"* Restar= ts: (=E2=9C=94) hot (=E2=9C=96) phased"} {"timestamp":"2022-06-08T21:26:41.367Z","pid":35695,"message":"* Preloading application"} ..Top level ::CompositeIO is deprecated, require 'multipart/post' and use `Multipart::Post::CompositeReadIO` instead! Top level ::Parts is deprecated, require 'multipart/post' and use `Multipart::Post::Parts` instead! ............................................{"timestamp":"2022-06-08T21:27:= 28.689Z","pid":35695,"message":"* Listening on unix:///usr/local/www/gitlab-ce/tmp/sockets/g itlab.socket"} {"timestamp":"2022-06-08T21:27:28.689Z","pid":35695,"message":"! WARNING: Detected 1 Thread(s) started in app boot:"} {"timestamp":"2022-06-08T21:27:28.689Z","pid":35695,"message":"! #\u003cThread:0x0000000834a57828 /usr/local/lib/ruby/gems/3.0/gems/rack-timeout-0.5.2/lib/rack/timeout/ support/scheduler.rb:73 sleep\u003e - /usr/local/lib/ruby/gems/3.0/gems/rack-timeout-0.5.2/lib/rack/timeout/suppo= rt/scheduler.rb:91:in `sleep'"} {"timestamp":"2022-06-08T21:27:28.689Z","pid":35695,"message":"Use Ctrl-C to stop"} Started in 48s. The GitLab web server with pid 35695 is running. The GitLab Sidekiq job dispatcher with pid 35705 is running. The GitLab Workhorse with pid 35711 is running. Gitaly with pid 35710 is running. GitLab and all its components are up and running. root@gitlab:/usr/local/www/gitlab-ce/log # ls -l sidekiq.log -rw-r--r-- 1 git www 826738 Jun 8 21:27 sidekiq.log root@gitlab:/usr/local/www/gitlab-ce/log # service gitlab status The GitLab web server with pid 35695 is running. The GitLab Sidekiq job dispatcher is not running. The GitLab Workhorse with pid 35711 is running. Gitaly with pid 35710 is running. root@gitlab:/usr/local/www/gitlab-ce/log # ``` The only error I see in /usr/local/www/gitlab-ce/log/sidekiq.log is ``` {"severity":"ERROR","time":"2022-06-05T22:24:21.142Z","message":"Heartbeat thread error: Error connecting to Redis on /var/run/redis/redis.sock (Errno::ENOENT)"} ``` but the redis socket does exist, and the redis process in running: ``` root@gitlab:/usr/local/www/gitlab-ce/log # ps ax | grep redis 60532 - SsJ 0:11.17 redis-server: /usr/local/bin/redis-server unixsocket:/var/run/redis/redis.sock (redis-server) ``` I set the log level for redis to `debug` but I see no errors in the log. Is there a way to set the sidekiq log level similarly? I didn't see one, but I might have missed something. I tried both a complete uninstall and reinstall of gitlab-ce 15.0.1 from packages, and sidekiq still crashed a few seconds after starting. I also d= id a complete uninstall and reinstall from ports with the same results. Without sidekiq, it is not possible to run continuous integration jobs, sin= ce they are dispatched by sidekiq. --=20 You are receiving this mail because: You are the assignee for the bug.=