From nobody Fri Apr 1 13:06:40 2022 X-Original-To: freebsd-questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A6CF51A44076 for ; Fri, 1 Apr 2022 13:06:45 +0000 (UTC) (envelope-from c.brinkhaus@t-online.de) Received: from mailout09.t-online.de (mailout09.t-online.de [194.25.134.84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mailout00.t-online.de", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KVL6F0ZGFz4gtS for ; Fri, 1 Apr 2022 13:06:44 +0000 (UTC) (envelope-from c.brinkhaus@t-online.de) Received: from fwd79.dcpf.telekom.de (fwd79.aul.t-online.de [10.223.144.105]) by mailout09.t-online.de (Postfix) with SMTP id EDE191511C; Fri, 1 Apr 2022 15:06:43 +0200 (CEST) Received: from celsius.local ([217.226.189.115]) by fwd79.t-online.de with (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384 encrypted) esmtp id 1naGzC-1PGHFh0; Fri, 1 Apr 2022 15:06:43 +0200 Received: by celsius.local (Postfix, from userid 1001) id 4E49B9008E; Fri, 1 Apr 2022 15:06:40 +0200 (CEST) Date: Fri, 1 Apr 2022 15:06:40 +0200 From: Christoph Brinkhaus To: Budi Janto Cc: William Dudley , freebsd-questions@freebsd.org Subject: Re: Fatal trap 12 Message-ID: References: <2f1cf9d8-b9b0-4ec1-50fc-ebd4f469a82b@studiokaraoke.co.id> <0958bfdb-ebad-c860-8138-6e7010293c43@studiokaraoke.co.id> List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <0958bfdb-ebad-c860-8138-6e7010293c43@studiokaraoke.co.id> X-TOI-EXPURGATEID: 150726::1648818403-00010BFA-6D6232A7/0/0 CLEAN NORMAL X-TOI-MSGID: af8ed9f0-f5e8-4aa0-a5cf-417c5a39dd27 X-Rspamd-Queue-Id: 4KVL6F0ZGFz4gtS X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of c.brinkhaus@t-online.de has no SPF policy when checking 194.25.134.84) smtp.mailfrom=c.brinkhaus@t-online.de X-Spamd-Result: default: False [-1.59 / 15.00]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[t-online.de]; RWL_MAILSPIKE_GOOD(0.00)[194.25.134.84:from]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:3320, ipnet:194.25.0.0/16, country:DE]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[t-online.de]; RECEIVED_SPAMHAUS_PBL(0.00)[217.226.189.115:received]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-0.99)[-0.990]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[t-online.de]; AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[194.25.134.84:from]; MLMMJ_DEST(0.00)[freebsd-questions]; R_SPF_NA(0.00)[no SPF record]; MID_RHS_NOT_FQDN(0.50)[]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org] X-ThisMailContainsUnwantedMimeParts: N Am Fri, Apr 01, 2022 at 04:49:52AM +0700 schrieb Budi Janto: Hello Budi! > On 4/1/22 04:31, William Dudley wrote: > > bad RAM? > > > > Bill Dudley > > I don't know exactly what happened, but I forgot to tell that ZFS > Scrubbing was running when this issue came up. > > # sysctl -a | grep 'hw.*mem' > hw.physmem: 17035481088 > hw.usermem: 14230319104 > hw.realmem: 17179869184 > hw.pci.host_mem_start: 2147483648 > hw.cbb.start_memory: 2281701376 > > # top -b > last pid: 2621; load averages: 0.81, 0.73, 0.54 up 0+00:54:00 > 04:44:40 > 28 processes: 1 running, 27 sleeping > CPU 0: 0.0% user, 0.0% nice, 6.2% system, 6.5% interrupt, 87.2% idle > CPU 1: 0.0% user, 0.0% nice, 6.7% system, 0.0% interrupt, 93.2% idle > CPU 2: 0.0% user, 0.0% nice, 6.0% system, 0.2% interrupt, 93.8% idle > CPU 3: 0.0% user, 0.0% nice, 6.0% system, 0.2% interrupt, 93.8% idle > Mem: 159M Active, 2714M Inact, 2673M Wired, 345M Buf, 10G Free > ARC: 795M Total, 717M MFU, 66M MRU, 10M Header, 1856K Other > 755M Compressed, 5172M Uncompressed, 6.85:1 Ratio > Swap: 3979M Total, 3979M Free > It might help to limit the arc. I have in /boot/loader.conf lines as below: # grep arc /boot/loader.conf vfs.zfs.compressed_arc_enabled="0" vfs.zfs.arc_max="500M" I am not sure why I have disabled compressed arc, too. It is likely due to some advise. > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU > COMMAND > 968 root 1 20 0 32M 12M select 0 0:03 0.00% > snmpd > 2227 mysql 37 20 0 2737M 279M select 3 0:02 0.00% > mysqld > 944 nobody 1 20 0 18M 7272K select 2 0:00 0.00% > openvpn > 938 ntpd 1 20 0 21M 5028K select 3 0:00 0.00% ntpd > 360 _pflogd 1 20 0 13M 2996K bpf 3 0:00 0.00% > pflogd > 2372 wilson 1 20 0 21M 9324K select 1 0:00 0.00% sshd > 999 mysql 1 52 0 13M 3020K wait 0 0:00 0.00% sh > 2378 root 1 20 0 16M 4840K pause 3 0:00 0.00% csh > 856 root 1 20 0 13M 2624K select 1 0:00 0.00% > syslogd > 1118 root 1 23 0 13M 2644K nanslp 3 0:00 0.00% cron > 2369 root 1 20 0 21M 9284K select 2 0:00 0.00% sshd > 2373 wilson 1 21 0 14M 4192K pause 2 0:00 0.00% csh > 2377 wilson 1 20 0 13M 3168K wait 2 0:00 0.00% su > 660 root 1 20 0 11M 1412K select 3 0:00 0.00% devd > 664 root 1 20 0 19M 5840K select 1 0:00 0.00% zfsd > 1187 root 1 52 0 13M 2268K ttyin 1 0:00 0.00% > getty > 1191 root 1 52 0 13M 2268K ttyin 3 0:00 0.00% > getty > 1184 root 1 52 0 13M 2268K ttyin 3 0:00 0.00% > getty > > # zpool status -v > pool: pool > state: ONLINE > scan: scrub in progress since Fri Apr 1 01:00:00 2022 > 4.75T scanned at 0B/s, 3.96T issued at 0B/s, 4.91T total > 0B repaired, 80.60% done, no estimated completion time > config: > > NAME STATE READ WRITE CKSUM > pool ONLINE 0 0 0 > diskid/DISK-WD-WMC1T1587601p1 ONLINE 0 0 0 > diskid/DISK-WD-WCC1T1170612p1 ONLINE 0 0 0 > diskid/DISK-Z1F5T8W1p1 ONLINE 0 0 0 > > errors: No known data errors > > Thank You. BTW: There has been a thread about memory issues in the FreeBSD forum where ZFS and mysql has been involved. As far as I remember it has been stated that mysql can be a memory hog, too. Limiting ARC has helped but I do not remember if there has been some additional tuning on mysql required, too. Kind regards, Christoph