From owner-freebsd-gnome@freebsd.org Tue Mar 26 22:39:21 2019 Return-Path: Delivered-To: freebsd-gnome@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E47931562347 for ; Tue, 26 Mar 2019 22:39:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 7BC198A2D0 for ; Tue, 26 Mar 2019 22:39:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 35D12156233F; Tue, 26 Mar 2019 22:39:20 +0000 (UTC) Delivered-To: gnome@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EDE8B156233D for ; Tue, 26 Mar 2019 22:39:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8C7AD8A2CD for ; Tue, 26 Mar 2019 22:39:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id BC46BCEDB for ; Tue, 26 Mar 2019 22:39:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x2QMdIuY055261 for ; Tue, 26 Mar 2019 22:39:18 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x2QMdIVd055260 for gnome@FreeBSD.org; Tue, 26 Mar 2019 22:39:18 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" From: bugzilla-noreply@freebsd.org To: gnome@FreeBSD.org Subject: maintainer-feedback requested: [Bug 236815] devel/glib20: loops over all possible file descriptors Date: Tue, 26 Mar 2019 22:39:18 +0000 X-Bugzilla-Type: request X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: gnome@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? Message-ID: In-Reply-To: References: X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-gnome@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: GNOME for FreeBSD -- porting and maintaining List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Mar 2019 22:39:21 -0000 Bugzilla Automation has asked freebsd-gnome mailing = list for maintainer-feedback: Bug 236815: devel/glib20: loops over all possible file descriptors https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D236815 --- Description --- For a server having 360GB of RAM and these defaults: kern.maxfiles: 11520879 kern.maxfilesperproc: 10368783 g_spawn_sync() invokes fork_exec_with_pipes() then do_exec() and finally fdwalk (set_cloexec, GINT_TO_POINTER(3)) fdwalk() function loops over all possible file descriptors no matter open or not and does fcntl(X,F_SETFD,FD_CLOEXEC) for every possible X. So, if I run the Midnight Commander and use F3 (view file) for a file called "natt.diff", mc_popen() function calls g_spawn_async_with_pipes() then g_spawn_sync() is called (see above) is over 10 million of system calls (fc= ntl) are performed for such a machine. This is quite slow and takes several seco= nds in total. "Internal view" is used (per defaults). This part of glib2 code is not used for Linux, though as it has fdwalk() in= its Here comes full backtrace of mc binary built with debugging symbols and kil= led while it was looping: #0 0x0000000801b984fa in _read () from /lib/libc.so.7 (gdb) bt #0 0x0000000801b984fa in _read () from /lib/libc.so.7 #1 0x0000000803220c46 in pthread_suspend_all_np () from /lib/libthr.so.3 #2 0x0000000801397a6e in g_spawn_close_pid () from /usr/local/lib/libglib-2.0.so.0 #3 0x0000000801396b37 in g_spawn_sync () from /usr/local/lib/libglib-2.0.s= o.0 #4 0x00000008013962e8 in g_spawn_async_with_pipes () from /usr/local/lib/libglib-2.0.so.0 #5 0x000000000046e47e in mc_popen (command=3D0x803842520 "/bin/sh /tmp/mc-root/mcextZNX6YZ", error=3D0x7fffffffdd58) at utilunix.c:525 #6 0x00000000004f05be in mcview_load_command_output (view=3D0x8038ac180, command=3D0x803842520 "/bin/sh /tmp/mc-root/mcextZNX6YZ") at datasource= .c:391 #7 0x000000000048ac39 in mcview_load (view=3D0x8038ac180, command=3D0x803842520 "/bin/sh /tmp/mc-root/mcextZNX6YZ", file=3D0x80381b9c0 "/usr/ports/security/ipsec-tools/natt.diff", start_l= ine=3D0, search_start=3D0, search_end=3D0) at mcviewer.c:312 #8 0x000000000048a9ac in mcview_viewer (command=3D0x803842520 "/bin/sh /tmp/mc-root/mcextZNX6YZ", file_vpath=3D0x8038c16e0, start_line=3D0, search_start=3D0, search_end= =3D0) at mcviewer.c:251 #9 0x0000000000423ff1 in exec_extension_view (target=3D0x0, cmd=3D0x803842520 "/bin/sh /tmp/mc-root/mcextZNX6YZ", filename_vpath=3D0x8038c16e0, start_line=3D0) at ext.c:376 #10 0x00000000004234cb in exec_extension (target=3D0x0, filename_vpath=3D0x8038c16e0, lc_data=3D0x8038946a0 "%view{ascii} /usr/local/libexec/mc/ext.d/misc.sh= view cat\n\n# ar library\ #11 0x0000000000422c5e in regex_command_for (target=3D0x0, filename_vpath=3D0x8038c16e0, action=3D0x7fffffffe530 "View", script_vpath=3D0x0) at ext.c:1025 #12 0x00000000004d0664 in regex_command (filename_vpath=3D0x8038c16e0, action=3D0x7fffffffe530 "View") at ext.h:30 #13 0x00000000004d053a in view_file_at_line (filename_vpath=3D0x8038c16e0, plain_view=3D0, #14 0x00000000004d06a1 in view_file (filename_vpath=3D0x8038c16e0, plain_vi= ew=3D0, internal=3D1) at cmd.c:600 #15 0x00000000004d0869 in do_view_cmd (normal=3D0) at cmd.c:157 #16 0x00000000004d06bb in view_cmd () at cmd.c:610 #17 0x000000000044478c in midnight_execute_cmd (sender=3D0x8038431c0, command=3D101) at midnight.c:1380 #18 0x0000000000443dc0 in midnight_callback (w=3D0x80385d600, sender=3D0x80= 38431c0, msg=3DMSG_ACTION, parm=3D101, data=3D0x0) at midnight.c:1580 #19 0x000000000041ad1a in send_message (w=3D0x80385d600, sender=3D0x8038431= c0, msg=3DMSG_ACTION, parm=3D101, data=3D0x0) at widget-common.h:210 #20 0x000000000041aabf in buttonbar_call (bb=3D0x8038431c0, i=3D2) at buttonbar.c:155 #21 0x000000000041a5e7 in buttonbar_callback (w=3D0x8038431c0, sender=3D0x0, msg=3DMSG_HOTKEY, parm=3D1003, data=3D0x0) at buttonbar.c:172 #22 0x000000000042003a in send_message (w=3D0x8038431c0, sender=3D0x0, msg=3DMSG_HOTKEY, parm=3D1003, data=3D0x0) at widget-common.h:210 #23 0x0000000000421c9a in dlg_try_hotkey (h=3D0x80385d600, d_key=3D1003) at dialog.c:435 #24 0x00000000004212d1 in dlg_key_event (h=3D0x80385d600, d_key=3D1003) at dialog.c:480 #25 0x0000000000420da7 in dlg_process_event (h=3D0x80385d600, key=3D1003, event=3D0x7fffffffe960) at dialog.c:1162 #26 0x0000000000421678 in frontend_dlg_run (h=3D0x80385d600) at dialog.c:542 #27 0x00000000004214ae in dlg_run (h=3D0x80385d600) at dialog.c:1193 #28 0x00000000004436bd in do_nc () at midnight.c:1791 #29 0x000000000041054b in main (argc=3D1, argv=3D0x7fffffffeb38) at main.c:= 409