Date: Sun, 25 Jun 2017 17:18:18 +0200 (CEST) From: =?ISO-8859-1?Q?Trond_Endrest=F8l?= <Trond.Endrestol@fagskolen.gjovik.no> To: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: Current amd64 new error or warning from today's current with ruby r320323 Message-ID: <alpine.BSF.2.21.1706251709340.37790@mail.fig.ol.no> In-Reply-To: <20170625134133.GA3437@kib.kiev.ua> References: <44028FBF-C8B1-4A69-B798-F4E28CFCD779@pozo.com> <20170625012359.GS3437@kib.kiev.ua> <040BF7D1-2FDF-415B-9A17-ADD608503F14@pozo.com> <20170625020441.GT3437@kib.kiev.ua> <219E6B59-EF90-443D-8B53-4C39C4D6CFBD@pozo.com> <20170625134133.GA3437@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, 25 Jun 2017 16:41+0300, Konstantin Belousov wrote: > On Sat, Jun 24, 2017 at 07:19:25PM -0700, Manfred Antar wrote: > > > > > On Jun 24, 2017, at 7:04 PM, Konstantin Belousov <kostikbel@gmail.com> wrote: > > > > > > On Sat, Jun 24, 2017 at 06:48:03PM -0700, Manfred Antar wrote: > > >> > > >>> On Jun 24, 2017, at 6:23 PM, Konstantin Belousov <kostikbel@gmail.com> wrote: > > >>> > > >>> On Sat, Jun 24, 2017 at 06:08:50PM -0700, Manfred Antar wrote: > > >>>> New world and kernel r320323 > > >>>> I get a new error or message when using ruby: > > >>>> > > >>>> > > >>>> /usr/local/sbin/portupgrade -av > > >>>> <main>: warning: pthread_create failed for timer: Resource temporarily unavailable, scheduling broken > > >>>> > > >>>> everything works just this message when using ruby. I recompiled ruby , still same message > > >>>> > > >>>> /usr/local/bin/ruby -v > > >>>> <main>: warning: pthread_create failed for timer: Resource temporarily unavailable, scheduling broken > > >>>> ruby 2.3.4p301 (2017-03-30 revision 58214) [amd64-freebsd12] > > >>>> > > >>>> Not sure what???s changed, I noticed some commits to vm stuff, maybe thats it. > > >>> > > >>> ktrace your failing ruby invocation, then post output of kdump -H somewhere. > > >>> > > >> > > >> Ok not sure if this is right , but this is what i did: > > >> > > >> (tmp)4637}ktrace /usr/local/bin/ruby -v > > >> <main>: warning: pthread_create failed for timer: Resource temporarily unavailable, scheduling broken > > >> ruby 2.3.4p301 (2017-03-30 revision 58214) [amd64-freebsd12] > > >> > > >> (tmp)4638}kdump -H -f ./ktrace.out > kdump.txt > > >> > > >> you can get kdump.txt at: > > >> > > >> http://www.pozo.com/kernel/kdump <http://www.pozo.com/kernel/kdump>.txt > > >> > > >> It???s not failing, I don???t think , I can do portupgrade and it works fine. > > >> I just get this new message > > > > > > I see what is going on, but it is somewhat strange that it happens. > > > > > > Do you run ruby in a jail with old (say, stable/10) libthr ? > > > Or do you have environment variable LIBPTHREAD_SPLITSTACK_MAIN set in > > > your environment ? > > > > > > Anyway, the rework of the stack grow indeed have incompatibility with the > > > old (pre-11) libthr, which tries to split main thread stack into smaller > > > stacks for the new threads. New stack grow code was specifically designed > > > to prevent this. Some hack would be needed there, to allow reuse of > > > the main stack gap. > > > > > > > Not running any jails > > all libraries are current as of today > > locate libthr. |xargs ls -l > > -r--r--r-- 1 root wheel 120240 Jun 24 12:50 /lib/libthr.so.3 > > -r--r--r-- 1 root wheel 256072 Jun 24 12:50 /usr/lib/libthr.a > > lrwxr-xr-x 1 root wheel 21 Jun 24 12:50 /usr/lib/libthr.so -> ../../lib/libthr.so.3 > > > > ldd /usr/local/bin/ruby > > -rwxr-xr-x 1 root wheel 2677552 Jun 24 18:22 /usr/local/lib/libruby23.so.2 > > -rwxr-xr-x 1 root wheel 43832 Jun 24 19:10 /usr/local/lib/libunwind.so.8.0.1 > > > > /usr/local/bin/ruby: > > libruby23.so.23 => /usr/local/lib/libruby23.so.23 (0x800a00000) > > libelf.so.2 => /lib/libelf.so.2 (0x800e9d000) > > libunwind.so.8 => /usr/local/lib/libunwind.so.8 (0x8010b5000) > > libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x8012ce000) > > libprocstat.so.1 => /usr/lib/libprocstat.so.1 (0x8014d1000) > > libcrypt.so.5 => /lib/libcrypt.so.5 (0x8016db000) > > libm.so.5 => /lib/libm.so.5 (0x8018f9000) > > libthr.so.3 => /lib/libthr.so.3 (0x801b26000) > > libc.so.7 => /lib/libc.so.7 (0x801d4e000) > > libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x802335000) > > liblzma.so.5 => /usr/lib/liblzma.so.5 (0x80254b000) > > libkvm.so.7 => /lib/libkvm.so.7 (0x802774000) > > libutil.so.9 => /lib/libutil.so.9 (0x802982000) > > > > no LIBPTHREAD_SPLITSTACK_MAIN set anywhere > > > > This is a fresh buildworld - installworld from noon today California time. > > > > here are env in make.conf: > > DEFAULT_VERSIONS= mysql=5.7 apache=2.4 python2=2.7 python3=3.4 ruby=2.3 perl5=5.24 php=5.6 tcltk=8.6 samba=4.4 ssl=base ncurses=base > > MALLOC_PRODUCTION=yes > > WITH_BDB_VERSION=5 > > > > env in src.conf: > > WITHOUT_DYNAMICROOT=yes > > WITHOUT_UNBOUND=yes > > WITHOUT_CASPER=yes > > WITHOUT_CAPSICUM=yes > > WITHOUT_DMAGENT=yes > > WITHOUT_PROFILE=yes > > WITHOUT_TESTS=yes > > WITHOUT_KERNEL_SYMBOLS=yes > > WITHOUT_DEBUG_FILES=1 > > WITHOUT_LIB32=yes > > INSTALL_NODEBUG=yes > > > > # Don't die on warnings > > NO_WERROR= > > WERROR= > > KERNCONF=pozo > > WITH_CCACHE_BUILD=yes > > Weird, I do not see where such behaviour could come from. I mean, > the behaviour of the libthr, assuming that the failing call is from > libthr and not from the ruby. > > Anyway, please try the following. Note that this might be not the final > fix, but I am interesting in seeing whether it fixed the issue for you. > I want to see the kdump output from ktrace _without_ -di, same as in > your first trace. > > diff --git a/lib/libc/sys/mprotect.2 b/lib/libc/sys/mprotect.2 > index 3be81c6582a..ca4d6307c44 100644 > --- a/lib/libc/sys/mprotect.2 > +++ b/lib/libc/sys/mprotect.2 > @@ -28,7 +28,7 @@ > .\" @(#)mprotect.2 8.1 (Berkeley) 6/9/93 > .\" $FreeBSD$ > .\" > -.Dd August 3, 2016 > +.Dd June 24, 2017 > .Dt MPROTECT 2 > .Os > .Sh NAME > @@ -80,10 +80,24 @@ and > arguments is not valid. > .It Bq Er EACCES > The calling process was not allowed to change > -the protection to the value specified by > -the > +the protection to the value specified by the > .Fa prot > argument. > +.It Bq Er ENOMEM > +The virtual address range specified by the > +.Fa addr > +and > +.Fa len > +arguments specify one or more pages which are not mapped. > +.It Bq Er ENOMEM > +The > +.Fa prot > +argument specifies > +.Dv PROT_WRITE > +on a > +.Dv MAP_PRIVATE > +mapping, and it would require more swap space than the system is able to > +supply for the private pages. > .El > .Sh SEE ALSO > .Xr madvise 2 , > diff --git a/sys/vm/vm_map.c b/sys/vm/vm_map.c > index 39cf303eaa0..b311b54d0f0 100644 > --- a/sys/vm/vm_map.c > +++ b/sys/vm/vm_map.c > @@ -1490,11 +1490,11 @@ vm_map_fixed(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > * If object is non-NULL, ref count must be bumped by caller > * prior to making call to account for the new entry. > */ > -int > -vm_map_find(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > - vm_offset_t *addr, /* IN/OUT */ > - vm_size_t length, vm_offset_t max_addr, int find_space, > - vm_prot_t prot, vm_prot_t max, int cow) > +static int > +vm_map_find1(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > + vm_offset_t *addr, /* IN/OUT */ > + vm_size_t length, vm_offset_t max_addr, int find_space, vm_prot_t prot, > + vm_prot_t max, int cow) > { > vm_offset_t alignment, initial_addr, start; > int result; > @@ -1556,6 +1556,26 @@ again: > return (result); > } > > +int > +vm_map_find(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > + vm_offset_t *addr, /* IN/OUT */ > + vm_size_t length, vm_offset_t max_addr, int find_space, vm_prot_t prot, > + vm_prot_t max, int cow) > +{ > + int rv; > + bool hinted; > + > + hinted = *addr != 0; > + for (;;) { > + rv = vm_map_find1(map, object, offset, addr, length, max_addr, > + find_space, prot, max, cow); > + if (rv != KERN_NO_SPACE || !hinted) > + return (rv); > + *addr = 0; > + hinted = false; > + } > +} > + > /* > * vm_map_simplify_entry: > * > @@ -1970,7 +1990,7 @@ vm_map_pmap_enter(vm_map_t map, vm_offset_t addr, vm_prot_t prot, > */ > int > vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > - vm_prot_t new_prot, boolean_t set_max) > + vm_prot_t new_prot, boolean_t set_max) > { > vm_map_entry_t current, entry; > vm_object_t obj; > @@ -1995,7 +2015,8 @@ vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > if (vm_map_lookup_entry(map, start, &entry)) { > vm_map_clip_start(map, entry, start); > } else { > - entry = entry->next; > + vm_map_unlock(map); > + return (KERN_INVALID_ADDRESS); > } > > /* > @@ -2003,6 +2024,8 @@ vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > */ > for (current = entry; current != &map->header && current->start < end; > current = current->next) { > + if ((current->eflags & MAP_ENTRY_GUARD) != 0) > + continue; > if (current->eflags & MAP_ENTRY_IS_SUB_MAP) { > vm_map_unlock(map); > return (KERN_INVALID_ARGUMENT); > @@ -2011,6 +2034,11 @@ vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > vm_map_unlock(map); > return (KERN_PROTECTION_FAILURE); > } > + if (current->end < end && (current->next == &map->header || > + current->next->start > current->end)) { > + vm_map_unlock(map); > + return (KERN_INVALID_ADDRESS); > + } > } > > /* > @@ -2712,9 +2740,9 @@ vm_map_wire(vm_map_t map, vm_offset_t start, vm_offset_t end, > * If VM_MAP_WIRE_HOLESOK was specified, skip this check. > */ > next_entry: > - if (((flags & VM_MAP_WIRE_HOLESOK) == 0) && > - (entry->end < end && (entry->next == &map->header || > - entry->next->start > entry->end))) { > + if ((flags & VM_MAP_WIRE_HOLESOK) == 0 && > + entry->end < end && (entry->next == &map->header || > + entry->next->start > entry->end)) { > end = entry->end; > rv = KERN_INVALID_ADDRESS; > goto done; > diff --git a/sys/vm/vm_mmap.c b/sys/vm/vm_mmap.c > index 4d8f6ad9ed7..09912250d52 100644 > --- a/sys/vm/vm_mmap.c > +++ b/sys/vm/vm_mmap.c > @@ -607,6 +607,7 @@ kern_mprotect(struct thread *td, uintptr_t addr0, size_t size, int prot) > case KERN_PROTECTION_FAILURE: > return (EACCES); > case KERN_RESOURCE_SHORTAGE: > + case KERN_INVALID_ADDRESS: > return (ENOMEM); > } > return (EINVAL); I'm seeing similar fallout on r320327, built from scratch while running r320293: Script started on Sun Jun 25 17:03:41 2017 # uname -aKU FreeBSD 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r320327: Sun Jun 25 16:35:02 CEST 2017 root@freebsd-head-uefi-zfs.bsd.net:/usr/obj/usr/src/sys/VBOX amd64 1200035 1200035 # /sbin/zfs mount -va Fatal error 'Cannot allocate red zone for initial thread' at line 392 in file /usr/src/lib/libthr/thread/thr_init.c (errno = 12) Abort trap (core dumped) # ls -l /lib/libthr* -r--r--r-- 1 root wheel 153336 Jun 25 16:39 /lib/libthr.so.3 # ls -l /usr/lib/libthr* -r--r--r-- 1 root wheel 256064 May 28 21:41 /usr/lib/libthr.a lrwxr-xr-x 1 root wheel 21 Jun 25 16:39 /usr/lib/libthr.so -> ../../lib/libthr.so.3 -r--r--r-- 1 root wheel 270362 May 28 21:41 /usr/lib/libthr_p.a -r--r--r-- 1 root wheel 45376 Apr 22 10:39 /usr/lib/libthread_db.a lrwxr-xr-x 1 root wheel 17 Jun 25 16:39 /usr/lib/libthread_db.so -> libthread_db.so.3 -r--r--r-- 1 root wheel 38776 Jun 25 16:39 /usr/lib/libthread_db.so.3 -r--r--r-- 1 root wheel 49496 Apr 22 10:39 /usr/lib/libthread_db_p.a # ldd /sbin/zfs /sbin/zfs: libjail.so.1 => /lib/libjail.so.1 (0x800247000) libnvpair.so.2 => /lib/libnvpair.so.2 (0x80024f000) libuutil.so.2 => /lib/libuutil.so.2 (0x800269000) libzfs_core.so.2 => /lib/libzfs_core.so.2 (0x800276000) libzfs.so.3 => /lib/libzfs.so.3 (0x800281000) libc.so.7 => /lib/libc.so.7 (0x8002cd000) libmd.so.6 => /lib/libmd.so.6 (0x8006d1000) libumem.so.2 => /lib/libumem.so.2 (0x8006eb000) libutil.so.9 => /lib/libutil.so.9 (0x8006f0000) libm.so.5 => /lib/libm.so.5 (0x800706000) libavl.so.2 => /lib/libavl.so.2 (0x800736000) libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x80073a000) libgeom.so.5 => /lib/libgeom.so.5 (0x800770000) libz.so.6 => /lib/libz.so.6 (0x800779000) libthr.so.3 => /lib/libthr.so.3 (0x800793000) libsbuf.so.6 => /lib/libsbuf.so.6 (0x8007bd000) # ls -l /sbin/zfs -r-xr-xr-x 1 root wheel 124296 Jun 25 16:39 /sbin/zfs # exit Script done on Sun Jun 25 17:04:32 2017 Although not relevant for this case, neither /usr/lib/libthr.a, nor /usr/lib/libthr_p.a, nor /usr/lib/libthread_db.a, nor /usr/lib/libthread_db_p.a has been updated by installworld, nor removed by delete-old-libs, in a very long time. Anything else I can contribute? I'll try your patch and see if I can get any further. -- +-------------------------------+------------------------------------+ | Vennlig hilsen, | Best regards, | | Trond Endrestøl, | Trond Endrestøl, | | IT-ansvarlig, | System administrator, | | Fagskolen Innlandet, | Gjøvik Technical College, Norway, | | tlf. mob. 952 62 567, | Cellular...: +47 952 62 567, | | sentralbord 61 14 54 00. | Switchboard: +47 61 14 54 00. | +-------------------------------+------------------------------------+ From owner-freebsd-current@freebsd.org Sun Jun 25 15:56:18 2017 Return-Path: <owner-freebsd-current@freebsd.org> Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D5A59D947D4 for <freebsd-current@mailman.ysv.freebsd.org>; Sun, 25 Jun 2017 15:56:18 +0000 (UTC) (envelope-from trond@fagskolen.gjovik.no) Received: from smtp.fagskolen.gjovik.no (smtp.fagskolen.gjovik.no [IPv6:2001:700:1100:1:200:ff:fe00:b]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.fagskolen.gjovik.no", Issuer "Fagskolen i Gj??vik" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 798D274553 for <freebsd-current@freebsd.org>; Sun, 25 Jun 2017 15:56:18 +0000 (UTC) (envelope-from trond@fagskolen.gjovik.no) Received: from mail.fig.ol.no (localhost [127.0.0.1]) by mail.fig.ol.no (8.15.2/8.15.2) with ESMTPS id v5PFu95h001449 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for <freebsd-current@freebsd.org>; Sun, 25 Jun 2017 17:56:09 +0200 (CEST) (envelope-from trond@fagskolen.gjovik.no) Received: from localhost (trond@localhost) by mail.fig.ol.no (8.15.2/8.15.2/Submit) with ESMTP id v5PFu9Fx001446 for <freebsd-current@freebsd.org>; Sun, 25 Jun 2017 17:56:09 +0200 (CEST) (envelope-from trond@fagskolen.gjovik.no) X-Authentication-Warning: mail.fig.ol.no: trond owned process doing -bs Date: Sun, 25 Jun 2017 17:56:09 +0200 (CEST) From: =?ISO-8859-1?Q?Trond_Endrest=F8l?= <Trond.Endrestol@fagskolen.gjovik.no> Sender: Trond.Endrestol@fagskolen.gjovik.no To: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: Current amd64 new error or warning from today's current with ruby r320323 In-Reply-To: <alpine.BSF.2.21.1706251709340.37790@mail.fig.ol.no> Message-ID: <alpine.BSF.2.21.1706251747180.37790@mail.fig.ol.no> References: <44028FBF-C8B1-4A69-B798-F4E28CFCD779@pozo.com> <20170625012359.GS3437@kib.kiev.ua> <040BF7D1-2FDF-415B-9A17-ADD608503F14@pozo.com> <20170625020441.GT3437@kib.kiev.ua> <219E6B59-EF90-443D-8B53-4C39C4D6CFBD@pozo.com> <20170625134133.GA3437@kib.kiev.ua> <alpine.BSF.2.21.1706251709340.37790@mail.fig.ol.no> User-Agent: Alpine 2.21 (BSF 202 2017-01-01) Organization: Fagskolen Innlandet OpenPGP: url=http://fig.ol.no/~trond/trond.key MIME-Version: 1.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail.fig.ol.no Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current <freebsd-current.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>; List-Post: <mailto:freebsd-current@freebsd.org> List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=subscribe> X-List-Received-Date: Sun, 25 Jun 2017 15:56:18 -0000 On Sun, 25 Jun 2017 17:18+0200, Trond Endrestøl wrote: > On Sun, 25 Jun 2017 16:41+0300, Konstantin Belousov wrote: > > > On Sat, Jun 24, 2017 at 07:19:25PM -0700, Manfred Antar wrote: > > > > > > > On Jun 24, 2017, at 7:04 PM, Konstantin Belousov <kostikbel@gmail.com> wrote: > > > > > > > > On Sat, Jun 24, 2017 at 06:48:03PM -0700, Manfred Antar wrote: > > > >> > > > >>> On Jun 24, 2017, at 6:23 PM, Konstantin Belousov <kostikbel@gmail.com> wrote: > > > >>> > > > >>> On Sat, Jun 24, 2017 at 06:08:50PM -0700, Manfred Antar wrote: > > > >>>> New world and kernel r320323 > > > >>>> I get a new error or message when using ruby: > > > >>>> > > > >>>> > > > >>>> /usr/local/sbin/portupgrade -av > > > >>>> <main>: warning: pthread_create failed for timer: Resource temporarily unavailable, scheduling broken > > > >>>> > > > >>>> everything works just this message when using ruby. I recompiled ruby , still same message > > > >>>> > > > >>>> /usr/local/bin/ruby -v > > > >>>> <main>: warning: pthread_create failed for timer: Resource temporarily unavailable, scheduling broken > > > >>>> ruby 2.3.4p301 (2017-03-30 revision 58214) [amd64-freebsd12] > > > >>>> > > > >>>> Not sure what???s changed, I noticed some commits to vm stuff, maybe thats it. > > > >>> > > > >>> ktrace your failing ruby invocation, then post output of kdump -H somewhere. > > > >>> > > > >> > > > >> Ok not sure if this is right , but this is what i did: > > > >> > > > >> (tmp)4637}ktrace /usr/local/bin/ruby -v > > > >> <main>: warning: pthread_create failed for timer: Resource temporarily unavailable, scheduling broken > > > >> ruby 2.3.4p301 (2017-03-30 revision 58214) [amd64-freebsd12] > > > >> > > > >> (tmp)4638}kdump -H -f ./ktrace.out > kdump.txt > > > >> > > > >> you can get kdump.txt at: > > > >> > > > >> http://www.pozo.com/kernel/kdump <http://www.pozo.com/kernel/kdump>.txt > > > >> > > > >> It???s not failing, I don???t think , I can do portupgrade and it works fine. > > > >> I just get this new message > > > > > > > > I see what is going on, but it is somewhat strange that it happens. > > > > > > > > Do you run ruby in a jail with old (say, stable/10) libthr ? > > > > Or do you have environment variable LIBPTHREAD_SPLITSTACK_MAIN set in > > > > your environment ? > > > > > > > > Anyway, the rework of the stack grow indeed have incompatibility with the > > > > old (pre-11) libthr, which tries to split main thread stack into smaller > > > > stacks for the new threads. New stack grow code was specifically designed > > > > to prevent this. Some hack would be needed there, to allow reuse of > > > > the main stack gap. > > > > > > > > > > Not running any jails > > > all libraries are current as of today > > > locate libthr. |xargs ls -l > > > -r--r--r-- 1 root wheel 120240 Jun 24 12:50 /lib/libthr.so.3 > > > -r--r--r-- 1 root wheel 256072 Jun 24 12:50 /usr/lib/libthr.a > > > lrwxr-xr-x 1 root wheel 21 Jun 24 12:50 /usr/lib/libthr.so -> ../../lib/libthr.so.3 > > > > > > ldd /usr/local/bin/ruby > > > -rwxr-xr-x 1 root wheel 2677552 Jun 24 18:22 /usr/local/lib/libruby23.so.2 > > > -rwxr-xr-x 1 root wheel 43832 Jun 24 19:10 /usr/local/lib/libunwind.so.8.0.1 > > > > > > /usr/local/bin/ruby: > > > libruby23.so.23 => /usr/local/lib/libruby23.so.23 (0x800a00000) > > > libelf.so.2 => /lib/libelf.so.2 (0x800e9d000) > > > libunwind.so.8 => /usr/local/lib/libunwind.so.8 (0x8010b5000) > > > libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x8012ce000) > > > libprocstat.so.1 => /usr/lib/libprocstat.so.1 (0x8014d1000) > > > libcrypt.so.5 => /lib/libcrypt.so.5 (0x8016db000) > > > libm.so.5 => /lib/libm.so.5 (0x8018f9000) > > > libthr.so.3 => /lib/libthr.so.3 (0x801b26000) > > > libc.so.7 => /lib/libc.so.7 (0x801d4e000) > > > libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x802335000) > > > liblzma.so.5 => /usr/lib/liblzma.so.5 (0x80254b000) > > > libkvm.so.7 => /lib/libkvm.so.7 (0x802774000) > > > libutil.so.9 => /lib/libutil.so.9 (0x802982000) > > > > > > no LIBPTHREAD_SPLITSTACK_MAIN set anywhere > > > > > > This is a fresh buildworld - installworld from noon today California time. > > > > > > here are env in make.conf: > > > DEFAULT_VERSIONS= mysql=5.7 apache=2.4 python2=2.7 python3=3.4 ruby=2.3 perl5=5.24 php=5.6 tcltk=8.6 samba=4.4 ssl=base ncurses=base > > > MALLOC_PRODUCTION=yes > > > WITH_BDB_VERSION=5 > > > > > > env in src.conf: > > > WITHOUT_DYNAMICROOT=yes > > > WITHOUT_UNBOUND=yes > > > WITHOUT_CASPER=yes > > > WITHOUT_CAPSICUM=yes > > > WITHOUT_DMAGENT=yes > > > WITHOUT_PROFILE=yes > > > WITHOUT_TESTS=yes > > > WITHOUT_KERNEL_SYMBOLS=yes > > > WITHOUT_DEBUG_FILES=1 > > > WITHOUT_LIB32=yes > > > INSTALL_NODEBUG=yes > > > > > > # Don't die on warnings > > > NO_WERROR= > > > WERROR= > > > KERNCONF=pozo > > > WITH_CCACHE_BUILD=yes > > > > Weird, I do not see where such behaviour could come from. I mean, > > the behaviour of the libthr, assuming that the failing call is from > > libthr and not from the ruby. > > > > Anyway, please try the following. Note that this might be not the final > > fix, but I am interesting in seeing whether it fixed the issue for you. > > I want to see the kdump output from ktrace _without_ -di, same as in > > your first trace. > > > > diff --git a/lib/libc/sys/mprotect.2 b/lib/libc/sys/mprotect.2 > > index 3be81c6582a..ca4d6307c44 100644 > > --- a/lib/libc/sys/mprotect.2 > > +++ b/lib/libc/sys/mprotect.2 > > @@ -28,7 +28,7 @@ > > .\" @(#)mprotect.2 8.1 (Berkeley) 6/9/93 > > .\" $FreeBSD$ > > .\" > > -.Dd August 3, 2016 > > +.Dd June 24, 2017 > > .Dt MPROTECT 2 > > .Os > > .Sh NAME > > @@ -80,10 +80,24 @@ and > > arguments is not valid. > > .It Bq Er EACCES > > The calling process was not allowed to change > > -the protection to the value specified by > > -the > > +the protection to the value specified by the > > .Fa prot > > argument. > > +.It Bq Er ENOMEM > > +The virtual address range specified by the > > +.Fa addr > > +and > > +.Fa len > > +arguments specify one or more pages which are not mapped. > > +.It Bq Er ENOMEM > > +The > > +.Fa prot > > +argument specifies > > +.Dv PROT_WRITE > > +on a > > +.Dv MAP_PRIVATE > > +mapping, and it would require more swap space than the system is able to > > +supply for the private pages. > > .El > > .Sh SEE ALSO > > .Xr madvise 2 , > > diff --git a/sys/vm/vm_map.c b/sys/vm/vm_map.c > > index 39cf303eaa0..b311b54d0f0 100644 > > --- a/sys/vm/vm_map.c > > +++ b/sys/vm/vm_map.c > > @@ -1490,11 +1490,11 @@ vm_map_fixed(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > > * If object is non-NULL, ref count must be bumped by caller > > * prior to making call to account for the new entry. > > */ > > -int > > -vm_map_find(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > > - vm_offset_t *addr, /* IN/OUT */ > > - vm_size_t length, vm_offset_t max_addr, int find_space, > > - vm_prot_t prot, vm_prot_t max, int cow) > > +static int > > +vm_map_find1(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > > + vm_offset_t *addr, /* IN/OUT */ > > + vm_size_t length, vm_offset_t max_addr, int find_space, vm_prot_t prot, > > + vm_prot_t max, int cow) > > { > > vm_offset_t alignment, initial_addr, start; > > int result; > > @@ -1556,6 +1556,26 @@ again: > > return (result); > > } > > > > +int > > +vm_map_find(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > > + vm_offset_t *addr, /* IN/OUT */ > > + vm_size_t length, vm_offset_t max_addr, int find_space, vm_prot_t prot, > > + vm_prot_t max, int cow) > > +{ > > + int rv; > > + bool hinted; > > + > > + hinted = *addr != 0; > > + for (;;) { > > + rv = vm_map_find1(map, object, offset, addr, length, max_addr, > > + find_space, prot, max, cow); > > + if (rv != KERN_NO_SPACE || !hinted) > > + return (rv); > > + *addr = 0; > > + hinted = false; > > + } > > +} > > + > > /* > > * vm_map_simplify_entry: > > * > > @@ -1970,7 +1990,7 @@ vm_map_pmap_enter(vm_map_t map, vm_offset_t addr, vm_prot_t prot, > > */ > > int > > vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > > - vm_prot_t new_prot, boolean_t set_max) > > + vm_prot_t new_prot, boolean_t set_max) > > { > > vm_map_entry_t current, entry; > > vm_object_t obj; > > @@ -1995,7 +2015,8 @@ vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > > if (vm_map_lookup_entry(map, start, &entry)) { > > vm_map_clip_start(map, entry, start); > > } else { > > - entry = entry->next; > > + vm_map_unlock(map); > > + return (KERN_INVALID_ADDRESS); > > } > > > > /* > > @@ -2003,6 +2024,8 @@ vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > > */ > > for (current = entry; current != &map->header && current->start < end; > > current = current->next) { > > + if ((current->eflags & MAP_ENTRY_GUARD) != 0) > > + continue; > > if (current->eflags & MAP_ENTRY_IS_SUB_MAP) { > > vm_map_unlock(map); > > return (KERN_INVALID_ARGUMENT); > > @@ -2011,6 +2034,11 @@ vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > > vm_map_unlock(map); > > return (KERN_PROTECTION_FAILURE); > > } > > + if (current->end < end && (current->next == &map->header || > > + current->next->start > current->end)) { > > + vm_map_unlock(map); > > + return (KERN_INVALID_ADDRESS); > > + } > > } > > > > /* > > @@ -2712,9 +2740,9 @@ vm_map_wire(vm_map_t map, vm_offset_t start, vm_offset_t end, > > * If VM_MAP_WIRE_HOLESOK was specified, skip this check. > > */ > > next_entry: > > - if (((flags & VM_MAP_WIRE_HOLESOK) == 0) && > > - (entry->end < end && (entry->next == &map->header || > > - entry->next->start > entry->end))) { > > + if ((flags & VM_MAP_WIRE_HOLESOK) == 0 && > > + entry->end < end && (entry->next == &map->header || > > + entry->next->start > entry->end)) { > > end = entry->end; > > rv = KERN_INVALID_ADDRESS; > > goto done; > > diff --git a/sys/vm/vm_mmap.c b/sys/vm/vm_mmap.c > > index 4d8f6ad9ed7..09912250d52 100644 > > --- a/sys/vm/vm_mmap.c > > +++ b/sys/vm/vm_mmap.c > > @@ -607,6 +607,7 @@ kern_mprotect(struct thread *td, uintptr_t addr0, size_t size, int prot) > > case KERN_PROTECTION_FAILURE: > > return (EACCES); > > case KERN_RESOURCE_SHORTAGE: > > + case KERN_INVALID_ADDRESS: > > return (ENOMEM); > > } > > return (EINVAL); > > I'm seeing similar fallout on r320327, built from scratch while > running r320293: > > Script started on Sun Jun 25 17:03:41 2017 > # uname -aKU > FreeBSD 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r320327: Sun Jun 25 16:35:02 CEST 2017 root@freebsd-head-uefi-zfs.bsd.net:/usr/obj/usr/src/sys/VBOX amd64 1200035 1200035 > # /sbin/zfs mount -va > Fatal error 'Cannot allocate red zone for initial thread' at line 392 in file /usr/src/lib/libthr/thread/thr_init.c (errno = 12) > Abort trap (core dumped) > # ls -l /lib/libthr* > -r--r--r-- 1 root wheel 153336 Jun 25 16:39 /lib/libthr.so.3 > # ls -l /usr/lib/libthr* > -r--r--r-- 1 root wheel 256064 May 28 21:41 /usr/lib/libthr.a > lrwxr-xr-x 1 root wheel 21 Jun 25 16:39 /usr/lib/libthr.so -> ../../lib/libthr.so.3 > -r--r--r-- 1 root wheel 270362 May 28 21:41 /usr/lib/libthr_p.a > -r--r--r-- 1 root wheel 45376 Apr 22 10:39 /usr/lib/libthread_db.a > lrwxr-xr-x 1 root wheel 17 Jun 25 16:39 /usr/lib/libthread_db.so -> libthread_db.so.3 > -r--r--r-- 1 root wheel 38776 Jun 25 16:39 /usr/lib/libthread_db.so.3 > -r--r--r-- 1 root wheel 49496 Apr 22 10:39 /usr/lib/libthread_db_p.a > # ldd /sbin/zfs > /sbin/zfs: > libjail.so.1 => /lib/libjail.so.1 (0x800247000) > libnvpair.so.2 => /lib/libnvpair.so.2 (0x80024f000) > libuutil.so.2 => /lib/libuutil.so.2 (0x800269000) > libzfs_core.so.2 => /lib/libzfs_core.so.2 (0x800276000) > libzfs.so.3 => /lib/libzfs.so.3 (0x800281000) > libc.so.7 => /lib/libc.so.7 (0x8002cd000) > libmd.so.6 => /lib/libmd.so.6 (0x8006d1000) > libumem.so.2 => /lib/libumem.so.2 (0x8006eb000) > libutil.so.9 => /lib/libutil.so.9 (0x8006f0000) > libm.so.5 => /lib/libm.so.5 (0x800706000) > libavl.so.2 => /lib/libavl.so.2 (0x800736000) > libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x80073a000) > libgeom.so.5 => /lib/libgeom.so.5 (0x800770000) > libz.so.6 => /lib/libz.so.6 (0x800779000) > libthr.so.3 => /lib/libthr.so.3 (0x800793000) > libsbuf.so.6 => /lib/libsbuf.so.6 (0x8007bd000) > # ls -l /sbin/zfs > -r-xr-xr-x 1 root wheel 124296 Jun 25 16:39 /sbin/zfs > # exit > > Script done on Sun Jun 25 17:04:32 2017 > > Although not relevant for this case, neither /usr/lib/libthr.a, nor > /usr/lib/libthr_p.a, nor /usr/lib/libthread_db.a, nor > /usr/lib/libthread_db_p.a has been updated by installworld, nor > removed by delete-old-libs, in a very long time. > > Anything else I can contribute? I'll try your patch and see if I can > get any further. Partial success! Your patch brought me past the libthr issue with /sbin/zfs. Like the OP, portupgrade (and ruby22) complains about unable to create a timer thread. I shall attempt to rebuild lang/ruby22 and emulators/virtualbox-ose-additions-nox11. I think I'll switch to ruby23 while I'm at it. -- +-------------------------------+------------------------------------+ | Vennlig hilsen, | Best regards, | | Trond Endrestøl, | Trond Endrestøl, | | IT-ansvarlig, | System administrator, | | Fagskolen Innlandet, | Gjøvik Technical College, Norway, | | tlf. mob. 952 62 567, | Cellular...: +47 952 62 567, | | sentralbord 61 14 54 00. | Switchboard: +47 61 14 54 00. | +-------------------------------+------------------------------------+ From owner-freebsd-current@freebsd.org Sun Jun 25 16:09:08 2017 Return-Path: <owner-freebsd-current@freebsd.org> Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E92F3D94B6F for <freebsd-current@mailman.ysv.freebsd.org>; Sun, 25 Jun 2017 16:09:08 +0000 (UTC) (envelope-from trond@fagskolen.gjovik.no) Received: from smtp.fagskolen.gjovik.no (smtp.fagskolen.gjovik.no [IPv6:2001:700:1100:1:200:ff:fe00:b]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.fagskolen.gjovik.no", Issuer "Fagskolen i Gj??vik" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8761B74BFB for <freebsd-current@freebsd.org>; Sun, 25 Jun 2017 16:09:08 +0000 (UTC) (envelope-from trond@fagskolen.gjovik.no) Received: from mail.fig.ol.no (localhost [127.0.0.1]) by mail.fig.ol.no (8.15.2/8.15.2) with ESMTPS id v5PG8xTZ001512 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for <freebsd-current@freebsd.org>; Sun, 25 Jun 2017 18:08:59 +0200 (CEST) (envelope-from trond@fagskolen.gjovik.no) Received: from localhost (trond@localhost) by mail.fig.ol.no (8.15.2/8.15.2/Submit) with ESMTP id v5PG8xc2001509 for <freebsd-current@freebsd.org>; Sun, 25 Jun 2017 18:08:59 +0200 (CEST) (envelope-from trond@fagskolen.gjovik.no) X-Authentication-Warning: mail.fig.ol.no: trond owned process doing -bs Date: Sun, 25 Jun 2017 18:08:59 +0200 (CEST) From: =?ISO-8859-1?Q?Trond_Endrest=F8l?= <Trond.Endrestol@fagskolen.gjovik.no> Sender: Trond.Endrestol@fagskolen.gjovik.no To: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: Current amd64 new error or warning from today's current with ruby r320323 In-Reply-To: <alpine.BSF.2.21.1706251747180.37790@mail.fig.ol.no> Message-ID: <alpine.BSF.2.21.1706251807230.37790@mail.fig.ol.no> References: <44028FBF-C8B1-4A69-B798-F4E28CFCD779@pozo.com> <20170625012359.GS3437@kib.kiev.ua> <040BF7D1-2FDF-415B-9A17-ADD608503F14@pozo.com> <20170625020441.GT3437@kib.kiev.ua> <219E6B59-EF90-443D-8B53-4C39C4D6CFBD@pozo.com> <20170625134133.GA3437@kib.kiev.ua> <alpine.BSF.2.21.1706251709340.37790@mail.fig.ol.no> <alpine.BSF.2.21.1706251747180.37790@mail.fig.ol.no> User-Agent: Alpine 2.21 (BSF 202 2017-01-01) Organization: Fagskolen Innlandet OpenPGP: url=http://fig.ol.no/~trond/trond.key MIME-Version: 1.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail.fig.ol.no Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current <freebsd-current.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>; List-Post: <mailto:freebsd-current@freebsd.org> List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=subscribe> X-List-Received-Date: Sun, 25 Jun 2017 16:09:09 -0000 On Sun, 25 Jun 2017 17:56+0200, Trond Endrestøl wrote: > On Sun, 25 Jun 2017 17:18+0200, Trond Endrestøl wrote: > > > On Sun, 25 Jun 2017 16:41+0300, Konstantin Belousov wrote: > > > > > On Sat, Jun 24, 2017 at 07:19:25PM -0700, Manfred Antar wrote: > > > > > > > > > On Jun 24, 2017, at 7:04 PM, Konstantin Belousov <kostikbel@gmail.com> wrote: > > > > > > > > > > On Sat, Jun 24, 2017 at 06:48:03PM -0700, Manfred Antar wrote: > > > > >> > > > > >>> On Jun 24, 2017, at 6:23 PM, Konstantin Belousov <kostikbel@gmail.com> wrote: > > > > >>> > > > > >>> On Sat, Jun 24, 2017 at 06:08:50PM -0700, Manfred Antar wrote: > > > > >>>> New world and kernel r320323 > > > > >>>> I get a new error or message when using ruby: > > > > >>>> > > > > >>>> > > > > >>>> /usr/local/sbin/portupgrade -av > > > > >>>> <main>: warning: pthread_create failed for timer: Resource temporarily unavailable, scheduling broken > > > > >>>> > > > > >>>> everything works just this message when using ruby. I recompiled ruby , still same message > > > > >>>> > > > > >>>> /usr/local/bin/ruby -v > > > > >>>> <main>: warning: pthread_create failed for timer: Resource temporarily unavailable, scheduling broken > > > > >>>> ruby 2.3.4p301 (2017-03-30 revision 58214) [amd64-freebsd12] > > > > >>>> > > > > >>>> Not sure what???s changed, I noticed some commits to vm stuff, maybe thats it. > > > > >>> > > > > >>> ktrace your failing ruby invocation, then post output of kdump -H somewhere. > > > > >>> > > > > >> > > > > >> Ok not sure if this is right , but this is what i did: > > > > >> > > > > >> (tmp)4637}ktrace /usr/local/bin/ruby -v > > > > >> <main>: warning: pthread_create failed for timer: Resource temporarily unavailable, scheduling broken > > > > >> ruby 2.3.4p301 (2017-03-30 revision 58214) [amd64-freebsd12] > > > > >> > > > > >> (tmp)4638}kdump -H -f ./ktrace.out > kdump.txt > > > > >> > > > > >> you can get kdump.txt at: > > > > >> > > > > >> http://www.pozo.com/kernel/kdump <http://www.pozo.com/kernel/kdump>.txt > > > > >> > > > > >> It???s not failing, I don???t think , I can do portupgrade and it works fine. > > > > >> I just get this new message > > > > > > > > > > I see what is going on, but it is somewhat strange that it happens. > > > > > > > > > > Do you run ruby in a jail with old (say, stable/10) libthr ? > > > > > Or do you have environment variable LIBPTHREAD_SPLITSTACK_MAIN set in > > > > > your environment ? > > > > > > > > > > Anyway, the rework of the stack grow indeed have incompatibility with the > > > > > old (pre-11) libthr, which tries to split main thread stack into smaller > > > > > stacks for the new threads. New stack grow code was specifically designed > > > > > to prevent this. Some hack would be needed there, to allow reuse of > > > > > the main stack gap. > > > > > > > > > > > > > Not running any jails > > > > all libraries are current as of today > > > > locate libthr. |xargs ls -l > > > > -r--r--r-- 1 root wheel 120240 Jun 24 12:50 /lib/libthr.so.3 > > > > -r--r--r-- 1 root wheel 256072 Jun 24 12:50 /usr/lib/libthr.a > > > > lrwxr-xr-x 1 root wheel 21 Jun 24 12:50 /usr/lib/libthr.so -> ../../lib/libthr.so.3 > > > > > > > > ldd /usr/local/bin/ruby > > > > -rwxr-xr-x 1 root wheel 2677552 Jun 24 18:22 /usr/local/lib/libruby23.so.2 > > > > -rwxr-xr-x 1 root wheel 43832 Jun 24 19:10 /usr/local/lib/libunwind.so.8.0.1 > > > > > > > > /usr/local/bin/ruby: > > > > libruby23.so.23 => /usr/local/lib/libruby23.so.23 (0x800a00000) > > > > libelf.so.2 => /lib/libelf.so.2 (0x800e9d000) > > > > libunwind.so.8 => /usr/local/lib/libunwind.so.8 (0x8010b5000) > > > > libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x8012ce000) > > > > libprocstat.so.1 => /usr/lib/libprocstat.so.1 (0x8014d1000) > > > > libcrypt.so.5 => /lib/libcrypt.so.5 (0x8016db000) > > > > libm.so.5 => /lib/libm.so.5 (0x8018f9000) > > > > libthr.so.3 => /lib/libthr.so.3 (0x801b26000) > > > > libc.so.7 => /lib/libc.so.7 (0x801d4e000) > > > > libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x802335000) > > > > liblzma.so.5 => /usr/lib/liblzma.so.5 (0x80254b000) > > > > libkvm.so.7 => /lib/libkvm.so.7 (0x802774000) > > > > libutil.so.9 => /lib/libutil.so.9 (0x802982000) > > > > > > > > no LIBPTHREAD_SPLITSTACK_MAIN set anywhere > > > > > > > > This is a fresh buildworld - installworld from noon today California time. > > > > > > > > here are env in make.conf: > > > > DEFAULT_VERSIONS= mysql=5.7 apache=2.4 python2=2.7 python3=3.4 ruby=2.3 perl5=5.24 php=5.6 tcltk=8.6 samba=4.4 ssl=base ncurses=base > > > > MALLOC_PRODUCTION=yes > > > > WITH_BDB_VERSION=5 > > > > > > > > env in src.conf: > > > > WITHOUT_DYNAMICROOT=yes > > > > WITHOUT_UNBOUND=yes > > > > WITHOUT_CASPER=yes > > > > WITHOUT_CAPSICUM=yes > > > > WITHOUT_DMAGENT=yes > > > > WITHOUT_PROFILE=yes > > > > WITHOUT_TESTS=yes > > > > WITHOUT_KERNEL_SYMBOLS=yes > > > > WITHOUT_DEBUG_FILES=1 > > > > WITHOUT_LIB32=yes > > > > INSTALL_NODEBUG=yes > > > > > > > > # Don't die on warnings > > > > NO_WERROR= > > > > WERROR= > > > > KERNCONF=pozo > > > > WITH_CCACHE_BUILD=yes > > > > > > Weird, I do not see where such behaviour could come from. I mean, > > > the behaviour of the libthr, assuming that the failing call is from > > > libthr and not from the ruby. > > > > > > Anyway, please try the following. Note that this might be not the final > > > fix, but I am interesting in seeing whether it fixed the issue for you. > > > I want to see the kdump output from ktrace _without_ -di, same as in > > > your first trace. > > > > > > diff --git a/lib/libc/sys/mprotect.2 b/lib/libc/sys/mprotect.2 > > > index 3be81c6582a..ca4d6307c44 100644 > > > --- a/lib/libc/sys/mprotect.2 > > > +++ b/lib/libc/sys/mprotect.2 > > > @@ -28,7 +28,7 @@ > > > .\" @(#)mprotect.2 8.1 (Berkeley) 6/9/93 > > > .\" $FreeBSD$ > > > .\" > > > -.Dd August 3, 2016 > > > +.Dd June 24, 2017 > > > .Dt MPROTECT 2 > > > .Os > > > .Sh NAME > > > @@ -80,10 +80,24 @@ and > > > arguments is not valid. > > > .It Bq Er EACCES > > > The calling process was not allowed to change > > > -the protection to the value specified by > > > -the > > > +the protection to the value specified by the > > > .Fa prot > > > argument. > > > +.It Bq Er ENOMEM > > > +The virtual address range specified by the > > > +.Fa addr > > > +and > > > +.Fa len > > > +arguments specify one or more pages which are not mapped. > > > +.It Bq Er ENOMEM > > > +The > > > +.Fa prot > > > +argument specifies > > > +.Dv PROT_WRITE > > > +on a > > > +.Dv MAP_PRIVATE > > > +mapping, and it would require more swap space than the system is able to > > > +supply for the private pages. > > > .El > > > .Sh SEE ALSO > > > .Xr madvise 2 , > > > diff --git a/sys/vm/vm_map.c b/sys/vm/vm_map.c > > > index 39cf303eaa0..b311b54d0f0 100644 > > > --- a/sys/vm/vm_map.c > > > +++ b/sys/vm/vm_map.c > > > @@ -1490,11 +1490,11 @@ vm_map_fixed(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > > > * If object is non-NULL, ref count must be bumped by caller > > > * prior to making call to account for the new entry. > > > */ > > > -int > > > -vm_map_find(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > > > - vm_offset_t *addr, /* IN/OUT */ > > > - vm_size_t length, vm_offset_t max_addr, int find_space, > > > - vm_prot_t prot, vm_prot_t max, int cow) > > > +static int > > > +vm_map_find1(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > > > + vm_offset_t *addr, /* IN/OUT */ > > > + vm_size_t length, vm_offset_t max_addr, int find_space, vm_prot_t prot, > > > + vm_prot_t max, int cow) > > > { > > > vm_offset_t alignment, initial_addr, start; > > > int result; > > > @@ -1556,6 +1556,26 @@ again: > > > return (result); > > > } > > > > > > +int > > > +vm_map_find(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > > > + vm_offset_t *addr, /* IN/OUT */ > > > + vm_size_t length, vm_offset_t max_addr, int find_space, vm_prot_t prot, > > > + vm_prot_t max, int cow) > > > +{ > > > + int rv; > > > + bool hinted; > > > + > > > + hinted = *addr != 0; > > > + for (;;) { > > > + rv = vm_map_find1(map, object, offset, addr, length, max_addr, > > > + find_space, prot, max, cow); > > > + if (rv != KERN_NO_SPACE || !hinted) > > > + return (rv); > > > + *addr = 0; > > > + hinted = false; > > > + } > > > +} > > > + > > > /* > > > * vm_map_simplify_entry: > > > * > > > @@ -1970,7 +1990,7 @@ vm_map_pmap_enter(vm_map_t map, vm_offset_t addr, vm_prot_t prot, > > > */ > > > int > > > vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > > > - vm_prot_t new_prot, boolean_t set_max) > > > + vm_prot_t new_prot, boolean_t set_max) > > > { > > > vm_map_entry_t current, entry; > > > vm_object_t obj; > > > @@ -1995,7 +2015,8 @@ vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > > > if (vm_map_lookup_entry(map, start, &entry)) { > > > vm_map_clip_start(map, entry, start); > > > } else { > > > - entry = entry->next; > > > + vm_map_unlock(map); > > > + return (KERN_INVALID_ADDRESS); > > > } > > > > > > /* > > > @@ -2003,6 +2024,8 @@ vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > > > */ > > > for (current = entry; current != &map->header && current->start < end; > > > current = current->next) { > > > + if ((current->eflags & MAP_ENTRY_GUARD) != 0) > > > + continue; > > > if (current->eflags & MAP_ENTRY_IS_SUB_MAP) { > > > vm_map_unlock(map); > > > return (KERN_INVALID_ARGUMENT); > > > @@ -2011,6 +2034,11 @@ vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > > > vm_map_unlock(map); > > > return (KERN_PROTECTION_FAILURE); > > > } > > > + if (current->end < end && (current->next == &map->header || > > > + current->next->start > current->end)) { > > > + vm_map_unlock(map); > > > + return (KERN_INVALID_ADDRESS); > > > + } > > > } > > > > > > /* > > > @@ -2712,9 +2740,9 @@ vm_map_wire(vm_map_t map, vm_offset_t start, vm_offset_t end, > > > * If VM_MAP_WIRE_HOLESOK was specified, skip this check. > > > */ > > > next_entry: > > > - if (((flags & VM_MAP_WIRE_HOLESOK) == 0) && > > > - (entry->end < end && (entry->next == &map->header || > > > - entry->next->start > entry->end))) { > > > + if ((flags & VM_MAP_WIRE_HOLESOK) == 0 && > > > + entry->end < end && (entry->next == &map->header || > > > + entry->next->start > entry->end)) { > > > end = entry->end; > > > rv = KERN_INVALID_ADDRESS; > > > goto done; > > > diff --git a/sys/vm/vm_mmap.c b/sys/vm/vm_mmap.c > > > index 4d8f6ad9ed7..09912250d52 100644 > > > --- a/sys/vm/vm_mmap.c > > > +++ b/sys/vm/vm_mmap.c > > > @@ -607,6 +607,7 @@ kern_mprotect(struct thread *td, uintptr_t addr0, size_t size, int prot) > > > case KERN_PROTECTION_FAILURE: > > > return (EACCES); > > > case KERN_RESOURCE_SHORTAGE: > > > + case KERN_INVALID_ADDRESS: > > > return (ENOMEM); > > > } > > > return (EINVAL); > > > > I'm seeing similar fallout on r320327, built from scratch while > > running r320293: > > > > Script started on Sun Jun 25 17:03:41 2017 > > # uname -aKU > > FreeBSD 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r320327: Sun Jun 25 16:35:02 CEST 2017 root@freebsd-head-uefi-zfs.bsd.net:/usr/obj/usr/src/sys/VBOX amd64 1200035 1200035 > > # /sbin/zfs mount -va > > Fatal error 'Cannot allocate red zone for initial thread' at line 392 in file /usr/src/lib/libthr/thread/thr_init.c (errno = 12) > > Abort trap (core dumped) > > # ls -l /lib/libthr* > > -r--r--r-- 1 root wheel 153336 Jun 25 16:39 /lib/libthr.so.3 > > # ls -l /usr/lib/libthr* > > -r--r--r-- 1 root wheel 256064 May 28 21:41 /usr/lib/libthr.a > > lrwxr-xr-x 1 root wheel 21 Jun 25 16:39 /usr/lib/libthr.so -> ../../lib/libthr.so.3 > > -r--r--r-- 1 root wheel 270362 May 28 21:41 /usr/lib/libthr_p.a > > -r--r--r-- 1 root wheel 45376 Apr 22 10:39 /usr/lib/libthread_db.a > > lrwxr-xr-x 1 root wheel 17 Jun 25 16:39 /usr/lib/libthread_db.so -> libthread_db.so.3 > > -r--r--r-- 1 root wheel 38776 Jun 25 16:39 /usr/lib/libthread_db.so.3 > > -r--r--r-- 1 root wheel 49496 Apr 22 10:39 /usr/lib/libthread_db_p.a > > # ldd /sbin/zfs > > /sbin/zfs: > > libjail.so.1 => /lib/libjail.so.1 (0x800247000) > > libnvpair.so.2 => /lib/libnvpair.so.2 (0x80024f000) > > libuutil.so.2 => /lib/libuutil.so.2 (0x800269000) > > libzfs_core.so.2 => /lib/libzfs_core.so.2 (0x800276000) > > libzfs.so.3 => /lib/libzfs.so.3 (0x800281000) > > libc.so.7 => /lib/libc.so.7 (0x8002cd000) > > libmd.so.6 => /lib/libmd.so.6 (0x8006d1000) > > libumem.so.2 => /lib/libumem.so.2 (0x8006eb000) > > libutil.so.9 => /lib/libutil.so.9 (0x8006f0000) > > libm.so.5 => /lib/libm.so.5 (0x800706000) > > libavl.so.2 => /lib/libavl.so.2 (0x800736000) > > libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x80073a000) > > libgeom.so.5 => /lib/libgeom.so.5 (0x800770000) > > libz.so.6 => /lib/libz.so.6 (0x800779000) > > libthr.so.3 => /lib/libthr.so.3 (0x800793000) > > libsbuf.so.6 => /lib/libsbuf.so.6 (0x8007bd000) > > # ls -l /sbin/zfs > > -r-xr-xr-x 1 root wheel 124296 Jun 25 16:39 /sbin/zfs > > # exit > > > > Script done on Sun Jun 25 17:04:32 2017 > > > > Although not relevant for this case, neither /usr/lib/libthr.a, nor > > /usr/lib/libthr_p.a, nor /usr/lib/libthread_db.a, nor > > /usr/lib/libthread_db_p.a has been updated by installworld, nor > > removed by delete-old-libs, in a very long time. > > > > Anything else I can contribute? I'll try your patch and see if I can > > get any further. > > Partial success! Your patch brought me past the libthr issue with > /sbin/zfs. > > Like the OP, portupgrade (and ruby22) complains about unable to create > a timer thread. I shall attempt to rebuild lang/ruby22 and > emulators/virtualbox-ose-additions-nox11. I think I'll switch to > ruby23 while I'm at it. The newly built portupgrade and ruby23 are limping away with: <main>: warning: pthread_create failed for time: Resource temporarily unavailable, scheduling broken At least I can get emulators/virtualbox-ose-additions-nox11 up to speed. -- +-------------------------------+------------------------------------+ | Vennlig hilsen, | Best regards, | | Trond Endrestøl, | Trond Endrestøl, | | IT-ansvarlig, | System administrator, | | Fagskolen Innlandet, | Gjøvik Technical College, Norway, | | tlf. mob. 952 62 567, | Cellular...: +47 952 62 567, | | sentralbord 61 14 54 00. | Switchboard: +47 61 14 54 00. | +-------------------------------+------------------------------------+ From owner-freebsd-current@freebsd.org Sun Jun 25 16:15:28 2017 Return-Path: <owner-freebsd-current@freebsd.org> Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7531ED94D80 for <freebsd-current@mailman.ysv.freebsd.org>; Sun, 25 Jun 2017 16:15:28 +0000 (UTC) (envelope-from trond@fagskolen.gjovik.no) Received: from smtp.fagskolen.gjovik.no (smtp.fagskolen.gjovik.no [IPv6:2001:700:1100:1:200:ff:fe00:b]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.fagskolen.gjovik.no", Issuer "Fagskolen i Gj??vik" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1740374FBA for <freebsd-current@freebsd.org>; Sun, 25 Jun 2017 16:15:27 +0000 (UTC) (envelope-from trond@fagskolen.gjovik.no) Received: from mail.fig.ol.no (localhost [127.0.0.1]) by mail.fig.ol.no (8.15.2/8.15.2) with ESMTPS id v5PGFJUO001561 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for <freebsd-current@freebsd.org>; Sun, 25 Jun 2017 18:15:19 +0200 (CEST) (envelope-from trond@fagskolen.gjovik.no) Received: from localhost (trond@localhost) by mail.fig.ol.no (8.15.2/8.15.2/Submit) with ESMTP id v5PGFICC001558 for <freebsd-current@freebsd.org>; Sun, 25 Jun 2017 18:15:18 +0200 (CEST) (envelope-from trond@fagskolen.gjovik.no) X-Authentication-Warning: mail.fig.ol.no: trond owned process doing -bs Date: Sun, 25 Jun 2017 18:15:18 +0200 (CEST) From: =?ISO-8859-1?Q?Trond_Endrest=F8l?= <Trond.Endrestol@fagskolen.gjovik.no> Sender: Trond.Endrestol@fagskolen.gjovik.no To: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: Current amd64 new error or warning from today's current with ruby r320323 In-Reply-To: <alpine.BSF.2.21.1706251807230.37790@mail.fig.ol.no> Message-ID: <alpine.BSF.2.21.1706251813090.37790@mail.fig.ol.no> References: <44028FBF-C8B1-4A69-B798-F4E28CFCD779@pozo.com> <20170625012359.GS3437@kib.kiev.ua> <040BF7D1-2FDF-415B-9A17-ADD608503F14@pozo.com> <20170625020441.GT3437@kib.kiev.ua> <219E6B59-EF90-443D-8B53-4C39C4D6CFBD@pozo.com> <20170625134133.GA3437@kib.kiev.ua> <alpine.BSF.2.21.1706251709340.37790@mail.fig.ol.no> <alpine.BSF.2.21.1706251747180.37790@mail.fig.ol.no> <alpine.BSF.2.21.1706251807230.37790@mail.fig.ol.no> User-Agent: Alpine 2.21 (BSF 202 2017-01-01) Organization: Fagskolen Innlandet OpenPGP: url=http://fig.ol.no/~trond/trond.key MIME-Version: 1.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail.fig.ol.no Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current <freebsd-current.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>; List-Post: <mailto:freebsd-current@freebsd.org> List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=subscribe> X-List-Received-Date: Sun, 25 Jun 2017 16:15:28 -0000 On Sun, 25 Jun 2017 18:08+0200, Trond Endrestøl wrote: > On Sun, 25 Jun 2017 17:56+0200, Trond Endrestøl wrote: > > > On Sun, 25 Jun 2017 17:18+0200, Trond Endrestøl wrote: > > > > > On Sun, 25 Jun 2017 16:41+0300, Konstantin Belousov wrote: > > > > > > > On Sat, Jun 24, 2017 at 07:19:25PM -0700, Manfred Antar wrote: > > > > > > > > > > > On Jun 24, 2017, at 7:04 PM, Konstantin Belousov <kostikbel@gmail.com> wrote: > > > > > > > > > > > > On Sat, Jun 24, 2017 at 06:48:03PM -0700, Manfred Antar wrote: > > > > > >> > > > > > >>> On Jun 24, 2017, at 6:23 PM, Konstantin Belousov <kostikbel@gmail.com> wrote: > > > > > >>> > > > > > >>> On Sat, Jun 24, 2017 at 06:08:50PM -0700, Manfred Antar wrote: > > > > > >>>> New world and kernel r320323 > > > > > >>>> I get a new error or message when using ruby: > > > > > >>>> > > > > > >>>> > > > > > >>>> /usr/local/sbin/portupgrade -av > > > > > >>>> <main>: warning: pthread_create failed for timer: Resource temporarily unavailable, scheduling broken > > > > > >>>> > > > > > >>>> everything works just this message when using ruby. I recompiled ruby , still same message > > > > > >>>> > > > > > >>>> /usr/local/bin/ruby -v > > > > > >>>> <main>: warning: pthread_create failed for timer: Resource temporarily unavailable, scheduling broken > > > > > >>>> ruby 2.3.4p301 (2017-03-30 revision 58214) [amd64-freebsd12] > > > > > >>>> > > > > > >>>> Not sure what???s changed, I noticed some commits to vm stuff, maybe thats it. > > > > > >>> > > > > > >>> ktrace your failing ruby invocation, then post output of kdump -H somewhere. > > > > > >>> > > > > > >> > > > > > >> Ok not sure if this is right , but this is what i did: > > > > > >> > > > > > >> (tmp)4637}ktrace /usr/local/bin/ruby -v > > > > > >> <main>: warning: pthread_create failed for timer: Resource temporarily unavailable, scheduling broken > > > > > >> ruby 2.3.4p301 (2017-03-30 revision 58214) [amd64-freebsd12] > > > > > >> > > > > > >> (tmp)4638}kdump -H -f ./ktrace.out > kdump.txt > > > > > >> > > > > > >> you can get kdump.txt at: > > > > > >> > > > > > >> http://www.pozo.com/kernel/kdump <http://www.pozo.com/kernel/kdump>.txt > > > > > >> > > > > > >> It???s not failing, I don???t think , I can do portupgrade and it works fine. > > > > > >> I just get this new message > > > > > > > > > > > > I see what is going on, but it is somewhat strange that it happens. > > > > > > > > > > > > Do you run ruby in a jail with old (say, stable/10) libthr ? > > > > > > Or do you have environment variable LIBPTHREAD_SPLITSTACK_MAIN set in > > > > > > your environment ? > > > > > > > > > > > > Anyway, the rework of the stack grow indeed have incompatibility with the > > > > > > old (pre-11) libthr, which tries to split main thread stack into smaller > > > > > > stacks for the new threads. New stack grow code was specifically designed > > > > > > to prevent this. Some hack would be needed there, to allow reuse of > > > > > > the main stack gap. > > > > > > > > > > > > > > > > Not running any jails > > > > > all libraries are current as of today > > > > > locate libthr. |xargs ls -l > > > > > -r--r--r-- 1 root wheel 120240 Jun 24 12:50 /lib/libthr.so.3 > > > > > -r--r--r-- 1 root wheel 256072 Jun 24 12:50 /usr/lib/libthr.a > > > > > lrwxr-xr-x 1 root wheel 21 Jun 24 12:50 /usr/lib/libthr.so -> ../../lib/libthr.so.3 > > > > > > > > > > ldd /usr/local/bin/ruby > > > > > -rwxr-xr-x 1 root wheel 2677552 Jun 24 18:22 /usr/local/lib/libruby23.so.2 > > > > > -rwxr-xr-x 1 root wheel 43832 Jun 24 19:10 /usr/local/lib/libunwind.so.8.0.1 > > > > > > > > > > /usr/local/bin/ruby: > > > > > libruby23.so.23 => /usr/local/lib/libruby23.so.23 (0x800a00000) > > > > > libelf.so.2 => /lib/libelf.so.2 (0x800e9d000) > > > > > libunwind.so.8 => /usr/local/lib/libunwind.so.8 (0x8010b5000) > > > > > libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x8012ce000) > > > > > libprocstat.so.1 => /usr/lib/libprocstat.so.1 (0x8014d1000) > > > > > libcrypt.so.5 => /lib/libcrypt.so.5 (0x8016db000) > > > > > libm.so.5 => /lib/libm.so.5 (0x8018f9000) > > > > > libthr.so.3 => /lib/libthr.so.3 (0x801b26000) > > > > > libc.so.7 => /lib/libc.so.7 (0x801d4e000) > > > > > libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x802335000) > > > > > liblzma.so.5 => /usr/lib/liblzma.so.5 (0x80254b000) > > > > > libkvm.so.7 => /lib/libkvm.so.7 (0x802774000) > > > > > libutil.so.9 => /lib/libutil.so.9 (0x802982000) > > > > > > > > > > no LIBPTHREAD_SPLITSTACK_MAIN set anywhere > > > > > > > > > > This is a fresh buildworld - installworld from noon today California time. > > > > > > > > > > here are env in make.conf: > > > > > DEFAULT_VERSIONS= mysql=5.7 apache=2.4 python2=2.7 python3=3.4 ruby=2.3 perl5=5.24 php=5.6 tcltk=8.6 samba=4.4 ssl=base ncurses=base > > > > > MALLOC_PRODUCTION=yes > > > > > WITH_BDB_VERSION=5 > > > > > > > > > > env in src.conf: > > > > > WITHOUT_DYNAMICROOT=yes > > > > > WITHOUT_UNBOUND=yes > > > > > WITHOUT_CASPER=yes > > > > > WITHOUT_CAPSICUM=yes > > > > > WITHOUT_DMAGENT=yes > > > > > WITHOUT_PROFILE=yes > > > > > WITHOUT_TESTS=yes > > > > > WITHOUT_KERNEL_SYMBOLS=yes > > > > > WITHOUT_DEBUG_FILES=1 > > > > > WITHOUT_LIB32=yes > > > > > INSTALL_NODEBUG=yes > > > > > > > > > > # Don't die on warnings > > > > > NO_WERROR= > > > > > WERROR= > > > > > KERNCONF=pozo > > > > > WITH_CCACHE_BUILD=yes > > > > > > > > Weird, I do not see where such behaviour could come from. I mean, > > > > the behaviour of the libthr, assuming that the failing call is from > > > > libthr and not from the ruby. > > > > > > > > Anyway, please try the following. Note that this might be not the final > > > > fix, but I am interesting in seeing whether it fixed the issue for you. > > > > I want to see the kdump output from ktrace _without_ -di, same as in > > > > your first trace. > > > > > > > > diff --git a/lib/libc/sys/mprotect.2 b/lib/libc/sys/mprotect.2 > > > > index 3be81c6582a..ca4d6307c44 100644 > > > > --- a/lib/libc/sys/mprotect.2 > > > > +++ b/lib/libc/sys/mprotect.2 > > > > @@ -28,7 +28,7 @@ > > > > .\" @(#)mprotect.2 8.1 (Berkeley) 6/9/93 > > > > .\" $FreeBSD$ > > > > .\" > > > > -.Dd August 3, 2016 > > > > +.Dd June 24, 2017 > > > > .Dt MPROTECT 2 > > > > .Os > > > > .Sh NAME > > > > @@ -80,10 +80,24 @@ and > > > > arguments is not valid. > > > > .It Bq Er EACCES > > > > The calling process was not allowed to change > > > > -the protection to the value specified by > > > > -the > > > > +the protection to the value specified by the > > > > .Fa prot > > > > argument. > > > > +.It Bq Er ENOMEM > > > > +The virtual address range specified by the > > > > +.Fa addr > > > > +and > > > > +.Fa len > > > > +arguments specify one or more pages which are not mapped. > > > > +.It Bq Er ENOMEM > > > > +The > > > > +.Fa prot > > > > +argument specifies > > > > +.Dv PROT_WRITE > > > > +on a > > > > +.Dv MAP_PRIVATE > > > > +mapping, and it would require more swap space than the system is able to > > > > +supply for the private pages. > > > > .El > > > > .Sh SEE ALSO > > > > .Xr madvise 2 , > > > > diff --git a/sys/vm/vm_map.c b/sys/vm/vm_map.c > > > > index 39cf303eaa0..b311b54d0f0 100644 > > > > --- a/sys/vm/vm_map.c > > > > +++ b/sys/vm/vm_map.c > > > > @@ -1490,11 +1490,11 @@ vm_map_fixed(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > > > > * If object is non-NULL, ref count must be bumped by caller > > > > * prior to making call to account for the new entry. > > > > */ > > > > -int > > > > -vm_map_find(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > > > > - vm_offset_t *addr, /* IN/OUT */ > > > > - vm_size_t length, vm_offset_t max_addr, int find_space, > > > > - vm_prot_t prot, vm_prot_t max, int cow) > > > > +static int > > > > +vm_map_find1(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > > > > + vm_offset_t *addr, /* IN/OUT */ > > > > + vm_size_t length, vm_offset_t max_addr, int find_space, vm_prot_t prot, > > > > + vm_prot_t max, int cow) > > > > { > > > > vm_offset_t alignment, initial_addr, start; > > > > int result; > > > > @@ -1556,6 +1556,26 @@ again: > > > > return (result); > > > > } > > > > > > > > +int > > > > +vm_map_find(vm_map_t map, vm_object_t object, vm_ooffset_t offset, > > > > + vm_offset_t *addr, /* IN/OUT */ > > > > + vm_size_t length, vm_offset_t max_addr, int find_space, vm_prot_t prot, > > > > + vm_prot_t max, int cow) > > > > +{ > > > > + int rv; > > > > + bool hinted; > > > > + > > > > + hinted = *addr != 0; > > > > + for (;;) { > > > > + rv = vm_map_find1(map, object, offset, addr, length, max_addr, > > > > + find_space, prot, max, cow); > > > > + if (rv != KERN_NO_SPACE || !hinted) > > > > + return (rv); > > > > + *addr = 0; > > > > + hinted = false; > > > > + } > > > > +} > > > > + > > > > /* > > > > * vm_map_simplify_entry: > > > > * > > > > @@ -1970,7 +1990,7 @@ vm_map_pmap_enter(vm_map_t map, vm_offset_t addr, vm_prot_t prot, > > > > */ > > > > int > > > > vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > > > > - vm_prot_t new_prot, boolean_t set_max) > > > > + vm_prot_t new_prot, boolean_t set_max) > > > > { > > > > vm_map_entry_t current, entry; > > > > vm_object_t obj; > > > > @@ -1995,7 +2015,8 @@ vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > > > > if (vm_map_lookup_entry(map, start, &entry)) { > > > > vm_map_clip_start(map, entry, start); > > > > } else { > > > > - entry = entry->next; > > > > + vm_map_unlock(map); > > > > + return (KERN_INVALID_ADDRESS); > > > > } > > > > > > > > /* > > > > @@ -2003,6 +2024,8 @@ vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > > > > */ > > > > for (current = entry; current != &map->header && current->start < end; > > > > current = current->next) { > > > > + if ((current->eflags & MAP_ENTRY_GUARD) != 0) > > > > + continue; > > > > if (current->eflags & MAP_ENTRY_IS_SUB_MAP) { > > > > vm_map_unlock(map); > > > > return (KERN_INVALID_ARGUMENT); > > > > @@ -2011,6 +2034,11 @@ vm_map_protect(vm_map_t map, vm_offset_t start, vm_offset_t end, > > > > vm_map_unlock(map); > > > > return (KERN_PROTECTION_FAILURE); > > > > } > > > > + if (current->end < end && (current->next == &map->header || > > > > + current->next->start > current->end)) { > > > > + vm_map_unlock(map); > > > > + return (KERN_INVALID_ADDRESS); > > > > + } > > > > } > > > > > > > > /* > > > > @@ -2712,9 +2740,9 @@ vm_map_wire(vm_map_t map, vm_offset_t start, vm_offset_t end, > > > > * If VM_MAP_WIRE_HOLESOK was specified, skip this check. > > > > */ > > > > next_entry: > > > > - if (((flags & VM_MAP_WIRE_HOLESOK) == 0) && > > > > - (entry->end < end && (entry->next == &map->header || > > > > - entry->next->start > entry->end))) { > > > > + if ((flags & VM_MAP_WIRE_HOLESOK) == 0 && > > > > + entry->end < end && (entry->next == &map->header || > > > > + entry->next->start > entry->end)) { > > > > end = entry->end; > > > > rv = KERN_INVALID_ADDRESS; > > > > goto done; > > > > diff --git a/sys/vm/vm_mmap.c b/sys/vm/vm_mmap.c > > > > index 4d8f6ad9ed7..09912250d52 100644 > > > > --- a/sys/vm/vm_mmap.c > > > > +++ b/sys/vm/vm_mmap.c > > > > @@ -607,6 +607,7 @@ kern_mprotect(struct thread *td, uintptr_t addr0, size_t size, int prot) > > > > case KERN_PROTECTION_FAILURE: > > > > return (EACCES); > > > > case KERN_RESOURCE_SHORTAGE: > > > > + case KERN_INVALID_ADDRESS: > > > > return (ENOMEM); > > > > } > > > > return (EINVAL); > > > > > > I'm seeing similar fallout on r320327, built from scratch while > > > running r320293: > > > > > > Script started on Sun Jun 25 17:03:41 2017 > > > # uname -aKU > > > FreeBSD 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r320327: Sun Jun 25 16:35:02 CEST 2017 root@freebsd-head-uefi-zfs.bsd.net:/usr/obj/usr/src/sys/VBOX amd64 1200035 1200035 > > > # /sbin/zfs mount -va > > > Fatal error 'Cannot allocate red zone for initial thread' at line 392 in file /usr/src/lib/libthr/thread/thr_init.c (errno = 12) > > > Abort trap (core dumped) > > > # ls -l /lib/libthr* > > > -r--r--r-- 1 root wheel 153336 Jun 25 16:39 /lib/libthr.so.3 > > > # ls -l /usr/lib/libthr* > > > -r--r--r-- 1 root wheel 256064 May 28 21:41 /usr/lib/libthr.a > > > lrwxr-xr-x 1 root wheel 21 Jun 25 16:39 /usr/lib/libthr.so -> ../../lib/libthr.so.3 > > > -r--r--r-- 1 root wheel 270362 May 28 21:41 /usr/lib/libthr_p.a > > > -r--r--r-- 1 root wheel 45376 Apr 22 10:39 /usr/lib/libthread_db.a > > > lrwxr-xr-x 1 root wheel 17 Jun 25 16:39 /usr/lib/libthread_db.so -> libthread_db.so.3 > > > -r--r--r-- 1 root wheel 38776 Jun 25 16:39 /usr/lib/libthread_db.so.3 > > > -r--r--r-- 1 root wheel 49496 Apr 22 10:39 /usr/lib/libthread_db_p.a > > > # ldd /sbin/zfs > > > /sbin/zfs: > > > libjail.so.1 => /lib/libjail.so.1 (0x800247000) > > > libnvpair.so.2 => /lib/libnvpair.so.2 (0x80024f000) > > > libuutil.so.2 => /lib/libuutil.so.2 (0x800269000) > > > libzfs_core.so.2 => /lib/libzfs_core.so.2 (0x800276000) > > > libzfs.so.3 => /lib/libzfs.so.3 (0x800281000) > > > libc.so.7 => /lib/libc.so.7 (0x8002cd000) > > > libmd.so.6 => /lib/libmd.so.6 (0x8006d1000) > > > libumem.so.2 => /lib/libumem.so.2 (0x8006eb000) > > > libutil.so.9 => /lib/libutil.so.9 (0x8006f0000) > > > libm.so.5 => /lib/libm.so.5 (0x800706000) > > > libavl.so.2 => /lib/libavl.so.2 (0x800736000) > > > libbsdxml.so.4 => /lib/libbsdxml.so.4 (0x80073a000) > > > libgeom.so.5 => /lib/libgeom.so.5 (0x800770000) > > > libz.so.6 => /lib/libz.so.6 (0x800779000) > > > libthr.so.3 => /lib/libthr.so.3 (0x800793000) > > > libsbuf.so.6 => /lib/libsbuf.so.6 (0x8007bd000) > > > # ls -l /sbin/zfs > > > -r-xr-xr-x 1 root wheel 124296 Jun 25 16:39 /sbin/zfs > > > # exit > > > > > > Script done on Sun Jun 25 17:04:32 2017 > > > > > > Although not relevant for this case, neither /usr/lib/libthr.a, nor > > > /usr/lib/libthr_p.a, nor /usr/lib/libthread_db.a, nor > > > /usr/lib/libthread_db_p.a has been updated by installworld, nor > > > removed by delete-old-libs, in a very long time. > > > > > > Anything else I can contribute? I'll try your patch and see if I can > > > get any further. > > > > Partial success! Your patch brought me past the libthr issue with > > /sbin/zfs. > > > > Like the OP, portupgrade (and ruby22) complains about unable to create > > a timer thread. I shall attempt to rebuild lang/ruby22 and > > emulators/virtualbox-ose-additions-nox11. I think I'll switch to > > ruby23 while I'm at it. > > The newly built portupgrade and ruby23 are limping away with: > > <main>: warning: pthread_create failed for time: Resource temporarily unavailable, scheduling broken > > At least I can get emulators/virtualbox-ose-additions-nox11 up to > speed. Or maybe I'm not so lucky, since ar dumped core while linking RuntimeGuestR3 and RuntimeGuestR3Shared. I'll stop posting while you get this sorted out. -- +-------------------------------+------------------------------------+ | Vennlig hilsen, | Best regards, | | Trond Endrestøl, | Trond Endrestøl, | | IT-ansvarlig, | System administrator, | | Fagskolen Innlandet, | Gjøvik Technical College, Norway, | | tlf. mob. 952 62 567, | Cellular...: +47 952 62 567, | | sentralbord 61 14 54 00. | Switchboard: +47 61 14 54 00. | +-------------------------------+------------------------------------+ From owner-freebsd-current@freebsd.org Sun Jun 25 16:41:22 2017 Return-Path: <owner-freebsd-current@freebsd.org> Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1FBD0D953B7 for <freebsd-current@mailman.ysv.freebsd.org>; Sun, 25 Jun 2017 16:41:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9B12075C7A for <freebsd-current@freebsd.org>; Sun, 25 Jun 2017 16:41:21 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v5PGfFh9076887 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 25 Jun 2017 19:41:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v5PGfFh9076887 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v5PGfFv2076886; Sun, 25 Jun 2017 19:41:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 25 Jun 2017 19:41:15 +0300 From: Konstantin Belousov <kostikbel@gmail.com> To: Manfred Antar <null@pozo.com> Cc: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: Current amd64 new error or warning from today's current with ruby r320323 Message-ID: <20170625164115.GD3437@kib.kiev.ua> References: <44028FBF-C8B1-4A69-B798-F4E28CFCD779@pozo.com> <20170625012359.GS3437@kib.kiev.ua> <040BF7D1-2FDF-415B-9A17-ADD608503F14@pozo.com> <20170625020441.GT3437@kib.kiev.ua> <219E6B59-EF90-443D-8B53-4C39C4D6CFBD@pozo.com> <20170625134133.GA3437@kib.kiev.ua> <3FEA16E1-673F-4C58-8362-C1DCC0680698@pozo.com> <20170625145008.GB3437@kib.kiev.ua> <D94B09A1-9090-4C6A-8EC4-19D2AFBBA316@pozo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <D94B09A1-9090-4C6A-8EC4-19D2AFBBA316@pozo.com> User-Agent: Mutt/1.8.3 (2017-05-23) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current <freebsd-current.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>; List-Post: <mailto:freebsd-current@freebsd.org> List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=subscribe> X-List-Received-Date: Sun, 25 Jun 2017 16:41:22 -0000 On Sun, Jun 25, 2017 at 08:21:33AM -0700, Manfred Antar wrote: > > > On Jun 25, 2017, at 7:50 AM, Konstantin Belousov <kostikbel@gmail.com> wrote: > > > > On Sun, Jun 25, 2017 at 07:43:25AM -0700, Manfred Antar wrote: > >> maybe message got reformatted in mail program (mac mail). > >> could you send me a tar file of the patch? > >> also not sure if ???patch -p1 <patchfile??? is the correct invocation of patch > >> > >> you could cc root@pozo.com <mailto:root@pozo.com> , that way i have copy on freebsd box and on mac. > > > > https://people.freebsd.org/~kib/misc/vm2.1.patch <https://people.freebsd.org/~kib/misc/vm2.1.patch>; > > OK patched and built new kernel \ > rebooted, > same ruby message. So it must be a ruby thing. > new kdump.txt at http://www.pozo.com/kernel/kdump.txt <http://www.pozo.com/kernel/kdump.txt>; > > also i???ll put a copy of my kernel config in same directory: > > http://www.pozo.com/kernel/pozo <http://www.pozo.com/kernel/pozo>; > > only one module is being loaded at boot: > (kernel)4908}kldstat > Id Refs Address Size Name > 1 5 0xffffffff80200000 10380a8 kernel > 2 1 0xffffffff8123a000 e13f50 nvidia.ko > > I can disable nvidia if it helps as I really only access this machine over the net or serial console. > No need, I understood why MAP_STACK failed in this case, thanks to the ktrace log. This is indeed something ruby-specific, or rather, triggered by ruby special use of libthr. It is not related to the main stack split. It seems that ruby requested very small stack for a new thread, only 5 pages in size. This size caused the stack gap to be correctly calculated as having zero size, because the whole stack is allocated by initial grow. But then there is no space for the guard page, which caused mapping failure for it, and overall stack mapping failure. Try this. https://people.freebsd.org/~kib/misc/vm2.2.patch
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.21.1706251709340.37790>