From owner-freebsd-threads@FreeBSD.ORG Sun Apr 13 20:59:03 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 440BA37B401 for ; Sun, 13 Apr 2003 20:59:03 -0700 (PDT) Received: from exchhz01.viatech.com.cn (ip-167-164-97-218.anlai.com [218.97.164.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3B39343F93 for ; Sun, 13 Apr 2003 20:58:57 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from davidw2k (ip-240-1-168-192.rev.dyxnet.com [192.168.1.240]) by exchhz01.viatech.com.cn with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id HLDQQ1LT; Mon, 14 Apr 2003 11:44:54 +0800 Message-ID: <006001c3023a$65fe01d0$f001a8c0@davidw2k> From: "David Xu" To: "Daniel Eischen" References: Date: Mon, 14 Apr 2003 12:00:33 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 cc: freebsd-threads@freebsd.org Subject: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Apr 2003 03:59:03 -0000 After tested Daniel's pthread patch, I found 50% of ACE test program are core dumpped on my machine. So I studied the libpthread source code after applied the patch, I found that the main problem is thread state transition is not atomic,=20 for example in thr_mutex: mutex_queue_enq(*m, curthread); curthread->data.mutex =3D *m; /* This thread is active and is in a critical region (holding the mutex lock); we should be able to safely set the state. */ THR_SET_STATE(curthread, PS_MUTEX_WAIT); /* Unlock the mutex structure: */ THR_LOCK_RELEASE(curthread, &(*m)->m_lock); /* Schedule the next thread: */ _thr_sched_switch(curthread); thread sets its state to PS_MUTEX_WAIT, and call _thr_sched_switch, but it is not under scheduler lock, so there is a race between THR_SET_STATE and thr_sched_switch. I have inserted _kse_critical_enter() before THR_SET_STATE, the code looks as following: mutex_queue_enq(*m, curthread); curthread->data.mutex =3D *m; _kse_critical_enter(); /* This thread is active and is in a critical region (holding the mutex lock); we should be able to safely set the state. */ THR_SET_STATE(curthread, PS_MUTEX_WAIT); /* Unlock the mutex structure: */ THR_LOCK_RELEASE(curthread, &(*m)->m_lock); /* Schedule the next thread: */ _thr_sched_switch(curthread); I also commented out most code in thr_lock_wait() and thr_lock_wakeup(), I think without better scheduler lock, these code has race condition, and in most case will this cause a thread be reinserted into runq while it=20 is already in this queue. now, I can run ACE test programs without any core dumpped, and only the following program are failed: Cached_Conn_Test Conn_Test MT_Reactor_Timer_Test Malloc_Test Process_Strategy_Test Thread_Pool_Test a complete log file is at: http://people.freebsd.org/~davidxu/run_test.log the libpthread package I modified is at: http://people.freebsd.org/~davidxu/libpthread.tgz Also, I can run crew test program without any problem. I think the whole scheduler lock should be reworked to allow state transition is in atomic, my change is not SMP safe, only works on UP, because kse_critical_enter is only works for UP system. If we fixed this scheduler lock problem, I think the libpthread will be stable enough. David Xu From owner-freebsd-threads@FreeBSD.ORG Sun Apr 13 21:55:42 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4B48A37B407; Sun, 13 Apr 2003 21:55:42 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id EE0B943F75; Sun, 13 Apr 2003 21:55:40 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3E4teBg016082; Mon, 14 Apr 2003 00:55:40 -0400 (EDT) Received: from localhost (eischen@localhost)h3E4tdhM016079; Mon, 14 Apr 2003 00:55:39 -0400 (EDT) Date: Mon, 14 Apr 2003 00:55:39 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: <006001c3023a$65fe01d0$f001a8c0@davidw2k> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Apr 2003 04:55:42 -0000 On Mon, 14 Apr 2003, David Xu wrote: > After tested Daniel's pthread patch, I found 50% of ACE test > program are core dumpped on my machine. So I studied the > libpthread source code after applied the patch, I found that > the main problem is thread state transition is not atomic, > for example in thr_mutex: I have an updated version that works for all but the Cached_Conn_Test (except for the ones that don't work with libc_r). I narrowed down the problem to joint and am working on a fix for that. > mutex_queue_enq(*m, curthread); > curthread->data.mutex = *m; > /* > This thread is active and is in a critical > region (holding the mutex lock); we should > be able to safely set the state. > */ > THR_SET_STATE(curthread, PS_MUTEX_WAIT); > > /* Unlock the mutex structure: */ > THR_LOCK_RELEASE(curthread, &(*m)->m_lock); > /* Schedule the next thread: */ > _thr_sched_switch(curthread); > > > thread sets its state to PS_MUTEX_WAIT, and call _thr_sched_switch, > but it is not under scheduler lock, so there is a race between > THR_SET_STATE and thr_sched_switch. Unlike libc_r, I disassociated thread state from being in any run or waiting queue. So it is OK if another thread comes along and makes the thread runnable. Making the thread runnable requires taking the scheduler lock, so by the time thr_sched_switch takes the scheduler lock, the thread could be runnable again. There are flags to indicate whether the thread is in the run queue or the wait queue, so we rely on those flags before adding/removing threads from those queues (instead of state). > I have inserted _kse_critical_enter() before THR_SET_STATE, > the code looks as following: > > mutex_queue_enq(*m, curthread); > curthread->data.mutex = *m; > _kse_critical_enter(); > /* > This thread is active and is in a critical > region (holding the mutex lock); we should > be able to safely set the state. > */ > THR_SET_STATE(curthread, PS_MUTEX_WAIT); > > /* Unlock the mutex structure: */ > THR_LOCK_RELEASE(curthread, &(*m)->m_lock); > /* Schedule the next thread: */ > _thr_sched_switch(curthread); > > I also commented out most code in thr_lock_wait() and > thr_lock_wakeup(), I think without better scheduler lock, > these code has race condition, and in most case will > this cause a thread be reinserted into runq while it > is already in this queue. It shouldn't any more :-) > now, I can run ACE test programs without any core dumpped, > and only the following program are failed: > > Cached_Conn_Test > Conn_Test > MT_Reactor_Timer_Test > Malloc_Test > Process_Strategy_Test > Thread_Pool_Test > > a complete log file is at: > http://people.freebsd.org/~davidxu/run_test.log > > the libpthread package I modified is at: > http://people.freebsd.org/~davidxu/libpthread.tgz I'll take a look at it to make sure I haven't missed anything. > Also, I can run crew test program without any problem. The next thing I want to try is KDE. It doesn't currently work, but I'm hopeful that with your changes and my last set of changes it will. > I think the whole scheduler lock should be reworked > to allow state transition is in atomic, my change is > not SMP safe, only works on UP, because kse_critical_enter > is only works for UP system. If we fixed this scheduler lock > problem, I think the libpthread will be stable enough. I thought that we might want to make changing the state and switching to the scheduler all under the same lock, but decided not to go that way. I was concerned that by requiring the scheduler lock to be held while atomically changing the thread state and switching to the scheduler might introduce weird locking order and wanted to avoid that. For instance, inherited priority adjustments and condition signal broadcasts. We should look at it again though, because it may make sense in most cases. I'll try to get a new patch set out after I fix pthread_join(). I should have posted my earlier version but I thought I could fix join quickly. It turns out it's a bit more tricky than I thought. One question about kse_create(). I was thinking about implementing pthread_setconcurrency(). We need to return an error if the requested concurrency level cannot be achieved. If we do return an error, though, we should not end up creating _any_ additional KSEs. If someone does: ret = pthread_setconcurrency(5); How do we know how many KSEs (within the same KSEG) we can create? And if we use ncpu or some other knob, it doesn't automatically mean we _will_ get the number of KSEs requested. We can currently only make one at a time, so we may end up with some additional KSEs but less than requested -- but we still have to return an error to the caller. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Sun Apr 13 22:45:45 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A2A5737B401 for ; Sun, 13 Apr 2003 22:45:45 -0700 (PDT) Received: from exchhz01.viatech.com.cn (ip-167-164-97-218.anlai.com [218.97.164.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id B14AD43FA3 for ; Sun, 13 Apr 2003 22:45:41 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from davidw2k (ip-240-1-168-192.rev.dyxnet.com [192.168.1.240]) by exchhz01.viatech.com.cn with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id HLDQQ1VJ; Mon, 14 Apr 2003 13:31:39 +0800 Message-ID: <001601c30249$4f7917b0$f001a8c0@davidw2k> From: "David Xu" To: "Daniel Eischen" References: Date: Mon, 14 Apr 2003 13:47:18 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Apr 2003 05:45:45 -0000 ----- Original Message -----=20 From: "Daniel Eischen" To: "David Xu" Cc: Sent: Monday, April 14, 2003 12:55 PM Subject: Re: libpthread patch > On Mon, 14 Apr 2003, David Xu wrote: >=20 > > After tested Daniel's pthread patch, I found 50% of ACE test > > program are core dumpped on my machine. So I studied the > > libpthread source code after applied the patch, I found that > > the main problem is thread state transition is not atomic,=20 > > for example in thr_mutex: >=20 > I have an updated version that works for all but the Cached_Conn_Test > (except for the ones that don't work with libc_r). I narrowed down > the problem to joint and am working on a fix for that. >=20 > > mutex_queue_enq(*m, curthread); > > curthread->data.mutex =3D *m; > > /* > > This thread is active and is in a critical > > region (holding the mutex lock); we should > > be able to safely set the state. > > */ > > THR_SET_STATE(curthread, PS_MUTEX_WAIT); > >=20 > > /* Unlock the mutex structure: */ > > THR_LOCK_RELEASE(curthread, &(*m)->m_lock); > > /* Schedule the next thread: */ > > _thr_sched_switch(curthread); > >=20 > >=20 > > thread sets its state to PS_MUTEX_WAIT, and call _thr_sched_switch, > > but it is not under scheduler lock, so there is a race between > > THR_SET_STATE and thr_sched_switch. >=20 > Unlike libc_r, I disassociated thread state from being in any > run or waiting queue. So it is OK if another thread comes > along and makes the thread runnable. Making the thread runnable > requires taking the scheduler lock, so by the time thr_sched_switch > takes the scheduler lock, the thread could be runnable again. > There are flags to indicate whether the thread is in the run > queue or the wait queue, so we rely on those flags before > adding/removing threads from those queues (instead of state). >=20 OK, I was concerned that kse_switchout_thread() does not check those flags, and because of some race conditions, it insists to insert or remove thread from those queues, and causes core dumpped. > > I have inserted _kse_critical_enter() before THR_SET_STATE, > > the code looks as following: > >=20 > > mutex_queue_enq(*m, curthread); > > curthread->data.mutex =3D *m; > > _kse_critical_enter(); > > /* > > This thread is active and is in a critical > > region (holding the mutex lock); we should > > be able to safely set the state. > > */ > > THR_SET_STATE(curthread, PS_MUTEX_WAIT); > >=20 > > /* Unlock the mutex structure: */ > > THR_LOCK_RELEASE(curthread, &(*m)->m_lock); > > /* Schedule the next thread: */ > > _thr_sched_switch(curthread); > >=20 > > I also commented out most code in thr_lock_wait() and > > thr_lock_wakeup(), I think without better scheduler lock, > > these code has race condition, and in most case will > > this cause a thread be reinserted into runq while it=20 > > is already in this queue. >=20 > It shouldn't any more :-) >=20 > > now, I can run ACE test programs without any core dumpped, > > and only the following program are failed: > >=20 > > Cached_Conn_Test > > Conn_Test > > MT_Reactor_Timer_Test > > Malloc_Test > > Process_Strategy_Test > > Thread_Pool_Test > >=20 > > a complete log file is at: > > http://people.freebsd.org/~davidxu/run_test.log > >=20 > > the libpthread package I modified is at: > > http://people.freebsd.org/~davidxu/libpthread.tgz >=20 > I'll take a look at it to make sure I haven't missed anything. >=20 > > Also, I can run crew test program without any problem. >=20 > The next thing I want to try is KDE. It doesn't currently > work, but I'm hopeful that with your changes and my last > set of changes it will. >=20 > > I think the whole scheduler lock should be reworked > > to allow state transition is in atomic, my change is > > not SMP safe, only works on UP, because kse_critical_enter > > is only works for UP system. If we fixed this scheduler lock > > problem, I think the libpthread will be stable enough. >=20 > I thought that we might want to make changing the state > and switching to the scheduler all under the same lock, > but decided not to go that way. I was concerned that by > requiring the scheduler lock to be held while atomically > changing the thread state and switching to the scheduler > might introduce weird locking order and wanted to avoid > that. For instance, inherited priority adjustments and > condition signal broadcasts. We should look at it again > though, because it may make sense in most cases. >=20 However, I found that because there are multiple schedule queues, is it possible that there is deadlock between those queues? I mean threads are inter-locking each other's schedule queue. > I'll try to get a new patch set out after I fix > pthread_join(). I should have posted my earlier version > but I thought I could fix join quickly. It turns out > it's a bit more tricky than I thought. >=20 > One question about kse_create(). I was thinking about > implementing pthread_setconcurrency(). We need to return > an error if the requested concurrency level cannot be > achieved. If we do return an error, though, we should > not end up creating _any_ additional KSEs. If someone > does: >=20 > ret =3D pthread_setconcurrency(5); >=20 > How do we know how many KSEs (within the same KSEG) we can > create? And if we use ncpu or some other knob, it doesn't > automatically mean we _will_ get the number of KSEs requested. > We can currently only make one at a time, so we may end up > with some additional KSEs but less than requested -- but we > still have to return an error to the caller. >=20 there is a sysctl kern.threads.virtual_cpu, it tells application that max kses can be created in same ksegroup. it is independent from physical CPU, although it may equal number of physical CPU. the sysctl can be changed by root, but does not have effect, unless kern.threads.debug =3D 1, I might change it to not masked by kern.threads.debug. current kse_create prevents userland from creating kses (upcall) more than kern.threads.virtual_cpu, it will return EPROCLIM if userland is trying to do. I think kern.threads.virtual_cpu is what you want. Notic about libpthread patch: libpthread_init does not link _kse_initial into it's kseg, why? I have linked it. and I also fixed scheduler lock in _thr_alloc() and _thr_free(). in pthread_cancel, I think it should use thr_add_ref not thr_find, because it is possible before pthread_cancel processes that thread, the thread may already be exited. > --=20 > Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Tue Apr 15 14:26:59 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9673537B401; Tue, 15 Apr 2003 14:26:59 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 95AE143FA3; Tue, 15 Apr 2003 14:26:58 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3FLQvBg009583; Tue, 15 Apr 2003 17:26:57 -0400 (EDT) Received: from localhost (eischen@localhost)h3FLQvKF009580; Tue, 15 Apr 2003 17:26:57 -0400 (EDT) Date: Tue, 15 Apr 2003 17:26:57 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: <001601c30249$4f7917b0$f001a8c0@davidw2k> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2003 21:26:59 -0000 There's an updated patch file available at (a slightly different place): http://people.freebsd.org/~deischen/kse/libpthread.diffs There's also an html'ized log of the ACE tests: http://people.freebsd.org/~deischen/kse/ace_build_logs/index.html The only real problems seem to be with the ACE tests: Cached_Conn_Test Process_Manager_Test And I think these have something to do with wait() or waitpid() not working correctly. David, do you know of any problems in this area? It seems that sometimes waitpid() is returning 0 and the next time it is called it returns the process id. I wonder if it is being interrupted by a signal (either the kernel doing it or the UTS by use of kse_thr_interrupt)? Other than that, things are looking pretty good. The code needs some cleanup; I'll try to do that and see if I can get rid of the %gs munging as well. On Mon, 14 Apr 2003, David Xu wrote: > > ----- Original Message ----- > From: "Daniel Eischen" > To: "David Xu" > Cc: > Sent: Monday, April 14, 2003 12:55 PM > Subject: Re: libpthread patch > > > > > I think the whole scheduler lock should be reworked > > > to allow state transition is in atomic, my change is > > > not SMP safe, only works on UP, because kse_critical_enter > > > is only works for UP system. If we fixed this scheduler lock > > > problem, I think the libpthread will be stable enough. > > > > I thought that we might want to make changing the state > > and switching to the scheduler all under the same lock, > > but decided not to go that way. I was concerned that by > > requiring the scheduler lock to be held while atomically > > changing the thread state and switching to the scheduler > > might introduce weird locking order and wanted to avoid > > that. For instance, inherited priority adjustments and > > condition signal broadcasts. We should look at it again > > though, because it may make sense in most cases. > > > > However, I found that because there are multiple schedule > queues, is it possible that there is deadlock between those > queues? I mean threads are inter-locking each other's schedule > queue. Right, but this is OK as long as you don't hold your own scheduler lock when you are trying to lock another one. > > I'll try to get a new patch set out after I fix > > pthread_join(). I should have posted my earlier version > > but I thought I could fix join quickly. It turns out > > it's a bit more tricky than I thought. > > > > One question about kse_create(). I was thinking about > > implementing pthread_setconcurrency(). We need to return > > an error if the requested concurrency level cannot be > > achieved. If we do return an error, though, we should > > not end up creating _any_ additional KSEs. If someone > > does: > > > > ret = pthread_setconcurrency(5); > > > > How do we know how many KSEs (within the same KSEG) we can > > create? And if we use ncpu or some other knob, it doesn't > > automatically mean we _will_ get the number of KSEs requested. > > We can currently only make one at a time, so we may end up > > with some additional KSEs but less than requested -- but we > > still have to return an error to the caller. > > > > there is a sysctl kern.threads.virtual_cpu, it tells application > that max kses can be created in same ksegroup. it is independent > from physical CPU, although it may equal number of physical CPU. > the sysctl can be changed by root, but does not have effect, > unless kern.threads.debug = 1, I might change it to not masked by > kern.threads.debug. > current kse_create prevents userland from creating kses (upcall) > more than kern.threads.virtual_cpu, it will return EPROCLIM > if userland is trying to do. > I think kern.threads.virtual_cpu is what you want. Right, but there is no easy way for the UTS to ask for 4 additional KSEs and be certain that it will get 4. It can check the sysctl knob, but the kernel may still fail to create additional KSEs (resource starvation?). Since the UTS can only create them one at a time, it won't _really_ know if it can create 4 until after it creates the 4th one. If there is an error creating KSE number 2, 3, or 4, it can still return an error to the caller (from pthread_setconcurrency()), but there is an unexpected side-effect of having _some_ additional KSEs created. This isn't a big deal for now, but to be really correct, I think we might have to destroy any additional KSEs if we didn't get the full amount requested. > Notic about libpthread patch: > libpthread_init does not link _kse_initial into it's kseg, why? > I have linked it. and I also fixed scheduler lock in _thr_alloc() > and _thr_free(). I'll take a look at those. > in pthread_cancel, I think it should use thr_add_ref not > thr_find, because it is possible before pthread_cancel processes > that thread, the thread may already be exited. I removed thr_find() and changed it to thr_ref_add(). There is also a new argument to thr_ref_add(, , /* include dead */ int) so you can specify whether or not you want to include dead threads in your search. pthread_join() wants to include dead threads, whereas most of the others don't. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Tue Apr 15 18:36:20 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5268737B401 for ; Tue, 15 Apr 2003 18:36:20 -0700 (PDT) Received: from exchhz01.viatech.com.cn (ip-167-164-97-218.anlai.com [218.97.164.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id B528F43F75 for ; Tue, 15 Apr 2003 18:36:15 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from davidw2k (ip-240-1-168-192.rev.dyxnet.com [192.168.1.240]) by exchhz01.viatech.com.cn with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id HLDQQML8; Wed, 16 Apr 2003 09:21:36 +0800 Message-ID: <005501c303b8$cda19390$f001a8c0@davidw2k> From: "David Xu" To: "\"Daniel Eischen\"" References: <000701c303a9$0cdd9370$0701a8c0@tiger> Date: Wed, 16 Apr 2003 09:37:55 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 01:36:20 -0000 ----- Original Message -----=20 From: "Daniel Eischen" To: "David Xu" Cc: ; "Craig Rodrigues" Sent: Wednesday, April 16, 2003 5:26 AM Subject: Re: libpthread patch > There's an updated patch file available at (a slightly different = place): >=20 > http://people.freebsd.org/~deischen/kse/libpthread.diffs >=20 Will test it. > There's also an html'ized log of the ACE tests: >=20 > http://people.freebsd.org/~deischen/kse/ace_build_logs/index.html >=20 > The only real problems seem to be with the ACE tests: >=20 > Cached_Conn_Test > Process_Manager_Test >=20 > And I think these have something to do with wait() or waitpid() > not working correctly. David, do you know of any problems in > this area? It seems that sometimes waitpid() is returning 0 > and the next time it is called it returns the process id. > I wonder if it is being interrupted by a signal (either the > kernel doing it or the UTS by use of kse_thr_interrupt)? >=20 Remember current signal handling for threaded program is broken in kernel, any signal can be lost in kernel because of thread exiting, for our M:N based threaded process, the case is worse than 1:1 because we exit thread more often than 1:1 threading, so any signal related tests will frequently be failed. some code in ACE I find : for (;;) { int result =3D ACE_OS::waitpid (this->getpid (), status, WNOHANG); if (result !=3D 0) return result; =20 ACE_Sig_Set alarm_or_child; =20 alarm_or_child.sig_add (SIGALRM); alarm_or_child.sig_add (SIGCHLD); ACE_Time_Value time_left =3D wait_until - ACE_OS::gettimeofday (); =20 // If ACE_OS::ualarm doesn't have sub-second resolution: time_left +=3D ACE_Time_Value (0, 500000); time_left.usec (0); =20 if (time_left <=3D ACE_Time_Value::zero) return 0; // timeout ACE_OS::ualarm (time_left); if (ACE_OS::sigwait (alarm_or_child) =3D=3D -1) return ACE_INVALID_PID; } ... so you see, the code expects SIGCHLD and SIGALRM, if SIGCHLD lost, it would timeout and return 0; I did not find waitpid has bug. BTW, I have a patch for kse_release to let it direct return to userland and not schedule an upcall. the bit 0 of km_flags in kse_mailbox is used as a hint to tell kernel not to schedule an upcall for the kse. http://people.freebsd.org/~davidxu/kse_release.diff If nobody objects it, I will commit it. David Xu From owner-freebsd-threads@FreeBSD.ORG Tue Apr 15 19:58:41 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1F90737B401; Tue, 15 Apr 2003 19:58:41 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id E4AAC43F85; Tue, 15 Apr 2003 19:58:39 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3G2wcBg025961; Tue, 15 Apr 2003 22:58:38 -0400 (EDT) Received: from localhost (eischen@localhost)h3G2wcdl025957; Tue, 15 Apr 2003 22:58:38 -0400 (EDT) Date: Tue, 15 Apr 2003 22:58:38 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: <005501c303b8$cda19390$f001a8c0@davidw2k> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 02:58:41 -0000 On Wed, 16 Apr 2003, David Xu wrote: > ----- Original Message ----- > From: "Daniel Eischen" > To: "David Xu" > Cc: ; "Craig Rodrigues" > Sent: Wednesday, April 16, 2003 5:26 AM > Subject: Re: libpthread patch > > > > There's an updated patch file available at (a slightly different place): > > > > http://people.freebsd.org/~deischen/kse/libpthread.diffs > > > > Will test it. I found another problem with one of my other tests. It doesn't seem to affect any of the ACE tests, though. I'll continue debugging it. > > There's also an html'ized log of the ACE tests: > > > > http://people.freebsd.org/~deischen/kse/ace_build_logs/index.html > > > > The only real problems seem to be with the ACE tests: > > > > Cached_Conn_Test > > Process_Manager_Test > > > > And I think these have something to do with wait() or waitpid() > > not working correctly. David, do you know of any problems in > > this area? It seems that sometimes waitpid() is returning 0 > > and the next time it is called it returns the process id. > > I wonder if it is being interrupted by a signal (either the > > kernel doing it or the UTS by use of kse_thr_interrupt)? > > > > Remember current signal handling for threaded program is > broken in kernel, any signal can be lost in kernel because > of thread exiting, for our M:N based threaded process, the > case is worse than 1:1 because we exit thread more often than > 1:1 threading, so any signal related tests will frequently > be failed. some code in ACE I find : > for (;;) > { > int result = ACE_OS::waitpid (this->getpid (), > status, > WNOHANG); > if (result != 0) > return result; > > ACE_Sig_Set alarm_or_child; > > alarm_or_child.sig_add (SIGALRM); > alarm_or_child.sig_add (SIGCHLD); > ACE_Time_Value time_left = wait_until - ACE_OS::gettimeofday (); > > // If ACE_OS::ualarm doesn't have sub-second resolution: > time_left += ACE_Time_Value (0, 500000); > time_left.usec (0); > > if (time_left <= ACE_Time_Value::zero) > return 0; // timeout > > ACE_OS::ualarm (time_left); > if (ACE_OS::sigwait (alarm_or_child) == -1) > return ACE_INVALID_PID; > } > ... > so you see, the code expects SIGCHLD and SIGALRM, if > SIGCHLD lost, it would timeout and return 0; > I did not find waitpid has bug. I thought it might also be the UTS trying to interrupt the thread (kse_thr_interruot) while it was in the kernel (assuming the UTS did get the signal). > BTW, I have a patch for kse_release to let it direct > return to userland and not schedule an upcall. > the bit 0 of km_flags in kse_mailbox is used as a hint > to tell kernel not to schedule an upcall for the kse. > http://people.freebsd.org/~davidxu/kse_release.diff I haven't tested that yet; that's on my list of things to do :-) > If nobody objects it, I will commit it. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Tue Apr 15 20:20:15 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 70AF837B401 for ; Tue, 15 Apr 2003 20:20:15 -0700 (PDT) Received: from exchhz01.viatech.com.cn (ip-167-164-97-218.anlai.com [218.97.164.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id 676FE43FAF for ; Tue, 15 Apr 2003 20:20:13 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from davidw2k (ip-240-1-168-192.rev.dyxnet.com [192.168.1.240]) by exchhz01.viatech.com.cn with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id HLDQQMWH; Wed, 16 Apr 2003 11:05:35 +0800 Message-ID: <005201c303c7$53a344d0$f001a8c0@davidw2k> From: "David Xu" To: "Daniel Eischen" References: Date: Wed, 16 Apr 2003 11:21:53 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 03:20:15 -0000 ----- Original Message -----=20 From: "Daniel Eischen" To: "David Xu" Cc: ; ""Craig Rodrigues"" = Sent: Wednesday, April 16, 2003 10:58 AM Subject: Re: libpthread patch > On Wed, 16 Apr 2003, David Xu wrote: >=20 > > ----- Original Message -----=20 > > From: "Daniel Eischen" > > To: "David Xu" > > Cc: ; "Craig Rodrigues" = > > Sent: Wednesday, April 16, 2003 5:26 AM > > Subject: Re: libpthread patch > >=20 > I thought it might also be the UTS trying to interrupt > the thread (kse_thr_interruot) while it was in the kernel > (assuming the UTS did get the signal). >=20 kse_thr_interrupt would cause that thread return EINTR, it should get -1 not zero. From owner-freebsd-threads@FreeBSD.ORG Tue Apr 15 22:52:29 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C0BA037B401 for ; Tue, 15 Apr 2003 22:52:29 -0700 (PDT) Received: from exchhz01.viatech.com.cn (ip-167-164-97-218.anlai.com [218.97.164.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id EBC0743FBF for ; Tue, 15 Apr 2003 22:52:26 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from davidw2k (ip-240-1-168-192.rev.dyxnet.com [192.168.1.240]) by exchhz01.viatech.com.cn with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id HLDQQN1K; Wed, 16 Apr 2003 13:37:48 +0800 Message-ID: <00f501c303dc$969bfec0$f001a8c0@davidw2k> From: "David Xu" To: "Daniel Eischen" References: Date: Wed, 16 Apr 2003 13:54:05 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 05:52:30 -0000 ----- Original Message -----=20 From: "Daniel Eischen" To: "David Xu" Cc: ; ""Craig Rodrigues"" = Sent: Wednesday, April 16, 2003 10:58 AM Subject: Re: libpthread patch > I thought it might also be the UTS trying to interrupt > the thread (kse_thr_interruot) while it was in the kernel > (assuming the UTS did get the signal). Can you try a patch for kern_exit.c ? http://people.freebsd.org/~davidxu/kern_exit.c.diff From owner-freebsd-threads@FreeBSD.ORG Tue Apr 15 23:12:38 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EB6A637B401 for ; Tue, 15 Apr 2003 23:12:38 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2777E43FBF for ; Tue, 15 Apr 2003 23:12:38 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h3G6Cbj79562 for ; Wed, 16 Apr 2003 02:12:37 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Wed, 16 Apr 2003 02:12:37 -0400 (EDT) From: Jeff Roberson To: threads@freebsd.org Message-ID: <20030416020803.D76635-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: struct kse X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 06:12:39 -0000 Can we totally nuke ke_oncpu since it's only used in one place? Also, KEF_DIDRUN is sched_4bsd specific. I may move this. If we move the KEF_IDLEKSE as a property of the idle thread instead and do something else with KEF_EXIT we could remove ke_flags. ke_pctcpu is also scheduler specific although it is referenced from kinfo_proc so I wont move it yet. Comments? Jeff From owner-freebsd-threads@FreeBSD.ORG Tue Apr 15 23:22:14 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 48C0637B401 for ; Tue, 15 Apr 2003 23:22:14 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 880DA43F85 for ; Tue, 15 Apr 2003 23:22:13 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h3G6MDr83201 for ; Wed, 16 Apr 2003 02:22:13 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Wed, 16 Apr 2003 02:22:12 -0400 (EDT) From: Jeff Roberson To: threads@freebsd.org Message-ID: <20030416021527.J76635-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: struct proc X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 06:22:14 -0000 p_maxthrwaits should go away. It is ok to do the wakeup everytime you transition from p_numthreads == max_threads_per_proc to p_numthreads == max_threads_per_proc - 1. p_profthreads also seems like overkill but I don't have anything to suggest at the moment. From owner-freebsd-threads@FreeBSD.ORG Tue Apr 15 23:25:36 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D6D8837B404 for ; Tue, 15 Apr 2003 23:25:36 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 10A2243F75 for ; Tue, 15 Apr 2003 23:25:36 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h3G6PZa84484 for ; Wed, 16 Apr 2003 02:25:35 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Wed, 16 Apr 2003 02:25:35 -0400 (EDT) From: Jeff Roberson To: threads@freebsd.org Message-ID: <20030416022215.R76635-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: struct thread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 06:25:37 -0000 Is td_last_kse necessary? What about td_lastcpu? They don't really seem to be used. Also, td_locks is unused, although it would be nice to have it implemented. td_sleeplocks should be ifdefed with WITNESS. Cheers, Jeff From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 09:30:18 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1908437B401; Wed, 16 Apr 2003 09:30:18 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0BF1743FBD; Wed, 16 Apr 2003 09:30:17 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3GGUFBg016017; Wed, 16 Apr 2003 12:30:15 -0400 (EDT) Received: from localhost (eischen@localhost)h3GGUFHo016014; Wed, 16 Apr 2003 12:30:15 -0400 (EDT) Date: Wed, 16 Apr 2003 12:30:15 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: <00f501c303dc$969bfec0$f001a8c0@davidw2k> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 16:30:18 -0000 On Wed, 16 Apr 2003, David Xu wrote: > > ----- Original Message ----- > From: "Daniel Eischen" > To: "David Xu" > Cc: ; ""Craig Rodrigues"" > Sent: Wednesday, April 16, 2003 10:58 AM > Subject: Re: libpthread patch > > > I thought it might also be the UTS trying to interrupt > > the thread (kse_thr_interruot) while it was in the kernel > > (assuming the UTS did get the signal). > > Can you try a patch for kern_exit.c ? > http://people.freebsd.org/~davidxu/kern_exit.c.diff No, it didn't work. Same thing. The application is calling waitpid() with WNOHANG and pid -1. I am pretty sure there is a child process because I added some debugging statements and could see that it was spawned, saw it with top(1), and also saw when the waitpid() was called. Everything looks good on this end. The next call to waitpid(-1) returns the missed process id. I am not sure why WNOHANG is causing waitpid to return 0 for libkse and -1 for libc_r. It seems as if the process is still running so waitpid with WNOHANG _should_ return 0. I am not sure why ACE expects it to return the process id if it is still active. The test seems to do the right thing with libc_r, so that's kind of confusing. See if you can figure it out. The test is Process_Manager_Test. I'll continue to debug it also, and clean up my patches in general. I'd like to commit soon so we can get other testers going. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 09:30:19 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A732037B401 for ; Wed, 16 Apr 2003 09:30:19 -0700 (PDT) Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by mx1.FreeBSD.org (Postfix) with ESMTP id DC39E43FBD for ; Wed, 16 Apr 2003 09:30:18 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by rwcrmhc51.attbi.com (rwcrmhc51) with ESMTP id <2003041616301805100hsoa8e>; Wed, 16 Apr 2003 16:30:18 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id JAA43553; Wed, 16 Apr 2003 09:30:15 -0700 (PDT) Date: Wed, 16 Apr 2003 09:30:08 -0700 (PDT) From: Julian Elischer To: Jeff Roberson In-Reply-To: <20030416020803.D76635-100000@mail.chesapeake.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: struct kse X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 16:30:20 -0000 On Wed, 16 Apr 2003, Jeff Roberson wrote: > > Can we totally nuke ke_oncpu since it's only used in one place? > > Also, KEF_DIDRUN is sched_4bsd specific. I may move this. yes. ke_oncpu was only a placeholder for me to replace it with ke_pincpu(jhb's name), the CPU that KSE is nailed (pinned) to if we implement affinity/nailing. However that'll probably require special variables so it should go away. > > If we move the KEF_IDLEKSE as a property of the idle thread instead and do > something else with KEF_EXIT we could remove ke_flags. yes, I was going to suggest that the idle flag could go to the thread.. it removes knowledge of the KSE from at least one file.. > > ke_pctcpu is also scheduler specific although it is referenced from > kinfo_proc so I wont move it yet. It's scheduler specific but the outside world is also scheduler specific in its expectations :-/ > > Comments? > Jeff > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 09:35:30 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7E3CD37B401; Wed, 16 Apr 2003 09:35:30 -0700 (PDT) Received: from sccrmhc03.attbi.com (sccrmhc03.attbi.com [204.127.202.63]) by mx1.FreeBSD.org (Postfix) with ESMTP id C06F543FB1; Wed, 16 Apr 2003 09:35:29 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by sccrmhc03.attbi.com (sccrmhc03) with ESMTP id <2003041616352800300bttqoe>; Wed, 16 Apr 2003 16:35:29 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id JAA43594; Wed, 16 Apr 2003 09:35:27 -0700 (PDT) Date: Wed, 16 Apr 2003 09:35:25 -0700 (PDT) From: Julian Elischer To: Jeff Roberson In-Reply-To: <20030416022215.R76635-100000@mail.chesapeake.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: John Baldwin Subject: Re: struct thread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 16:35:30 -0000 On Wed, 16 Apr 2003, Jeff Roberson wrote: > Is td_last_kse necessary? What about td_lastcpu? They don't really seem > to be used. td_last_kse and td_last_cpu were used in some experimental cpu affinity code that I gave up on (i.e. ran out of time). The idea was that the system would attempt to first schedule the thread on teh cpu it was last on , and if not available, on teh kse that it last ran on. I never removed the items but was hoping that someone, seeing the names there would feel tempted to implement affinity.. (Alfred mumbled about trying it). > > Also, td_locks is unused, although it would be nice to have it > implemented. I think ithis is a jhb field > > td_sleeplocks should be ifdefed with WITNESS. ditto > > Cheers, > Jeff > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 09:36:01 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5256137B401; Wed, 16 Apr 2003 09:36:01 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7439F43FD7; Wed, 16 Apr 2003 09:36:00 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3GGZvBg016860; Wed, 16 Apr 2003 12:35:57 -0400 (EDT) Received: from localhost (eischen@localhost)h3GGZv01016856; Wed, 16 Apr 2003 12:35:57 -0400 (EDT) Date: Wed, 16 Apr 2003 12:35:57 -0400 (EDT) From: Daniel Eischen To: John Polstra In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: deischen@freebsd.org cc: jeff@freebsd.org cc: freebsd-threads@freebsd.org Subject: Re: May I add pthread_[gs]etconcurrency to the threads libraries? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 16:36:01 -0000 On Wed, 16 Apr 2003, John Polstra wrote: > Hi Guys, > > Sergey Osokin sent me patches to add the standard > pthread_[gs]etconcurrency functions to our various threads > libraries. I reviewed them and they're OK. The functions don't do > anything significant, but they fill the need for this part of the > API. > > OK if I commit them this weekend? The changes don't change anything > else. They just add stuff. I'm about to implement them for real in libpthread. I'd appreciate you not adding them to that. I've got a slew of other changes that I want add to it very soon. They don't seem to make sense for libthr and libc_r unless it returns ENOTSUP. libthr is 1:1, so it is meaning less there as well as libc_r. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 09:39:54 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3AFE037B401; Wed, 16 Apr 2003 09:39:54 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5C58343F85; Wed, 16 Apr 2003 09:39:53 -0700 (PDT) (envelope-from arr@watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.9/8.12.9) with ESMTP id h3GGdgrD024768; Wed, 16 Apr 2003 12:39:42 -0400 (EDT) (envelope-from arr@watson.org) Received: from localhost (arr@localhost)h3GGdfsM024765; Wed, 16 Apr 2003 12:39:41 -0400 (EDT) (envelope-from arr@watson.org) X-Authentication-Warning: fledge.watson.org: arr owned process doing -bs Date: Wed, 16 Apr 2003 12:39:40 -0400 (EDT) From: "Andrew R. Reiter" To: Julian Elischer In-Reply-To: Message-ID: <20030416123901.W85450@fledge.watson.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: John Baldwin cc: threads@freebsd.org Subject: Re: struct thread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 16:39:54 -0000 On Wed, 16 Apr 2003, Julian Elischer wrote: : : :On Wed, 16 Apr 2003, Jeff Roberson wrote: : :> Is td_last_kse necessary? What about td_lastcpu? They don't really seem :> to be used. :td_last_kse and td_last_cpu were used in some experimental cpu affinity :code that I gave up on (i.e. ran out of time). :The idea was that the system would attempt to first schedule :the thread on teh cpu it was last on , and if not available, on teh kse :that it last ran on. I never removed the items but was hoping that :someone, seeing the names there would feel tempted to :implement affinity.. (Alfred mumbled about trying it). Yea, I do remember Alfred offering up a patch that implemented affinity. There was some debate on arch@ perhaps regarding it. : :> :> Also, td_locks is unused, although it would be nice to have it :> implemented. : :I think ithis is a jhb field : :> :> td_sleeplocks should be ifdefed with WITNESS. : :ditto : :> :> Cheers, :> Jeff :> :> _______________________________________________ :> freebsd-threads@freebsd.org mailing list :> http://lists.freebsd.org/mailman/listinfo/freebsd-threads :> To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" :> : :_______________________________________________ :freebsd-threads@freebsd.org mailing list :http://lists.freebsd.org/mailman/listinfo/freebsd-threads :To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" : -- Andrew R. Reiter arr@watson.org arr@FreeBSD.org From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 09:42:14 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6450337B401; Wed, 16 Apr 2003 09:42:14 -0700 (PDT) Received: from wall.polstra.com (wall-gw.polstra.com [206.213.73.130]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4AFCE43F85; Wed, 16 Apr 2003 09:42:11 -0700 (PDT) (envelope-from jdp@polstra.com) Received: from strings.polstra.com (strings.polstra.com [206.213.73.20]) by wall.polstra.com (8.12.3p2/8.12.3) with ESMTP id h3GGgAds022903; Wed, 16 Apr 2003 09:42:10 -0700 (PDT) (envelope-from jdp@polstra.com) Message-ID: X-Mailer: XFMail 1.5.1 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Wed, 16 Apr 2003 09:42:10 -0700 (PDT) Organization: Polstra & Co., Inc. From: John Polstra To: Daniel Eischen cc: deischen@freebsd.org cc: jeff@freebsd.org cc: freebsd-threads@freebsd.org Subject: Re: May I add pthread_[gs]etconcurrency to the threads libraries X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 16:42:14 -0000 Daniel Eischen wrote: > On Wed, 16 Apr 2003, John Polstra wrote: > >> Hi Guys, >> >> Sergey Osokin sent me patches to add the standard >> pthread_[gs]etconcurrency functions to our various threads >> libraries. I reviewed them and they're OK. The functions don't do >> anything significant, but they fill the need for this part of the >> API. >> >> OK if I commit them this weekend? The changes don't change anything >> else. They just add stuff. > > I'm about to implement them for real in libpthread. I'd appreciate > you not adding them to that. I've got a slew of other changes that > I want add to it very soon. OK, I'm glad I asked. :-) Please take what you can from Sergey's patch, and credit him appropriately: http://ozz.pp.ru/patches/patch-pthread.2 There's a man page in there you should be able to use, at least. > They don't seem to make sense for libthr and libc_r unless it > returns ENOTSUP. libthr is 1:1, so it is meaning less there > as well as libc_r. They're not allowed to return ENOTSUP. The Single Unix Specification says that if they're not implemented they must return success after doing nothing. John From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 09:43:00 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CC62437B401; Wed, 16 Apr 2003 09:43:00 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id B011543FCB; Wed, 16 Apr 2003 09:42:59 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h3GGgeC76025; Wed, 16 Apr 2003 12:42:40 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Wed, 16 Apr 2003 12:42:40 -0400 (EDT) From: Jeff Roberson To: Daniel Eischen In-Reply-To: Message-ID: <20030416123956.Q76635-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: deischen@freebsd.org cc: jeff@freebsd.org cc: freebsd-threads@freebsd.org cc: John Polstra Subject: Re: May I add pthread_[gs]etconcurrency to the threads libraries? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 16:43:01 -0000 On Wed, 16 Apr 2003, Daniel Eischen wrote: > On Wed, 16 Apr 2003, John Polstra wrote: > > > Hi Guys, > > > > Sergey Osokin sent me patches to add the standard > > pthread_[gs]etconcurrency functions to our various threads > > libraries. I reviewed them and they're OK. The functions don't do > > anything significant, but they fill the need for this part of the > > API. > > > > OK if I commit them this weekend? The changes don't change anything > > else. They just add stuff. > > I'm about to implement them for real in libpthread. I'd appreciate > you not adding them to that. I've got a slew of other changes that > I want add to it very soon. > > They don't seem to make sense for libthr and libc_r unless it > returns ENOTSUP. libthr is 1:1, so it is meaning less there > as well as libc_r. It is not meaningless. Please go ahead and implement stubs if you like. I intended to implement a thr_setconcurrency that these can eventually use. I was just going to implement it as a simple counter on the proc instead of using a full blown KSE structure just for this. 1:1 means one kernel thread for each user thread. It doesn't say anything about how many of those threads may run at a given time. It also doesn't force any scheduling scope or scheduling algorithm. Cheers, Jeff From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 10:09:48 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C8A6237B401; Wed, 16 Apr 2003 10:09:48 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 163C243F3F; Wed, 16 Apr 2003 10:09:48 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3GH9lBg021753; Wed, 16 Apr 2003 13:09:47 -0400 (EDT) Received: from localhost (eischen@localhost)h3GH9lWn021750; Wed, 16 Apr 2003 13:09:47 -0400 (EDT) Date: Wed, 16 Apr 2003 13:09:46 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 17:09:49 -0000 On Wed, 16 Apr 2003, Daniel Eischen wrote: > On Wed, 16 Apr 2003, David Xu wrote: > > > > > ----- Original Message ----- > > From: "Daniel Eischen" > > To: "David Xu" > > Cc: ; ""Craig Rodrigues"" > > Sent: Wednesday, April 16, 2003 10:58 AM > > Subject: Re: libpthread patch > > > > > I thought it might also be the UTS trying to interrupt > > > the thread (kse_thr_interruot) while it was in the kernel > > > (assuming the UTS did get the signal). > > > > Can you try a patch for kern_exit.c ? > > http://people.freebsd.org/~davidxu/kern_exit.c.diff Found it! The library's version of sigaction was not allowing a signal handler for SIGCHLD to be installed; it was leftover from libc_r. Process_Manager_Test now passes. The only test that doesn't pass now is Cached_Conn_Test. This is with your kernel (kern_exit.c.diff) patch applied. I have not tried it without your patch. I'll do some more testing and post an updated patch. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 11:06:25 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 93AEB37B401 for ; Wed, 16 Apr 2003 11:06:25 -0700 (PDT) Received: from mail.speakeasy.net (mail12.speakeasy.net [216.254.0.212]) by mx1.FreeBSD.org (Postfix) with ESMTP id DDD2343F85 for ; Wed, 16 Apr 2003 11:06:22 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 22457 invoked from network); 16 Apr 2003 18:06:28 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender )encrypted SMTP for ; 16 Apr 2003 18:06:28 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.8/8.12.8) with ESMTP id h3GI6JOv072878; Wed, 16 Apr 2003 14:06:20 -0400 (EDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.4 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Wed, 16 Apr 2003 14:06:22 -0400 (EDT) From: John Baldwin To: Julian Elischer cc: threads@freebsd.org Subject: Re: struct thread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 18:06:25 -0000 On 16-Apr-2003 Julian Elischer wrote: > > > On Wed, 16 Apr 2003, Jeff Roberson wrote: > >> Is td_last_kse necessary? What about td_lastcpu? They don't really seem >> to be used. > td_last_kse and td_last_cpu were used in some experimental cpu affinity > code that I gave up on (i.e. ran out of time). > The idea was that the system would attempt to first schedule > the thread on teh cpu it was last on , and if not available, on teh kse > that it last ran on. I never removed the items but was hoping that > someone, seeing the names there would feel tempted to > implement affinity.. (Alfred mumbled about trying it). kg_slpq isn't used either. >> >> Also, td_locks is unused, although it would be nice to have it >> implemented. > > I think ithis is a jhb field It used to be p_locks in struct proc used in the lockmgr code as the comment indicates and predates both SMPng and KSE: short td_locks; /* (k) DEBUG: lockmgr count of locks */ I have no attachment to it and it doesn't seem to be used anymore. >> td_sleeplocks should be ifdefed with WITNESS. > > ditto Is struct thread's size and layout part of the ABI as struct proc's is? I think it effectively is since libkvm is going to want to examine thread structures, so I'm afraid a variant sized struct thread would be a bad thing. Other than that, td_sleeplocks could be #ifdef WITNESS. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 11:16:34 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8CC3C37B40B; Wed, 16 Apr 2003 11:16:34 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id D47B243F93; Wed, 16 Apr 2003 11:16:29 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3GIGQBg001912; Wed, 16 Apr 2003 14:16:26 -0400 (EDT) Received: from localhost (eischen@localhost)h3GIGQYU001909; Wed, 16 Apr 2003 14:16:26 -0400 (EDT) Date: Wed, 16 Apr 2003 14:16:26 -0400 (EDT) From: Daniel Eischen To: Jeff Roberson In-Reply-To: <20030416123956.Q76635-100000@mail.chesapeake.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: deischen@freebsd.org cc: jeff@freebsd.org cc: freebsd-threads@freebsd.org cc: John Polstra Subject: Re: May I add pthread_[gs]etconcurrency to the threads libraries? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 18:16:34 -0000 On Wed, 16 Apr 2003, Jeff Roberson wrote: > On Wed, 16 Apr 2003, Daniel Eischen wrote: > > > On Wed, 16 Apr 2003, John Polstra wrote: > > > > > Hi Guys, > > > > > > Sergey Osokin sent me patches to add the standard > > > pthread_[gs]etconcurrency functions to our various threads > > > libraries. I reviewed them and they're OK. The functions don't do > > > anything significant, but they fill the need for this part of the > > > API. > > > > > > OK if I commit them this weekend? The changes don't change anything > > > else. They just add stuff. > > > > I'm about to implement them for real in libpthread. I'd appreciate > > you not adding them to that. I've got a slew of other changes that > > I want add to it very soon. > > > > They don't seem to make sense for libthr and libc_r unless it > > returns ENOTSUP. libthr is 1:1, so it is meaning less there > > as well as libc_r. > > It is not meaningless. Please go ahead and implement stubs if you like. > I intended to implement a thr_setconcurrency that these can eventually > use. I was just going to implement it as a simple counter on the proc > instead of using a full blown KSE structure just for this. > > 1:1 means one kernel thread for each user thread. It doesn't say anything > about how many of those threads may run at a given time. It also doesn't > force any scheduling scope or scheduling algorithm. OK, I was under the impression that libthr would always schedule threads (KSEs) to as many processors as possible. It'd be neat to eventually softly bind them (KSE's in general) too :-) -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 11:44:30 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0E0C837B401; Wed, 16 Apr 2003 11:44:30 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id B2DDE43FA3; Wed, 16 Apr 2003 11:44:26 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h3GIiNZ48121; Wed, 16 Apr 2003 14:44:23 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Wed, 16 Apr 2003 14:44:23 -0400 (EDT) From: Jeff Roberson To: Daniel Eischen In-Reply-To: Message-ID: <20030416144217.K76635-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: deischen@freebsd.org cc: jeff@freebsd.org cc: freebsd-threads@freebsd.org cc: John Polstra Subject: Re: May I add pthread_[gs]etconcurrency to the threads libraries? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 18:44:30 -0000 On Wed, 16 Apr 2003, Daniel Eischen wrote: > On Wed, 16 Apr 2003, Jeff Roberson wrote: > > On Wed, 16 Apr 2003, Daniel Eischen wrote: > > > > > On Wed, 16 Apr 2003, John Polstra wrote: > > > > > > > Hi Guys, > > > > > > > > Sergey Osokin sent me patches to add the standard > > > > pthread_[gs]etconcurrency functions to our various threads > > > > libraries. I reviewed them and they're OK. The functions don't do > > > > anything significant, but they fill the need for this part of the > > > > API. > > > > > > > > OK if I commit them this weekend? The changes don't change anything > > > > else. They just add stuff. > > > > > > I'm about to implement them for real in libpthread. I'd appreciate > > > you not adding them to that. I've got a slew of other changes that > > > I want add to it very soon. > > > > > > They don't seem to make sense for libthr and libc_r unless it > > > returns ENOTSUP. libthr is 1:1, so it is meaning less there > > > as well as libc_r. > > > > It is not meaningless. Please go ahead and implement stubs if you like. > > I intended to implement a thr_setconcurrency that these can eventually > > use. I was just going to implement it as a simple counter on the proc > > instead of using a full blown KSE structure just for this. > > > > 1:1 means one kernel thread for each user thread. It doesn't say anything > > about how many of those threads may run at a given time. It also doesn't > > force any scheduling scope or scheduling algorithm. > > OK, I was under the impression that libthr would always schedule > threads (KSEs) to as many processors as possible. It'd be neat > to eventually softly bind them (KSE's in general) too :-) > > -- > Dan Eischen > If by softly bind them you mean CPU affinity, ULE does this. It does not support hard binding at the moment. I think KSEs are overengineered as a mechanism for enforcing concurrency. Really a counter of running/runnable threads in the proc would suffice. I'm starting to warm up to the idea of moving kse out of the rest of the system. It could be confined to kern_mn.c or whatever it is eventually called. Cheers, Jeff From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 12:03:23 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2A82F37B401; Wed, 16 Apr 2003 12:03:23 -0700 (PDT) Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by mx1.FreeBSD.org (Postfix) with ESMTP id 12B1D43FAF; Wed, 16 Apr 2003 12:03:22 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by rwcrmhc51.attbi.com (rwcrmhc51) with ESMTP id <2003041619032105100hshlse>; Wed, 16 Apr 2003 19:03:21 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id MAA44581; Wed, 16 Apr 2003 12:03:20 -0700 (PDT) Date: Wed, 16 Apr 2003 12:03:18 -0700 (PDT) From: Julian Elischer To: Jeff Roberson In-Reply-To: <20030416144217.K76635-100000@mail.chesapeake.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: deischen@freebsd.org cc: jeff@freebsd.org cc: John Polstra cc: Daniel Eischen cc: freebsd-threads@freebsd.org Subject: Re: May I add pthread_[gs]etconcurrency to the threads libraries? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 19:03:23 -0000 On Wed, 16 Apr 2003, Jeff Roberson wrote: > > On Wed, 16 Apr 2003, Daniel Eischen wrote: > > > On Wed, 16 Apr 2003, Jeff Roberson wrote: > > > On Wed, 16 Apr 2003, Daniel Eischen wrote: > > > > > > > On Wed, 16 Apr 2003, John Polstra wrote: > > > > > > > > > Hi Guys, > > > > > > > > > > Sergey Osokin sent me patches to add the standard > > > > > pthread_[gs]etconcurrency functions to our various threads > > > > > libraries. I reviewed them and they're OK. The functions don't do > > > > > anything significant, but they fill the need for this part of the > > > > > API. > > > > > > > > > > OK if I commit them this weekend? The changes don't change anything > > > > > else. They just add stuff. > > > > > > > > I'm about to implement them for real in libpthread. I'd appreciate > > > > you not adding them to that. I've got a slew of other changes that > > > > I want add to it very soon. > > > > > > > > They don't seem to make sense for libthr and libc_r unless it > > > > returns ENOTSUP. libthr is 1:1, so it is meaning less there > > > > as well as libc_r. > > > > > > It is not meaningless. Please go ahead and implement stubs if you like. > > > I intended to implement a thr_setconcurrency that these can eventually > > > use. I was just going to implement it as a simple counter on the proc > > > instead of using a full blown KSE structure just for this. > > > > > > 1:1 means one kernel thread for each user thread. It doesn't say anything > > > about how many of those threads may run at a given time. It also doesn't > > > force any scheduling scope or scheduling algorithm. > > > > OK, I was under the impression that libthr would always schedule > > threads (KSEs) to as many processors as possible. It'd be neat > > to eventually softly bind them (KSE's in general) too :-) > > > > -- > > Dan Eischen > > > > If by softly bind them you mean CPU affinity, ULE does this. It does not > support hard binding at the moment. I think KSEs are overengineered as a > mechanism for enforcing concurrency. Really a counter of running/runnable > threads in the proc would suffice. > > I'm starting to warm up to the idea of moving kse out of the rest of the > system. It could be confined to kern_mn.c or whatever it is eventually > called. kern_kse.c :-) M:N threads will run on any scheduler that knows about threads. This goes along with what we were discussing about separating the threadding system from the scheduling system. KSE threading is a misnomer. ALmost everywhere you see the term "KSE" being used in userland threading discussion it shuold be replaced by: "upcall context" (or UPCTX?) which is closely connected with a UTS instance. Originally KSEs were a simple method a fairness. Not perfect but simple and effective. By limiting the KSEs to the number of CPUs one could limit the ability of a process to flood the run queue. While it was envisionned as a part of splitting up the proc structure, it SHOULD have been envisionned as part of the scheduler. Unfortunatly we hadn't come up with the idea of having a separate upcall-context structure yet, so it was doing double duty as that as well, which clouded the picture. When that was split out as part of the threading system, the KSE that was left was plainly part of the scheduling system. (and an implementation detail at that) If we had had an upcall-context structure to start with, it is likely that I would have first gone for a simpler threads--only (but unfair) scheduler, and KSEs would have been only created when someone tried to make it fairer. :-) > > Cheers, > Jeff > > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 12:07:03 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5711837B405; Wed, 16 Apr 2003 12:07:01 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 92C5C43FB1; Wed, 16 Apr 2003 12:07:00 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3GJ6xBg009350; Wed, 16 Apr 2003 15:06:59 -0400 (EDT) Received: from localhost (eischen@localhost)h3GJ6xAQ009346; Wed, 16 Apr 2003 15:06:59 -0400 (EDT) Date: Wed, 16 Apr 2003 15:06:59 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 19:07:03 -0000 On Wed, 16 Apr 2003, Daniel Eischen wrote: > On Wed, 16 Apr 2003, Daniel Eischen wrote: > > > On Wed, 16 Apr 2003, David Xu wrote: > > > > > > > > ----- Original Message ----- > > > From: "Daniel Eischen" > > > To: "David Xu" > > > Cc: ; ""Craig Rodrigues"" > > > Sent: Wednesday, April 16, 2003 10:58 AM > > > Subject: Re: libpthread patch > > > > > > > I thought it might also be the UTS trying to interrupt > > > > the thread (kse_thr_interruot) while it was in the kernel > > > > (assuming the UTS did get the signal). > > > > > > Can you try a patch for kern_exit.c ? > > > http://people.freebsd.org/~davidxu/kern_exit.c.diff > > Found it! The library's version of sigaction was not allowing > a signal handler for SIGCHLD to be installed; it was leftover > from libc_r. Process_Manager_Test now passes. The only test > that doesn't pass now is Cached_Conn_Test. This is with your > kernel (kern_exit.c.diff) patch applied. I have not tried > it without your patch. > > I'll do some more testing and post an updated patch. Heh, heh, cool! KDE and konqueror now work with libkse :-) -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 14:06:02 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 622F937B401; Wed, 16 Apr 2003 14:06:02 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7DBFC43F75; Wed, 16 Apr 2003 14:05:59 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3GL5wBg027558; Wed, 16 Apr 2003 17:05:58 -0400 (EDT) Received: from localhost (eischen@localhost)h3GL5w9i027553; Wed, 16 Apr 2003 17:05:58 -0400 (EDT) Date: Wed, 16 Apr 2003 17:05:58 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 21:06:02 -0000 There's a new patch available at: http://people.freebsd.org/~deischen/kse/libpthread.diffs This passes all the ACE tests that libc_r passes, with the exception of Cached_Conn_Test. It also seems to work with KDE, konqueror, kwrite, kmail, etc. I don't have mozilla built (and am dreading trying to), but it would be interesting to see if it works with that. If no-one has any objections, I'd like to commit this soon. I'll let David review and comment to it first. David, I didn't add critical regions to _thr_alloc() and _thr_free(). I think that whenever they are used, we are already in a critical region or operating on an upcall. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 14:07:14 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B494A37B401; Wed, 16 Apr 2003 14:07:14 -0700 (PDT) Received: from sccrmhc01.attbi.com (sccrmhc01.attbi.com [204.127.202.61]) by mx1.FreeBSD.org (Postfix) with ESMTP id BD1AF43FBF; Wed, 16 Apr 2003 14:07:13 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by sccrmhc01.attbi.com (sccrmhc01) with ESMTP id <2003041621071200100bsbs7e>; Wed, 16 Apr 2003 21:07:12 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA45538; Wed, 16 Apr 2003 14:07:10 -0700 (PDT) Date: Wed, 16 Apr 2003 14:07:09 -0700 (PDT) From: Julian Elischer To: Jeff Roberson In-Reply-To: <20030416144217.K76635-100000@mail.chesapeake.net> Message-ID: MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="0-1677771262-1050527229=:44009" cc: deischen@freebsd.org cc: jeff@freebsd.org cc: John Polstra cc: Daniel Eischen cc: freebsd-threads@freebsd.org Subject: Re: May I add pthread_[gs]etconcurrency to the threads libraries? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 21:07:15 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. Send mail to mime@docserver.cac.washington.edu for more info. --0-1677771262-1050527229=:44009 Content-Type: TEXT/PLAIN; charset=US-ASCII On Wed, 16 Apr 2003, Jeff Roberson wrote: > > I'm starting to warm up to the idea of moving kse out of the rest of the > system. It could be confined to kern_mn.c or whatever it is eventually > called. Jeff I'd like to start some small diffs that resemble the one included.. This one changes shched_clock to use a thread. I'd also like to commit a change to make KEF_IDLEKSE to TDF_IDLETHREAD as you suggested. Here I'm running with a file sys/ksevar.h that has teh kse structure in it. eventually it would be in teh scheduler .c file so I am not going to commit that change yet.. comments? Julian --0-1677771262-1050527229=:44009 Content-Type: TEXT/PLAIN; charset=US-ASCII; name=xxx Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Content-Disposition: attachment; filename=xxx SW5kZXg6IHN5cy9zY2hlZC5oDQo9PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09DQpS Q1MgZmlsZTogL3JlcG9zL3Byb2plY3RzL21pcnJvcmVkL2ZyZWVic2Qvc3Jj L3N5cy9zeXMvc2NoZWQuaCx2DQpyZXRyaWV2aW5nIHJldmlzaW9uIDEuNA0K ZGlmZiAtdSAtcjEuNCBzY2hlZC5oDQotLS0gc3lzL3NjaGVkLmgJMjAwMy8w NC8xMSAwMzozOTowNgkxLjQNCisrKyBzeXMvc2NoZWQuaAkyMDAzLzA0LzE2 IDIwOjU5OjUyDQpAQCAtNjgsNyArNjgsNyBAQA0KICAqLw0KIHZvaWQJc2No ZWRfYWRkKHN0cnVjdCBrc2UgKmtlKTsNCiBzdHJ1Y3Qga3NlICpzY2hlZF9j aG9vc2Uodm9pZCk7DQotdm9pZAlzY2hlZF9jbG9jayhzdHJ1Y3Qga3NlICpr ZSk7DQordm9pZAlzY2hlZF9jbG9jayhzdHJ1Y3QgdGhyZWFkICp0ZCk7DQog dm9pZAlzY2hlZF9leGl0X2tzZShzdHJ1Y3Qga3NlICprZSwgc3RydWN0IGtz ZSAqY2hpbGQpOw0KIHZvaWQJc2NoZWRfZm9ya19rc2Uoc3RydWN0IGtzZSAq a2UsIHN0cnVjdCBrc2UgKmNoaWxkKTsNCiB2b2lkCXNjaGVkX3JlbShzdHJ1 Y3Qga3NlICprZSk7DQpJbmRleDoga2Vybi9rZXJuX2Nsb2NrLmMNCj09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT0NClJDUyBmaWxlOiAvcmVwb3MvcHJvamVjdHMv bWlycm9yZWQvZnJlZWJzZC9zcmMvc3lzL2tlcm4va2Vybl9jbG9jay5jLHYN CnJldHJpZXZpbmcgcmV2aXNpb24gMS4xNTYNCmRpZmYgLXUgLXIxLjE1NiBr ZXJuX2Nsb2NrLmMNCi0tLSBrZXJuL2tlcm5fY2xvY2suYwkyMDAzLzA0LzEx IDAzOjM5OjA3CTEuMTU2DQorKysga2Vybi9rZXJuX2Nsb2NrLmMJMjAwMy8w NC8xNiAyMDo1OTo1Mg0KQEAgLTM1Nyw3ICszNTcsNiBAQA0KIAlzdHJ1Y3Qg cnVzYWdlICpydTsNCiAJc3RydWN0IHZtc3BhY2UgKnZtOw0KIAlzdHJ1Y3Qg dGhyZWFkICp0ZDsNCi0Jc3RydWN0IGtzZSAqa2U7DQogCXN0cnVjdCBwcm9j ICpwOw0KIAlsb25nIHJzczsNCiANCkBAIC0zNjUsNyArMzY0LDYgQEANCiAJ cCA9IHRkLT50ZF9wcm9jOw0KIA0KIAltdHhfbG9ja19zcGluX2ZsYWdzKCZz Y2hlZF9sb2NrLCBNVFhfUVVJRVQpOw0KLQlrZSA9IHRkLT50ZF9rc2U7DQog CWlmIChDTEtGX1VTRVJNT0RFKGZyYW1lKSkgew0KIAkJLyoNCiAJCSAqIENo YXJnZSB0aGUgdGltZSBhcyBhcHByb3ByaWF0ZS4NCkBAIC0zNzMsNyArMzcx LDcgQEANCiAJCWlmIChwLT5wX2ZsYWcgJiBQX1RIUkVBREVEKQ0KIAkJCXRo cmVhZF9zdGF0Y2xvY2soMSk7DQogCQlwLT5wX3V0aWNrcysrOw0KLQkJaWYg KGtlLT5rZV9rc2VncnAtPmtnX25pY2UgPiBOWkVSTykNCisJCWlmICh0ZC0+ dGRfa3NlZ3JwLT5rZ19uaWNlID4gTlpFUk8pDQogCQkJY3BfdGltZVtDUF9O SUNFXSsrOw0KIAkJZWxzZQ0KIAkJCWNwX3RpbWVbQ1BfVVNFUl0rKzsNCkBA IC00MDUsNyArNDAzLDcgQEANCiAJCX0NCiAJfQ0KIA0KLQlzY2hlZF9jbG9j ayhrZSk7DQorCXNjaGVkX2Nsb2NrKHRkKTsNCiANCiAJLyogVXBkYXRlIHJl c291cmNlIHVzYWdlIGludGVncmFscyBhbmQgbWF4aW11bXMuICovDQogCWlm ICgocHN0YXRzID0gcC0+cF9zdGF0cykgIT0gTlVMTCAmJg0KSW5kZXg6IGtl cm4vc2NoZWRfNGJzZC5jDQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09DQpSQ1Mg ZmlsZTogL3JlcG9zL3Byb2plY3RzL21pcnJvcmVkL2ZyZWVic2Qvc3JjL3N5 cy9rZXJuL3NjaGVkXzRic2QuYyx2DQpyZXRyaWV2aW5nIHJldmlzaW9uIDEu MTUNCmRpZmYgLXUgLXIxLjE1IHNjaGVkXzRic2QuYw0KLS0tIGtlcm4vc2No ZWRfNGJzZC5jCTIwMDMvMDQvMTEgMDM6Mzk6NDgJMS4xNQ0KKysrIGtlcm4v c2NoZWRfNGJzZC5jCTIwMDMvMDQvMTYgMjA6NTk6NTINCkBAIC00MzgsMTMg KzQzOCwxMyBAQA0KICAqIHJ1biBtdWNoIHJlY2VudGx5LCBhbmQgdG8gcm91 bmQtcm9iaW4gYW1vbmcgb3RoZXIgcHJvY2Vzc2VzLg0KICAqLw0KIHZvaWQN Ci1zY2hlZF9jbG9jayhzdHJ1Y3Qga3NlICprZSkNCitzY2hlZF9jbG9jayhz dHJ1Y3QgdGhyZWFkICp0ZCkNCiB7DQogCXN0cnVjdCBrc2VncnAgKmtnOw0K LQlzdHJ1Y3QgdGhyZWFkICp0ZDsNCisJc3RydWN0IGtzZSAqa2U7DQogDQot CWtnID0ga2UtPmtlX2tzZWdycDsNCi0JdGQgPSBrZS0+a2VfdGhyZWFkOw0K KwlrZyA9IHRkLT50ZF9rc2VncnA7DQorCWtlID0gdGQtPnRkX2tzZTsNCiAN CiAJa2UtPmtlX3NjaGVkLT5za2VfY3B0aWNrcysrOw0KIAlrZy0+a2dfZXN0 Y3B1ID0gRVNUQ1BVTElNKGtnLT5rZ19lc3RjcHUgKyAxKTsNCkluZGV4OiBr ZXJuL3NjaGVkX3VsZS5jDQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09DQpSQ1Mg ZmlsZTogL3JlcG9zL3Byb2plY3RzL21pcnJvcmVkL2ZyZWVic2Qvc3JjL3N5 cy9rZXJuL3NjaGVkX3VsZS5jLHYNCnJldHJpZXZpbmcgcmV2aXNpb24gMS4y OA0KZGlmZiAtdSAtcjEuMjggc2NoZWRfdWxlLmMNCi0tLSBrZXJuL3NjaGVk X3VsZS5jCTIwMDMvMDQvMTIgMjI6MzM6MjQJMS4yOA0KKysrIGtlcm4vc2No ZWRfdWxlLmMJMjAwMy8wNC8xNiAyMDo1OTo1Mg0KQEAgLTg1NSwxMSArODU1 LDExIEBADQogfQ0KIA0KIHZvaWQNCi1zY2hlZF9jbG9jayhzdHJ1Y3Qga3Nl ICprZSkNCitzY2hlZF9jbG9jayhzdHJ1Y3QgdGhyZWFkICp0ZCkNCiB7DQog CXN0cnVjdCBrc2VxICprc2VxOw0KIAlzdHJ1Y3Qga3NlZ3JwICprZzsNCi0J c3RydWN0IHRocmVhZCAqdGQ7DQorCXN0cnVjdCBrc2UgKmtlDQogI2lmIDAN CiAJc3RydWN0IGtzZSAqbmtlOw0KICNlbmRpZg0KQEAgLTg4MCw4ICs4ODAs OCBAQA0KIAkJCXRpY2tpbmNyID0gMTsNCiAJfQ0KIA0KLQl0ZCA9IGtlLT5r ZV90aHJlYWQ7DQotCWtnID0ga2UtPmtlX2tzZWdycDsNCisJa2UgPSB0ZC0+ dGRfa3NlOw0KKwlrZyA9IHRkLT50ZF9rc2VncnA7DQogDQogCW10eF9hc3Nl cnQoJnNjaGVkX2xvY2ssIE1BX09XTkVEKTsNCiAJS0FTU0VSVCgodGQgIT0g TlVMTCksICgic2NoZWRjbG9jazogbnVsbCB0aHJlYWQgcG9pbnRlciIpKTsN Cg== --0-1677771262-1050527229=:44009-- From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 14:18:36 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9825037B401; Wed, 16 Apr 2003 14:18:36 -0700 (PDT) Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1249F43FB1; Wed, 16 Apr 2003 14:18:36 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by rwcrmhc52.attbi.com (rwcrmhc52) with ESMTP id <2003041621183505200eotsje>; Wed, 16 Apr 2003 21:18:35 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA45636; Wed, 16 Apr 2003 14:18:34 -0700 (PDT) Date: Wed, 16 Apr 2003 14:18:33 -0700 (PDT) From: Julian Elischer To: Daniel Eischen In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: David Xu cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 21:18:36 -0000 On Wed, 16 Apr 2003, Daniel Eischen wrote: > There's a new patch available at: > > http://people.freebsd.org/~deischen/kse/libpthread.diffs > > This passes all the ACE tests that libc_r passes, with the > exception of Cached_Conn_Test. > > It also seems to work with KDE, konqueror, kwrite, kmail, etc. > I don't have mozilla built (and am dreading trying to), but > it would be interesting to see if it works with that. > > If no-one has any objections, I'd like to commit this > soon. I'll let David review and comment to it first. good going... a commit would be welcome. > > David, I didn't add critical regions to _thr_alloc() and > _thr_free(). I think that whenever they are used, we > are already in a critical region or operating on an upcall. > > -- > Dan Eischen > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 15:59:33 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0C2EB37B401 for ; Wed, 16 Apr 2003 15:59:33 -0700 (PDT) Received: from sccrmhc01.attbi.com (sccrmhc01.attbi.com [204.127.202.61]) by mx1.FreeBSD.org (Postfix) with ESMTP id 62B0B43F3F for ; Wed, 16 Apr 2003 15:59:32 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by sccrmhc01.attbi.com (sccrmhc01) with ESMTP id <2003041622593100100buh3le>; Wed, 16 Apr 2003 22:59:31 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id PAA46301; Wed, 16 Apr 2003 15:59:28 -0700 (PDT) Date: Wed, 16 Apr 2003 15:59:26 -0700 (PDT) From: Julian Elischer To: Jeff Roberson In-Reply-To: <20030416144217.K76635-100000@mail.chesapeake.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Patches for threads/scheduler abstraction. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 22:59:33 -0000 Jeff I am working on some patches to do teh abstraction. I suspect that you are also doing this.. I would like to send you what I have before we diverge too much. What I will have when I have done this is a system that does exactly what it does now, but with code moved around, and a new file kern_kse.c. This is a step towards whate we discussed but I'm worried that if we don't co-ordinate I'll be duplicating work that you are doing.. Julian From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 16:29:13 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EAF4637B401; Wed, 16 Apr 2003 16:29:13 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 33ACA43FE0; Wed, 16 Apr 2003 16:29:12 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from tiger (davidxu@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with SMTP id h3GNTAUp096888; Wed, 16 Apr 2003 16:29:11 -0700 (PDT) (envelope-from davidxu@freebsd.org) Message-ID: <004c01c30470$9e36ddf0$0701a8c0@tiger> From: "David Xu" To: "Daniel Eischen" References: Date: Thu, 17 Apr 2003 07:33:42 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: David Xu List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 23:29:14 -0000 ----- Original Message -----=20 From: "Daniel Eischen" To: "David Xu" Cc: Sent: Thursday, April 17, 2003 5:05 AM Subject: Re: libpthread patch > There's a new patch available at: >=20 > http://people.freebsd.org/~deischen/kse/libpthread.diffs >=20 > This passes all the ACE tests that libc_r passes, with the > exception of Cached_Conn_Test. >=20 > It also seems to work with KDE, konqueror, kwrite, kmail, etc. > I don't have mozilla built (and am dreading trying to), but > it would be interesting to see if it works with that. >=20 Cool! > If no-one has any objections, I'd like to commit this > soon. I'll let David review and comment to it first. >=20 > David, I didn't add critical regions to _thr_alloc() and > _thr_free(). I think that whenever they are used, we > are already in a critical region or operating on an upcall. >=20 Hmm, I don't like to put malloc calling under critical section, it is better to put it under a lock, otherwise this would cause dead=20 lock. suppose that an user thread is calling malloc(), and heap manager got malloc spinlock, then it does somethings and the thread is preempted by upcall from kernel, now UTS switches to another thread, that thread starts to call pthread_create, so UTS kernel enters a critical region = first, and calls malloc, this would cause dead lock, because UTS is under = critical region and no context switch could happen. Also I don't like thr_free under critical region, I think a GC thread is = still needed to recycle zombie thread and free extra memory, UTS kernel=20 should't be blocked by user thread. Despite this, I think the patch = should be committed.=20 David Xu > --=20 > Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 16:46:04 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B0D2137B401 for ; Wed, 16 Apr 2003 16:46:04 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 02F9843FA3 for ; Wed, 16 Apr 2003 16:46:04 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3GNk3Bg019771; Wed, 16 Apr 2003 19:46:03 -0400 (EDT) Received: from localhost (eischen@localhost)h3GNk2nG019768; Wed, 16 Apr 2003 19:46:02 -0400 (EDT) Date: Wed, 16 Apr 2003 19:46:02 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: <004c01c30470$9e36ddf0$0701a8c0@tiger> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 23:46:05 -0000 On Thu, 17 Apr 2003, David Xu wrote: > > ----- Original Message ----- > From: "Daniel Eischen" > To: "David Xu" > Cc: > Sent: Thursday, April 17, 2003 5:05 AM > Subject: Re: libpthread patch > > > > There's a new patch available at: > > > > http://people.freebsd.org/~deischen/kse/libpthread.diffs > > > > This passes all the ACE tests that libc_r passes, with the > > exception of Cached_Conn_Test. > > > > It also seems to work with KDE, konqueror, kwrite, kmail, etc. > > I don't have mozilla built (and am dreading trying to), but > > it would be interesting to see if it works with that. > > > > Cool! > > > If no-one has any objections, I'd like to commit this > > soon. I'll let David review and comment to it first. > > > > David, I didn't add critical regions to _thr_alloc() and > > _thr_free(). I think that whenever they are used, we > > are already in a critical region or operating on an upcall. > > > > Hmm, I don't like to put malloc calling under critical section, > it is better to put it under a lock, otherwise this would cause dead > lock. suppose that an user thread is calling malloc(), and heap manager > got malloc spinlock, then it does somethings and the thread is preempted > by upcall from kernel, now UTS switches to another thread, that thread > starts to call pthread_create, so UTS kernel enters a critical region first, > and calls malloc, this would cause dead lock, because UTS is under critical > region and no context switch could happen. Hmm, I see what you mean. We could put spinlock in critical region and that may solve the problem, but I eventually want to see spinlocks go away and replace the very few that we have in libc with mutexes. > Also I don't like thr_free under critical region, I think a GC thread is still > needed to recycle zombie thread and free extra memory, UTS kernel > should't be blocked by user thread. Despite this, I think the patch should > be committed. I'll work on adding the GC thread back in. I really wanted to get rid of it so that the KSE can exit when threadcount == 0, but now we've got to make allowances for the extra thread in the main KSEG. Keep looking at the patch for anything else you might see. We still need a way to deliver signals and look for async cancel points in CPU-bound threads. The attempt to add a signal frame with signalcontext() doesn't seem to work which is why it is commented out. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 16:52:10 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7128837B404 for ; Wed, 16 Apr 2003 16:52:10 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3655143F75 for ; Wed, 16 Apr 2003 16:52:09 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h3GNq4o14261; Wed, 16 Apr 2003 19:52:04 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Wed, 16 Apr 2003 19:52:04 -0400 (EDT) From: Jeff Roberson To: Julian Elischer In-Reply-To: Message-ID: <20030416195037.C76635-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: Patches for threads/scheduler abstraction. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2003 23:52:10 -0000 On Wed, 16 Apr 2003, Julian Elischer wrote: > > Jeff I am working on some patches to do teh abstraction. > I suspect that you are also doing this.. > I would like to send you what I have before we diverge too much. > > What I will have when I have done this is a system that does exactly > what it does now, but with code moved around, > and a new file kern_kse.c. Ok, go ahead and send it. > > This is a step towards whate we discussed but I'm worried that if we > don't co-ordinate I'll be duplicating work that you are doing.. I have been mildly disappointed with FreeBSD lately and have not put any effort into it. I will look over your patches though. > > Julian > > From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 18:21:28 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B56A037B401 for ; Wed, 16 Apr 2003 18:21:28 -0700 (PDT) Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3EF6F43F3F for ; Wed, 16 Apr 2003 18:21:28 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by rwcrmhc52.attbi.com (rwcrmhc52) with ESMTP id <2003041701212705200epo73e>; Thu, 17 Apr 2003 01:21:27 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id SAA47141; Wed, 16 Apr 2003 18:21:26 -0700 (PDT) Date: Wed, 16 Apr 2003 18:21:24 -0700 (PDT) From: Julian Elischer To: Jeff Roberson In-Reply-To: <20030416195037.C76635-100000@mail.chesapeake.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: Patches for threads/scheduler abstraction. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 01:21:29 -0000 On Wed, 16 Apr 2003, Jeff Roberson wrote: > > On Wed, 16 Apr 2003, Julian Elischer wrote: > > > > > Jeff I am working on some patches to do teh abstraction. > > I suspect that you are also doing this.. > > I would like to send you what I have before we diverge too much. > > > > What I will have when I have done this is a system that does exactly > > what it does now, but with code moved around, > > and a new file kern_kse.c. > > Ok, go ahead and send it. > > > > > This is a step towards whate we discussed but I'm worried that if we > > don't co-ordinate I'll be duplicating work that you are doing.. > > I have been mildly disappointed with FreeBSD lately and have not put any > effort into it. I will look over your patches though. OK I will send them shortly. These patches are NOT CLEAN by which I mean I have not done any tidying up yet. It is still in the "cat dragged this over the fence" state. However it does compile cleanly with only four files knowing what a kse is (Not counting ULE, I have not touched ULE yet) (and I would actualy rather that you did.. you understand it better). !!!!!!I have not tried to RUN this yet!!!!!! The files that know what a kse is are: ref3# grep ksevar.h kern/* kern/kern_switch.c:#include kern/kern_thread.c:#include kern/sched_4bsd.c:#include kern/sched_4bsd_kse.c:#include (note that kern_thr.c is not one of them) My plan is: eventually, sched_4bsd.c will only do the original BSD scheduling (except with threads instead of procsses) i.e. unfair, but simple and easy to benchmark on non-threaded apps. (or on threaded apps with no competing processes) sched_4bsd_kse.c will suck in the current contents of kern_switch.c and the KSE'd version of sched_4bsd.c. this will act like the current 4bsd scheduler WITH KSEs switch.c will go away (unless you want it as part of ULE). kern_thread.c will forget all about KSEs (but its not there yet). I'll send the patches in an hour or so.. My wife's calling :-/ > > > > > Julian > > > > > > From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 18:35:16 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2D34737B401 for ; Wed, 16 Apr 2003 18:35:16 -0700 (PDT) Received: from exchhz01.viatech.com.cn (ip-167-164-97-218.anlai.com [218.97.164.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id CEE9D43F85 for ; Wed, 16 Apr 2003 18:35:13 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from davidw2k (ip-240-1-168-192.rev.dyxnet.com [192.168.1.240]) by exchhz01.viatech.com.cn with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id HLDQQQ3R; Thu, 17 Apr 2003 09:20:44 +0800 Message-ID: <001501c30481$d1d7a9d0$f001a8c0@davidw2k> From: "David Xu" To: "Daniel Eischen" References: Date: Thu, 17 Apr 2003 09:36:51 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 01:35:16 -0000 ----- Original Message -----=20 From: "Daniel Eischen" To: "David Xu" Cc: Sent: Thursday, April 17, 2003 7:46 AM Subject: Re: libpthread patch > On Thu, 17 Apr 2003, David Xu wrote: >=20 > >=20 > > ----- Original Message -----=20 > > From: "Daniel Eischen" > > To: "David Xu" > > Cc: > > Sent: Thursday, April 17, 2003 5:05 AM > > Subject: Re: libpthread patch > >=20 > >=20 > > > There's a new patch available at: > > >=20 > > > http://people.freebsd.org/~deischen/kse/libpthread.diffs > > >=20 > > > This passes all the ACE tests that libc_r passes, with the > > > exception of Cached_Conn_Test. > > >=20 > > > It also seems to work with KDE, konqueror, kwrite, kmail, etc. > > > I don't have mozilla built (and am dreading trying to), but > > > it would be interesting to see if it works with that. > > >=20 > >=20 > > Cool! > >=20 > > > If no-one has any objections, I'd like to commit this > > > soon. I'll let David review and comment to it first. > > >=20 > > > David, I didn't add critical regions to _thr_alloc() and > > > _thr_free(). I think that whenever they are used, we > > > are already in a critical region or operating on an upcall. > > >=20 > >=20 > > Hmm, I don't like to put malloc calling under critical section, > > it is better to put it under a lock, otherwise this would cause dead = > > lock. suppose that an user thread is calling malloc(), and heap = manager > > got malloc spinlock, then it does somethings and the thread is = preempted > > by upcall from kernel, now UTS switches to another thread, that = thread > > starts to call pthread_create, so UTS kernel enters a critical = region first, > > and calls malloc, this would cause dead lock, because UTS is under = critical > > region and no context switch could happen. >=20 > Hmm, I see what you mean. We could put spinlock in critical region > and that may solve the problem, but I eventually want to see spinlocks > go away and replace the very few that we have in libc with mutexes. >=20 > > Also I don't like thr_free under critical region, I think a GC = thread is still > > needed to recycle zombie thread and free extra memory, UTS kernel=20 > > should't be blocked by user thread. Despite this, I think the patch = should > > be committed.=20 >=20 > I'll work on adding the GC thread back in. I really wanted to > get rid of it so that the KSE can exit when threadcount =3D=3D 0, > but now we've got to make allowances for the extra thread > in the main KSEG. >=20 If you don't want to use GC thread, you can free extra zoombie threads in thr_alloc(), everytime when thr_alloc is called, if it finds there is too many zoombie threads, free them. > Keep looking at the patch for anything else you might see. > We still need a way to deliver signals and look for async > cancel points in CPU-bound threads. The attempt to add a > signal frame with signalcontext() doesn't seem to work > which is why it is commented out. >=20 I will look. > --=20 > Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 19:03:29 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EF60237B401; Wed, 16 Apr 2003 19:03:29 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2757343FAF; Wed, 16 Apr 2003 19:03:29 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3H23SBg008134; Wed, 16 Apr 2003 22:03:28 -0400 (EDT) Received: from localhost (eischen@localhost)h3H23RlQ008131; Wed, 16 Apr 2003 22:03:28 -0400 (EDT) Date: Wed, 16 Apr 2003 22:03:27 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: <001501c30481$d1d7a9d0$f001a8c0@davidw2k> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 02:03:30 -0000 On Thu, 17 Apr 2003, David Xu wrote: > ----- Original Message ----- > From: "Daniel Eischen" > To: "David Xu" > Cc: > Sent: Thursday, April 17, 2003 7:46 AM > Subject: Re: libpthread patch [ Snip ] > If you don't want to use GC thread, you can free extra zoombie > threads in thr_alloc(), everytime when thr_alloc is called, if it > finds there is too many zoombie threads, free them. Neat; I think that would work. Thanks :-) -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 19:18:22 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3C97737B401 for ; Wed, 16 Apr 2003 19:18:22 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6627543FAF for ; Wed, 16 Apr 2003 19:18:21 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h3H2II991070; Wed, 16 Apr 2003 22:18:18 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Wed, 16 Apr 2003 22:18:17 -0400 (EDT) From: Jeff Roberson To: Julian Elischer In-Reply-To: Message-ID: <20030416221535.B76635-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: Patches for threads/scheduler abstraction. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 02:18:22 -0000 On Wed, 16 Apr 2003, Julian Elischer wrote: > > OK I will send them shortly. > > These patches are NOT CLEAN > by which I mean I have not done any tidying up yet. > > It is still in the "cat dragged this over the fence" state. > However it does compile cleanly with only four files knowing what a kse > is (Not counting ULE, I have not touched ULE yet) (and I would actualy > rather that you did.. you understand it better). > > !!!!!!I have not tried to RUN this yet!!!!!! > The files that know what a kse is are: > ref3# grep ksevar.h kern/* > kern/kern_switch.c:#include > kern/kern_thread.c:#include > kern/sched_4bsd.c:#include > kern/sched_4bsd_kse.c:#include > (note that kern_thr.c is not one of them) > > My plan is: > eventually, sched_4bsd.c will only do the original BSD > scheduling (except with threads instead of procsses) > i.e. unfair, but simple and easy to benchmark on non-threaded apps. > (or on threaded apps with no competing processes) > > sched_4bsd_kse.c will suck in the current contents of kern_switch.c > and the KSE'd version of sched_4bsd.c. > > this will act like the current 4bsd scheduler WITH KSEs > There is a better way to do this. If you would listen to my tiered approach you could end up with one copy of sched_4bsd and not two. The kse specific logic could just sit in kern_kse which would not tell sched_4bsd about threads until they owned the KSE. We could even do this without the kse structure as a go between. > switch.c will go away (unless you want it as part of ULE). > > kern_thread.c will forget all about KSEs > (but its not there yet). > > I'll send the patches in an hour or so.. > My wife's calling :-/ > > > > > > > > > > > > > > Julian > > > > > > > > > > > From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 20:21:49 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C6A6F37B41D for ; Wed, 16 Apr 2003 20:21:49 -0700 (PDT) Received: from stork.mail.pas.earthlink.net (stork.mail.pas.earthlink.net [207.217.120.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id 42E5343FBF for ; Wed, 16 Apr 2003 20:21:49 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0039.cvx40-bradley.dialup.earthlink.net ([216.244.42.39] helo=mindspring.com) by stork.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 195zxp-0007ck-00; Wed, 16 Apr 2003 20:21:30 -0700 Message-ID: <3E9E197A.6CD504E2@mindspring.com> Date: Wed, 16 Apr 2003 20:03:22 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Daniel Eischen References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a48534a8a8ca45d257b9394e9a1406c47e387f7b89c61deb1d350badd9bab72f9c350badd9bab72f9c cc: David Xu cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 03:21:50 -0000 Daniel Eischen wrote: > On Thu, 17 Apr 2003, David Xu wrote: > > Hmm, I don't like to put malloc calling under critical section, > > it is better to put it under a lock, otherwise this would cause dead > > lock. suppose that an user thread is calling malloc(), and heap manager > > got malloc spinlock, then it does somethings and the thread is preempted > > by upcall from kernel, now UTS switches to another thread, that thread > > starts to call pthread_create, so UTS kernel enters a critical region first, > > and calls malloc, this would cause dead lock, because UTS is under critical > > region and no context switch could happen. > > Hmm, I see what you mean. We could put spinlock in critical region > and that may solve the problem, but I eventually want to see spinlocks > go away and replace the very few that we have in libc with mutexes. Critical sectioning AND locking are a bad idea. They are orthogonal technologies which are intended to solve the same basic problems. You guys need to decide whether you are locking code paths or if you are locking data objects. Pick one and only one, and stick with it consistently, or you will shoot your feet. So far, the FreeBSD 5.0 locking has been pretty schitzophrenic. -- Terry From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 20:24:47 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5FCD937B401 for ; Wed, 16 Apr 2003 20:24:47 -0700 (PDT) Received: from stork.mail.pas.earthlink.net (stork.mail.pas.earthlink.net [207.217.120.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id E0E3E43F75 for ; Wed, 16 Apr 2003 20:24:46 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0039.cvx40-bradley.dialup.earthlink.net ([216.244.42.39] helo=mindspring.com) by stork.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 19600l-0000MI-00; Wed, 16 Apr 2003 20:24:32 -0700 Message-ID: <3E9E1E1F.29B8E197@mindspring.com> Date: Wed, 16 Apr 2003 20:23:11 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Daniel Eischen References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a48534a8a8ca45d2575002796f761f6543350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: David Xu cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 03:24:47 -0000 Daniel Eischen wrote: > On Thu, 17 Apr 2003, David Xu wrote: > > Hmm, I don't like to put malloc calling under critical section, > > it is better to put it under a lock, otherwise this would cause dead > > lock. suppose that an user thread is calling malloc(), and heap manager > > got malloc spinlock, then it does somethings and the thread is preempted > > by upcall from kernel, now UTS switches to another thread, that thread > > starts to call pthread_create, so UTS kernel enters a critical region first, > > and calls malloc, this would cause dead lock, because UTS is under critical > > region and no context switch could happen. > > Hmm, I see what you mean. We could put spinlock in critical region > and that may solve the problem, but I eventually want to see spinlocks > go away and replace the very few that we have in libc with mutexes. Critical sectioning AND locking are a bad idea. They are orthogonal technologies which are intended to solve the same basic problems. You guys need to decide whether you are locking code paths or if you are locking data objects. Pick one and only one, and stick with it consistently, or you will shoot your feet. So far, the FreeBSD 5.0 locking has been pretty schitzophrenic. -- Terry From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 22:43:39 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 759E337B401 for ; Wed, 16 Apr 2003 22:43:39 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id C040D43FDD for ; Wed, 16 Apr 2003 22:43:38 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3H5hcBg006838; Thu, 17 Apr 2003 01:43:38 -0400 (EDT) Received: from localhost (eischen@localhost)h3H5haDL006828; Thu, 17 Apr 2003 01:43:36 -0400 (EDT) Date: Thu, 17 Apr 2003 01:43:36 -0400 (EDT) From: Daniel Eischen To: Terry Lambert In-Reply-To: <3E9E197A.6CD504E2@mindspring.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: David Xu cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 05:43:39 -0000 On Wed, 16 Apr 2003, Terry Lambert wrote: > Daniel Eischen wrote: > > On Thu, 17 Apr 2003, David Xu wrote: > > > Hmm, I don't like to put malloc calling under critical section, > > > it is better to put it under a lock, otherwise this would cause dead > > > lock. suppose that an user thread is calling malloc(), and heap manager > > > got malloc spinlock, then it does somethings and the thread is preempted > > > by upcall from kernel, now UTS switches to another thread, that thread > > > starts to call pthread_create, so UTS kernel enters a critical region first, > > > and calls malloc, this would cause dead lock, because UTS is under critical > > > region and no context switch could happen. > > > > Hmm, I see what you mean. We could put spinlock in critical region > > and that may solve the problem, but I eventually want to see spinlocks > > go away and replace the very few that we have in libc with mutexes. > > Critical sectioning AND locking are a bad idea. They are > orthogonal technologies which are intended to solve the same > basic problems. The critical section is to prevent the thread from being swapped out by the kernel and sent to another KSE. That's it; it's not meant to do the same thing as locking. There's per-kse stuff that needs to be accessed that isn't correct if the thread gets run on another KSE, regardless of whether or not a lock protects it. If it wasn't possible for the kernel to send completed threads from one KSE to another (within the same KSE group), we probably wouldn't need critical sections (at least as currently implemented). -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Wed Apr 16 23:55:58 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5994E37B401 for ; Wed, 16 Apr 2003 23:55:58 -0700 (PDT) Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id 756A043F93 for ; Wed, 16 Apr 2003 23:55:57 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by rwcrmhc52.attbi.com (rwcrmhc52) with ESMTP id <2003041706555405200enuk6e>; Thu, 17 Apr 2003 06:55:54 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id XAA49182; Wed, 16 Apr 2003 23:55:51 -0700 (PDT) Date: Wed, 16 Apr 2003 23:55:50 -0700 (PDT) From: Julian Elischer To: Jeff Roberson In-Reply-To: <20030416221535.B76635-100000@mail.chesapeake.net> Message-ID: MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="0-499056429-1050562550=:94222" cc: freebsd-threads@freebsd.org Subject: Re: Patches for threads/scheduler abstraction. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 06:55:58 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. Send mail to mime@docserver.cac.washington.edu for more info. --0-499056429-1050562550=:94222 Content-Type: TEXT/PLAIN; charset=US-ASCII On Wed, 16 Apr 2003, Jeff Roberson wrote: > > On Wed, 16 Apr 2003, Julian Elischer wrote: > > > > > OK I will send them shortly. > > > > These patches are NOT CLEAN > > by which I mean I have not done any tidying up yet. > > > > It is still in the "cat dragged this over the fence" state. > > However it does compile cleanly with only four files knowing what a kse > > is (Not counting ULE, I have not touched ULE yet) (and I would actualy > > rather that you did.. you understand it better). > > > > !!!!!!I have not tried to RUN this yet!!!!!! > > The files that know what a kse is are: > > ref3# grep ksevar.h kern/* > > kern/kern_switch.c:#include > > kern/kern_thread.c:#include > > kern/sched_4bsd.c:#include > > kern/sched_4bsd_kse.c:#include > > (note that kern_thr.c is not one of them) > > > > My plan is: > > eventually, sched_4bsd.c will only do the original BSD > > scheduling (except with threads instead of procsses) > > i.e. unfair, but simple and easy to benchmark on non-threaded apps. > > (or on threaded apps with no competing processes) > > > > sched_4bsd_kse.c will suck in the current contents of kern_switch.c > > and the KSE'd version of sched_4bsd.c. > > > > this will act like the current 4bsd scheduler WITH KSEs > > > > There is a better way to do this. If you would listen to my tiered > approach you could end up with one copy of sched_4bsd and not two. > The kse specific logic could just sit in kern_kse which would not > tell sched_4bsd about threads until they owned the KSE. We could > even do this without the kse structure as a go between. I think that is an overcomplication. the two BSD based files would be very different. For a start the simple one would be queueing threads on the run queues. A system compiled with that scheduelr would have no KSEs anywhere in the entire kernel. The kse one would be queueing KSEs. I don't see how you can do this with a shared file. Anyhow, the following hack (totaly unoptimised.... notice the existance of sched_td_exit, sched_exit_thread, and sched_thr_exit.. I just haven't got to cleaning it) is not so advanced that the question of tiered schedulers is relevant yet.. this patch just shows that it is possible to get the KSEs out of the rst of the system. > > > switch.c will go away (unless you want it as part of ULE). > > > > kern_thread.c will forget all about KSEs > > (but its not there yet). > > > > I'll send the patches in an hour or so.. > > My wife's calling :-/ > > > > > > > > > > > > > > > > > > > > > > > Julian > > > > > > > > > > > > > > > > > > --0-499056429-1050562550=:94222 Content-Type: APPLICATION/octet-stream; name="hack.tgz" Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Content-Disposition: attachment; filename="hack.tgz" dW4ta3NlLmRpZmYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAADAxMDA2NDQAMDAwMDAwMAAwMDAwMDAwADAwMDAwMDc1MTUz ADA3NjQ3NDQ2MDAwADAxMTc1NAAgMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB1c3RhciAgAHJvb3QA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAd2hlZWwAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAA/IHN5cy9scy5nbW9uCj8gc3lzL2NvbmYvbHMu Z21vbgo/IHN5cy9pMzg2L2NvbmYvUkVGMwo/IHN5cy9pMzg2L2NvbmYvbHMu Z21vbgo/IHN5cy9rZXJuL3NjaGVkXzRic2Rfa3NlLmMKPyBzeXMvc3lzL2tz ZXZhci5oCj8gc3lzL3N5cy9scy5nbW9uCkluZGV4OiBzeXMvY29uZi9maWxl cwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09ClJDUyBmaWxlOiAvcmVwb3MvcHJv amVjdHMvbWlycm9yZWQvZnJlZWJzZC9zcmMvc3lzL2NvbmYvZmlsZXMsdgpy ZXRyaWV2aW5nIHJldmlzaW9uIDEuNzgyCmRpZmYgLXUgLXIxLjc4MiBmaWxl cwotLS0gc3lzL2NvbmYvZmlsZXMJMjAwMy8wNC8xNSAwNDowODowMQkxLjc4 MgorKysgc3lzL2NvbmYvZmlsZXMJMjAwMy8wNC8xNyAwNjozNDo1MApAQCAt MTA1Myw2ICsxMDUzLDcgQEAKIGtlcm4vbWQ0Yy5jCQlvcHRpb25hbCBuZXRz bWIKIGtlcm4vbWQ1Yy5jCQlzdGFuZGFyZAoga2Vybi9zY2hlZF80YnNkLmMJ b3B0aW9uYWwgc2NoZWRfNGJzZAora2Vybi9zY2hlZF80YnNkX2tzZS5jCW9w dGlvbmFsIHNjaGVkXzRic2QKIGtlcm4vc2NoZWRfdWxlLmMJb3B0aW9uYWwg c2NoZWRfdWxlCiBrZXJuL3N1YnJfYXV0b2NvbmYuYwlzdGFuZGFyZAoga2Vy bi9zdWJyX2JsaXN0LmMJc3RhbmRhcmQKSW5kZXg6IHN5cy9pMzg2L2kzODYv bWFjaGRlcC5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KUkNTIGZpbGU6IC9y ZXBvcy9wcm9qZWN0cy9taXJyb3JlZC9mcmVlYnNkL3NyYy9zeXMvaTM4Ni9p Mzg2L21hY2hkZXAuYyx2CnJldHJpZXZpbmcgcmV2aXNpb24gMS41NjEKZGlm ZiAtdSAtcjEuNTYxIG1hY2hkZXAuYwotLS0gc3lzL2kzODYvaTM4Ni9tYWNo ZGVwLmMJMjAwMy8wNC8wMiAyMzo1MzoyOAkxLjU2MQorKysgc3lzL2kzODYv aTM4Ni9tYWNoZGVwLmMJMjAwMy8wNC8xNyAwNjozNDo1NApAQCAtMTkzMSwx MSArMTkzMSw3IEBACiAJICAgKHRocmVhZDAudGRfa3N0YWNrICsgS1NUQUNL X1BBR0VTICogUEFHRV9TSVpFKSAtIDE7CiAJYXRkZXZiYXNlID0gSVNBX0hP TEVfU1RBUlQgKyBLRVJOQkFTRTsKIAotCS8qCi0gCSAqIFRoaXMgbWF5IGJl IGRvbmUgYmV0dGVyIGxhdGVyIGlmIGl0IGdldHMgbW9yZSBoaWdoIGxldmVs Ci0gCSAqIGNvbXBvbmVudHMgaW4gaXQuIElmIHNvIGp1c3QgbGluayB0ZC0+ dGRfcHJvYyBoZXJlLgotCSAqLwotCXByb2NfbGlua3VwKCZwcm9jMCwgJmtz ZWdycDAsICZrc2UwLCAmdGhyZWFkMCk7CisJdGhyZWFkMC50ZF9wcm9jID0g JnByb2MwOyAvKiBzb21lIHN0dWZmIGFzc3VtZXMgdGhpcyBpcyBhbHJlYWR5 IGRvbmUgKi8KIAogCW1ldGFkYXRhX21pc3NpbmcgPSAwOwogCWlmIChib290 aW5mby5iaV9tb2R1bGVwKSB7CkluZGV4OiBzeXMva2Vybi9pbml0X21haW4u Ywo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09ClJDUyBmaWxlOiAvcmVwb3MvcHJv amVjdHMvbWlycm9yZWQvZnJlZWJzZC9zcmMvc3lzL2tlcm4vaW5pdF9tYWlu LmMsdgpyZXRyaWV2aW5nIHJldmlzaW9uIDEuMjI5CmRpZmYgLXUgLXIxLjIy OSBpbml0X21haW4uYwotLS0gc3lzL2tlcm4vaW5pdF9tYWluLmMJMjAwMy8w NC8xMyAyMToyOToxMAkxLjIyOQorKysgc3lzL2tlcm4vaW5pdF9tYWluLmMJ MjAwMy8wNC8xNyAwNjozNDo1NQpAQCAtODgsNyArODgsNiBAQAogc3RhdGlj IHN0cnVjdCBwZ3JwIHBncnAwOwogc3RydWN0CXByb2MgcHJvYzA7CiBzdHJ1 Y3QJdGhyZWFkIHRocmVhZDA7Ci1zdHJ1Y3QJa3NlIGtzZTA7CiBzdHJ1Y3QJ a3NlZ3JwIGtzZWdycDA7CiBzdGF0aWMgc3RydWN0IHByb2NzaWcgcHJvY3Np ZzA7CiBzdGF0aWMgc3RydWN0IGZpbGVkZXNjMCBmaWxlZGVzYzA7CkBAIC0z MDgsMTggKzMwNywxMiBAQAogCXJlZ2lzdGVyIHVuc2lnbmVkIGk7CiAJc3Ry dWN0IHRocmVhZCAqdGQ7CiAJc3RydWN0IGtzZWdycCAqa2c7Ci0Jc3RydWN0 IGtzZSAqa2U7CiAKIAlHSUFOVF9SRVFVSVJFRDsKIAlwID0gJnByb2MwOwog CXRkID0gJnRocmVhZDA7Ci0Ja2UgPSAma3NlMDsKIAlrZyA9ICZrc2VncnAw OwogCi0Ja2UtPmtlX3NjaGVkID0ga3NlMF9zY2hlZDsKLQlrZy0+a2dfc2No ZWQgPSBrc2VncnAwX3NjaGVkOwotCXAtPnBfc2NoZWQgPSBwcm9jMF9zY2hl ZDsKLQl0ZC0+dGRfc2NoZWQgPSB0aHJlYWQwX3NjaGVkOwogCiAJLyoKIAkg KiBJbml0aWFsaXplIG1hZ2ljIG51bWJlci4KQEAgLTMyOSw5ICszMjIsMTEg QEAKIAkvKgogCSAqIEluaXRpYWxpemUgdGhyZWFkLCBwcm9jZXNzIGFuZCBw Z3JwIHN0cnVjdHVyZXMuCiAJICovCi0JcHJvY2luaXQoKTsKLQl0aHJlYWRp bml0KCk7CisJcHJvY2luaXQoKTsJCS8qIHNldCB1cCBwcm9jIHpvbmUgKi8K Kwl0aHJlYWRpbml0KCk7CQkvKiBzZXQgdXAgdGhyZWFkLCB1cGNhbGwgYW5k IEtTRUdSUCB6b25lcyAqLworCXNjaGVkaW5pdCgpOwkJLyogc2V0IHVwIHpv bmVzIGV0Yy4gZm9yIHRoZSBzY2hlZHVsZXIgKi8KIAorCiAJLyoKIAkgKiBJ bml0aWFsaXplIHNsZWVwIHF1ZXVlIGhhc2ggdGFibGUKIAkgKi8KQEAgLTM2 MCwxMiArMzU1LDcgQEAKIAogCXAtPnBfc3lzZW50ID0gJm51bGxfc3lzdmVj OwogCi0JLyoKLQkgKiBwcm9jX2xpbmt1cCB3YXMgYWxyZWFkeSBkb25lIGlu IGluaXRfaTM4NigpIG9yIGFscGhhaW5pdCgpIGV0Yy4KLQkgKiBiZWNhdXNl IHRoZSBlYXJsaWVyIGNvZGUgbmVlZGVkIHRvIGZvbGxvdyB0ZC0+dGRfcHJv Yy4gT3RoZXJ3aXNlCi0JICogSSB3b3VsZCBoYXZlIGRvbmUgaXQgaGVyZS4u IG1heWJlIHRoaXMgbWVhbnMgdGhpcyBzaG91bGQgYmUKLQkgKiBkb25lIGVh cmxpZXIgdG9vLgotCSAqLworCXByb2NfbGlua3VwKCZwcm9jMCwgJmtzZWdy cDAsICZ0aHJlYWQwKTsKIAlwLT5wX2ZsYWcgPSBQX1NZU1RFTTsKIAlwLT5w X3NmbGFnID0gUFNfSU5NRU07CiAJcC0+cF9zdGF0ZSA9IFBSU19OT1JNQUw7 CkBAIC0zNzUsMTAgKzM2NSw3IEBACiAJa2ctPmtnX3VzZXJfcHJpID0gUFVT RVI7CiAJdGQtPnRkX3ByaW9yaXR5ID0gUFZNOwogCXRkLT50ZF9iYXNlX3By aSA9IFBVU0VSOwotCXRkLT50ZF9rc2UgPSBrZTsgLyogWFhYS1NFICovCiAJ dGQtPnRkX29uY3B1ID0gMDsKLQlrZS0+a2Vfc3RhdGUgPSBLRVNfVEhSRUFE OwotCWtlLT5rZV90aHJlYWQgPSB0ZDsKIAlwLT5wX3BlZXJzID0gMDsKIAlw LT5wX2xlYWRlciA9IHA7CiAKSW5kZXg6IHN5cy9rZXJuL2tlcm5fY2xvY2su Ywo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09ClJDUyBmaWxlOiAvcmVwb3MvcHJv amVjdHMvbWlycm9yZWQvZnJlZWJzZC9zcmMvc3lzL2tlcm4va2Vybl9jbG9j ay5jLHYKcmV0cmlldmluZyByZXZpc2lvbiAxLjE1NgpkaWZmIC11IC1yMS4x NTYga2Vybl9jbG9jay5jCi0tLSBzeXMva2Vybi9rZXJuX2Nsb2NrLmMJMjAw My8wNC8xMSAwMzozOTowNwkxLjE1NgorKysgc3lzL2tlcm4va2Vybl9jbG9j ay5jCTIwMDMvMDQvMTcgMDY6MzQ6NTUKQEAgLTM1Nyw3ICszNTcsNiBAQAog CXN0cnVjdCBydXNhZ2UgKnJ1OwogCXN0cnVjdCB2bXNwYWNlICp2bTsKIAlz dHJ1Y3QgdGhyZWFkICp0ZDsKLQlzdHJ1Y3Qga3NlICprZTsKIAlzdHJ1Y3Qg cHJvYyAqcDsKIAlsb25nIHJzczsKIApAQCAtMzY1LDcgKzM2NCw2IEBACiAJ cCA9IHRkLT50ZF9wcm9jOwogCiAJbXR4X2xvY2tfc3Bpbl9mbGFncygmc2No ZWRfbG9jaywgTVRYX1FVSUVUKTsKLQlrZSA9IHRkLT50ZF9rc2U7CiAJaWYg KENMS0ZfVVNFUk1PREUoZnJhbWUpKSB7CiAJCS8qCiAJCSAqIENoYXJnZSB0 aGUgdGltZSBhcyBhcHByb3ByaWF0ZS4KQEAgLTM3Myw3ICszNzEsNyBAQAog CQlpZiAocC0+cF9mbGFnICYgUF9USFJFQURFRCkKIAkJCXRocmVhZF9zdGF0 Y2xvY2soMSk7CiAJCXAtPnBfdXRpY2tzKys7Ci0JCWlmIChrZS0+a2Vfa3Nl Z3JwLT5rZ19uaWNlID4gTlpFUk8pCisJCWlmICh0ZC0+dGRfa3NlZ3JwLT5r Z19uaWNlID4gTlpFUk8pCiAJCQljcF90aW1lW0NQX05JQ0VdKys7CiAJCWVs c2UKIAkJCWNwX3RpbWVbQ1BfVVNFUl0rKzsKQEAgLTQwNSw3ICs0MDMsNyBA QAogCQl9CiAJfQogCi0Jc2NoZWRfY2xvY2soa2UpOworCXNjaGVkX2Nsb2Nr KHRkKTsKIAogCS8qIFVwZGF0ZSByZXNvdXJjZSB1c2FnZSBpbnRlZ3JhbHMg YW5kIG1heGltdW1zLiAqLwogCWlmICgocHN0YXRzID0gcC0+cF9zdGF0cykg IT0gTlVMTCAmJgpJbmRleDogc3lzL2tlcm4va2Vybl9mb3JrLmMKPT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PQpSQ1MgZmlsZTogL3JlcG9zL3Byb2plY3RzL21p cnJvcmVkL2ZyZWVic2Qvc3JjL3N5cy9rZXJuL2tlcm5fZm9yay5jLHYKcmV0 cmlldmluZyByZXZpc2lvbiAxLjE5MQpkaWZmIC11IC1yMS4xOTEga2Vybl9m b3JrLmMKLS0tIHN5cy9rZXJuL2tlcm5fZm9yay5jCTIwMDMvMDQvMTEgMDM6 Mzk6MDcJMS4xOTEKKysrIHN5cy9rZXJuL2tlcm5fZm9yay5jCTIwMDMvMDQv MTcgMDY6MzQ6NTUKQEAgLTIxNyw3ICsyMTcsNiBAQAogCXN0cnVjdCBmaWxl ZGVzYyAqZmQ7CiAJc3RydWN0IHByb2MgKnAxID0gdGQtPnRkX3Byb2M7CiAJ c3RydWN0IHRocmVhZCAqdGQyOwotCXN0cnVjdCBrc2UgKmtlMjsKIAlzdHJ1 Y3Qga3NlZ3JwICprZzI7CiAJc3RydWN0IHNpZ2FjdHMgKm5ld3NpZ2FjdHM7 CiAJc3RydWN0IHByb2NzaWcgKm5ld3Byb2NzaWc7CkBAIC00NDgsNyArNDQ3 LDYgQEAKIAkgKi8KIAl0ZDIgPSBGSVJTVF9USFJFQURfSU5fUFJPQyhwMik7 CiAJa2cyID0gRklSU1RfS1NFR1JQX0lOX1BST0MocDIpOwotCWtlMiA9IEZJ UlNUX0tTRV9JTl9LU0VHUlAoa2cyKTsKIAogCS8qIEFsbG9jYXRlIGFuZCBz d2l0Y2ggdG8gYW4gYWx0ZXJuYXRlIGtzdGFjayBpZiBzcGVjaWZpZWQgKi8K IAlpZiAocGFnZXMgIT0gMCkKQEAgLTQ1OCw4ICs0NTYsNiBAQAogCiAJYnpl cm8oJnAyLT5wX3N0YXJ0emVybywKIAkgICAgKHVuc2lnbmVkKSBSQU5HRU9G KHN0cnVjdCBwcm9jLCBwX3N0YXJ0emVybywgcF9lbmR6ZXJvKSk7Ci0JYnpl cm8oJmtlMi0+a2Vfc3RhcnR6ZXJvLAotCSAgICAodW5zaWduZWQpIFJBTkdF T0Yoc3RydWN0IGtzZSwga2Vfc3RhcnR6ZXJvLCBrZV9lbmR6ZXJvKSk7CiAJ Ynplcm8oJnRkMi0+dGRfc3RhcnR6ZXJvLAogCSAgICAodW5zaWduZWQpIFJB TkdFT0Yoc3RydWN0IHRocmVhZCwgdGRfc3RhcnR6ZXJvLCB0ZF9lbmR6ZXJv KSk7CiAJYnplcm8oJmtnMi0+a2dfc3RhcnR6ZXJvLApAQCAtNDc3LDExICs0 NzMsNiBAQAogCSAgICAodW5zaWduZWQpIFJBTkdFT0Yoc3RydWN0IGtzZWdy cCwga2dfc3RhcnRjb3B5LCBrZ19lbmRjb3B5KSk7CiAjdW5kZWYgUkFOR0VP RgogCi0JLyogU2V0IHVwIHRoZSB0aHJlYWQgYXMgYW4gYWN0aXZlIHRocmVh ZCAoYXMgaWYgcnVubmFibGUpLiAqLwotCWtlMi0+a2Vfc3RhdGUgPSBLRVNf VEhSRUFEOwotCWtlMi0+a2VfdGhyZWFkID0gdGQyOwotCXRkMi0+dGRfa3Nl ID0ga2UyOwotCiAJLyoKIAkgKiBEdXBsaWNhdGUgc3ViLXN0cnVjdHVyZXMg YXMgbmVlZGVkLgogCSAqIEluY3JlYXNlIHJlZmVyZW5jZSBjb3VudHMgb24g c2hhcmVkIG9iamVjdHMuCkBAIC00OTYsNyArNDg3LDcgQEAKIAkgKiBBbGxv dyB0aGUgc2NoZWR1bGVyIHRvIGFkanVzdCB0aGUgcHJpb3JpdHkgb2YgdGhl IGNoaWxkIGFuZAogCSAqIHBhcmVudCB3aGlsZSB3ZSBob2xkIHRoZSBzY2hl ZF9sb2NrLgogCSAqLwotCXNjaGVkX2ZvcmsocDEsIHAyKTsKKwlzY2hlZF9m b3JrKHRkLCBwMik7CiAKIAltdHhfdW5sb2NrX3NwaW4oJnNjaGVkX2xvY2sp OwogCXAyLT5wX3VjcmVkID0gY3Job2xkKHRkLT50ZF91Y3JlZCk7CkluZGV4 OiBzeXMva2Vybi9rZXJuX2lkbGUuYwo9PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 ClJDUyBmaWxlOiAvcmVwb3MvcHJvamVjdHMvbWlycm9yZWQvZnJlZWJzZC9z cmMvc3lzL2tlcm4va2Vybl9pZGxlLmMsdgpyZXRyaWV2aW5nIHJldmlzaW9u IDEuMjkKZGlmZiAtdSAtcjEuMjkga2Vybl9pZGxlLmMKLS0tIHN5cy9rZXJu L2tlcm5faWRsZS5jCTIwMDIvMTAvMTIgMDU6MzI6MjMJMS4yOQorKysgc3lz L2tlcm4va2Vybl9pZGxlLmMJMjAwMy8wNC8xNyAwNjozNDo1NQpAQCAtNjUs NyArNjUsNyBAQAogCQlwLT5wX3N0YXRlID0gUFJTX05PUk1BTDsKIAkJdGQg PSBGSVJTVF9USFJFQURfSU5fUFJPQyhwKTsKIAkJdGQtPnRkX3N0YXRlID0g VERTX0NBTl9SVU47Ci0JCXRkLT50ZF9rc2UtPmtlX2ZsYWdzIHw9IEtFRl9J RExFS1NFOyAKKwkJdGQtPnRkX2ZsYWdzIHw9IFRERl9JRExFVEQ7IAogI2lm ZGVmIFNNUAogCX0KICNlbmRpZgpJbmRleDogc3lzL2tlcm4va2Vybl9pbnRy LmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PQpSQ1MgZmlsZTogL3JlcG9zL3By b2plY3RzL21pcnJvcmVkL2ZyZWVic2Qvc3JjL3N5cy9rZXJuL2tlcm5faW50 ci5jLHYKcmV0cmlldmluZyByZXZpc2lvbiAxLjkxCmRpZmYgLXUgLXIxLjkx IGtlcm5faW50ci5jCi0tLSBzeXMva2Vybi9rZXJuX2ludHIuYwkyMDAzLzAz LzA0IDIxOjAxOjQyCTEuOTEKKysrIHN5cy9rZXJuL2tlcm5faW50ci5jCTIw MDMvMDQvMTcgMDY6MzQ6NTUKQEAgLTM5Niw3ICszOTYsNyBAQAogCQkJS0FT U0VSVCgoVERfSVNfUlVOTklORyhjdGQpKSwKIAkJCSAgICAoIml0aHJlYWRf c2NoZWR1bGU6IEJhZCBzdGF0ZSBmb3IgY3VydGhyZWFkLiIpKTsKIAkJCWN0 ZC0+dGRfcHJvYy0+cF9zdGF0cy0+cF9ydS5ydV9uaXZjc3crKzsKLQkJCWlm IChjdGQtPnRkX2tzZS0+a2VfZmxhZ3MgJiBLRUZfSURMRUtTRSkKKwkJCWlm IChjdGQtPnRkX2ZsYWdzICYgVERGX0lETEVURCkKIAkJCQljdGQtPnRkX3N0 YXRlID0gVERTX0NBTl9SVU47IC8qIFhYWEtTRSAqLwogCQkJbWlfc3dpdGNo KCk7CiAJCX0gZWxzZSB7CkluZGV4OiBzeXMva2Vybi9rZXJuX2t0ci5jCj09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT0KUkNTIGZpbGU6IC9yZXBvcy9wcm9qZWN0 cy9taXJyb3JlZC9mcmVlYnNkL3NyYy9zeXMva2Vybi9rZXJuX2t0ci5jLHYK cmV0cmlldmluZyByZXZpc2lvbiAxLjM1CmRpZmYgLXUgLXIxLjM1IGtlcm5f a3RyLmMKLS0tIHN5cy9rZXJuL2tlcm5fa3RyLmMJMjAwMy8wMy8xMSAxOTo1 NjoxNgkxLjM1CisrKyBzeXMva2Vybi9rZXJuX2t0ci5jCTIwMDMvMDQvMTcg MDY6MzQ6NTUKQEAgLTE5NSw3ICsxOTUsNyBAQAogI2lmZGVmIEtUUl9BTFEK IAlpZiAoa3RyX2FscV9lbmFibGVkICYmCiAJICAgIHRkLT50ZF9jcml0bmVz dCA9PSAwICYmCi0JICAgICh0ZC0+dGRfa3NlLT5rZV9mbGFncyAmIEtFRl9J RExFS1NFKSA9PSAwICYmCisJICAgICh0ZC0+dGRfZmxhZ3MgJiBUREZfSURM RVREKSA9PSAwICYmCiAJICAgIHRkICE9IGFsZF90aHJlYWQpIHsKIAkJaWYg KGt0cl9hbHFfbWF4ICYmIGt0cl9hbHFfY250ID4ga3RyX2FscV9tYXgpCiAJ CQlnb3RvIGRvbmU7CkluZGV4OiBzeXMva2Vybi9rZXJuX3Byb2MuYwo9PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09ClJDUyBmaWxlOiAvcmVwb3MvcHJvamVjdHMv bWlycm9yZWQvZnJlZWJzZC9zcmMvc3lzL2tlcm4va2Vybl9wcm9jLmMsdgpy ZXRyaWV2aW5nIHJldmlzaW9uIDEuMTc4CmRpZmYgLXUgLXIxLjE3OCBrZXJu X3Byb2MuYwotLS0gc3lzL2tlcm4va2Vybl9wcm9jLmMJMjAwMy8wNC8xMCAx NzozNTo0NAkxLjE3OAorKysgc3lzL2tlcm4va2Vybl9wcm9jLmMJMjAwMy8w NC8xNyAwNjozNDo1NQpAQCAtMTQ4LDE4ICsxNDgsMTcgQEAKIAlzdHJ1Y3Qg cHJvYyAqcDsKIAlzdHJ1Y3QgdGhyZWFkICp0ZDsKIAlzdHJ1Y3Qga3NlZ3Jw ICprZzsKLQlzdHJ1Y3Qga3NlICprZTsKIAogCS8qIElOVkFSSUFOVFMgY2hl Y2tzIGdvIGhlcmUgKi8KIAlwID0gKHN0cnVjdCBwcm9jICopbWVtOworCXRk ID0gRklSU1RfVEhSRUFEX0lOX1BST0MocCk7CiAJS0FTU0VSVCgocC0+cF9u dW10aHJlYWRzID09IDEpLAogCSAgICAoImJhZCBudW1iZXIgb2YgdGhyZWFk cyBpbiBleGl0aW5nIHByb2Nlc3MiKSk7Ci0gICAgICAgIHRkID0gRklSU1Rf VEhSRUFEX0lOX1BST0MocCk7CisjaWZkZWYgSU5WQVJJQU5UUwogCUtBU1NF UlQoKHRkICE9IE5VTEwpLCAoInByb2NfZHRvcjogYmFkIHRocmVhZCBwb2lu dGVyIikpOwogICAgICAgICBrZyA9IEZJUlNUX0tTRUdSUF9JTl9QUk9DKHAp OwogCUtBU1NFUlQoKGtnICE9IE5VTEwpLCAoInByb2NfZHRvcjogYmFkIGtn IHBvaW50ZXIiKSk7Ci0gICAgICAgIGtlID0gRklSU1RfS1NFX0lOX0tTRUdS UChrZyk7Ci0JS0FTU0VSVCgoa2UgIT0gTlVMTCksICgicHJvY19kdG9yOiBi YWQga2UgcG9pbnRlciIpKTsKKyNlbmRpZgogCiAJLyogRGlzcG9zZSBvZiBh biBhbHRlcm5hdGUga3N0YWNrLCBpZiBpdCBleGlzdHMuCiAJICogWFhYIFdo YXQgaWYgdGhlcmUgYXJlIG1vcmUgdGhhbiBvbmUgdGhyZWFkIGluIHRoZSBw cm9jPwpAQCAtMTY4LDE0ICsxNjcsNiBAQAogCSAqLwogCWlmICgoKHAtPnBf ZmxhZyAmIFBfS1RIUkVBRCkgIT0gMCkgJiYgKHRkLT50ZF9hbHRrc3RhY2sg IT0gMCkpCiAJCXBtYXBfZGlzcG9zZV9hbHRrc3RhY2sodGQpOwotCi0JLyoK LQkgKiBXZSB3YW50IHRvIG1ha2Ugc3VyZSB3ZSBrbm93IHRoZSBpbml0aWFs IGxpbmthZ2VzLgotCSAqIHNvIGZvciBub3cgdGVhciB0aGVtIGRvd24gYW5k IHJlbWFrZSB0aGVtLgotCSAqIFRoaXMgaXMgcHJvYmFibHkgdW4tbmVlZGVk IGFzIHdlIGNhbiBwcm9iYWJseSByZWx5Ci0JICogb24gdGhlIHN0YXRlIGNv bWluZyBpbiBoZXJlIGZyb20gd2FpdDQoKS4KLQkgKi8KLQlwcm9jX2xpbmt1 cChwLCBrZywga2UsIHRkKTsKIH0KIAogLyoKQEAgLTE4NywxNSArMTc4LDE0 IEBACiAJc3RydWN0IHByb2MgKnA7CiAJc3RydWN0IHRocmVhZCAqdGQ7CiAJ c3RydWN0IGtzZWdycCAqa2c7Ci0Jc3RydWN0IGtzZSAqa2U7CiAKIAlwID0g KHN0cnVjdCBwcm9jICopbWVtOwogCXAtPnBfc2NoZWQgPSAoc3RydWN0IHBf c2NoZWQgKikmcFsxXTsKIAl2bV9wcm9jX25ldyhwKTsKIAl0ZCA9IHRocmVh ZF9hbGxvYygpOwotCWtlID0ga3NlX2FsbG9jKCk7CiAJa2cgPSBrc2VncnBf YWxsb2MoKTsKLQlwcm9jX2xpbmt1cChwLCBrZywga2UsIHRkKTsKKwlwcm9j X2xpbmt1cChwLCBrZywgdGQpOworCXNjaGVkX25ld3Byb2MocCwga2csIHRk KTsKIH0KIAogLyoKQEAgLTIwNyw3ICsxOTcsNiBAQAogCXN0cnVjdCBwcm9j ICpwOwogCXN0cnVjdCB0aHJlYWQgKnRkOwogCXN0cnVjdCBrc2VncnAgKmtn OwotCXN0cnVjdCBrc2UgKmtlOwogCiAJcCA9IChzdHJ1Y3QgcHJvYyAqKW1l bTsKIAlLQVNTRVJUKChwLT5wX251bXRocmVhZHMgPT0gMSksCkBAIC0yMTYs MTIgKzIwNSwxMCBAQAogCUtBU1NFUlQoKHRkICE9IE5VTEwpLCAoInByb2Nf ZHRvcjogYmFkIHRocmVhZCBwb2ludGVyIikpOwogICAgICAgICBrZyA9IEZJ UlNUX0tTRUdSUF9JTl9QUk9DKHApOwogCUtBU1NFUlQoKGtnICE9IE5VTEwp LCAoInByb2NfZHRvcjogYmFkIGtnIHBvaW50ZXIiKSk7Ci0gICAgICAgIGtl ID0gRklSU1RfS1NFX0lOX0tTRUdSUChrZyk7Ci0JS0FTU0VSVCgoa2UgIT0g TlVMTCksICgicHJvY19kdG9yOiBiYWQga2UgcG9pbnRlciIpKTsKIAl2bV9w cm9jX2Rpc3Bvc2UocCk7CisJc2NoZWRfZGVzdHJveXByb2MocCwga2csIHRk KTsKIAl0aHJlYWRfZnJlZSh0ZCk7CiAJa3NlZ3JwX2ZyZWUoa2cpOwotCWtz ZV9mcmVlKGtlKTsKIH0KIAogLyoKQEAgLTYxMSw3ICs1OTgsNiBAQAogewog CXN0cnVjdCB0aHJlYWQgKnRkOwogCXN0cnVjdCB0aHJlYWQgKnRkMDsKLQlz dHJ1Y3Qga3NlICprZTsKIAlzdHJ1Y3Qga3NlZ3JwICprZzsKIAlzdHJ1Y3Qg dHR5ICp0cDsKIAlzdHJ1Y3Qgc2Vzc2lvbiAqc3A7CkBAIC03MTksOCArNzA1 LDYgQEAKIAkJLyogdnZ2IFhYWEtTRSAqLwogCQlpZiAoIShwLT5wX2ZsYWcg JiBQX1RIUkVBREVEKSkgewogCQkJa2cgPSB0ZC0+dGRfa3NlZ3JwOwotCQkJ a2UgPSB0ZC0+dGRfa3NlOwotCQkJS0FTU0VSVCgoa2UgIT0gTlVMTCksICgi ZmlsbF9raW5mb19wcm9jOiBOdWxsIEtTRSIpKTsKIAkJCWJpbnRpbWUydGlt ZXZhbCgmcC0+cF9ydW50aW1lLCAmdHYpOwogCQkJa3AtPmtpX3J1bnRpbWUg PQogCQkJICAgIHR2LnR2X3NlYyAqICh1X2ludDY0X3QpMTAwMDAwMCArIHR2 LnR2X3VzZWM7CkBAIC03NDIsOSArNzI2LDggQEAKIAkJCWtwLT5raV9wY2Ig PSB0ZC0+dGRfcGNiOwogCQkJa3AtPmtpX2tzdGFjayA9ICh2b2lkICopdGQt PnRkX2tzdGFjazsKIAotCQkJLyogVGhpbmdzIGluIHRoZSBrc2UgKi8KLQkJ CWtwLT5raV9ycWluZGV4ID0ga2UtPmtlX3JxaW5kZXg7Ci0JCQlrcC0+a2lf cGN0Y3B1ID0gc2NoZWRfcGN0Y3B1KGtlKTsKKwkJCWtwLT5raV9ycWluZGV4 ID0gMDsgLyogY2FuJ3QgZ2V0IHRoaXMgbm93ICovCisJCQlrcC0+a2lfcGN0 Y3B1ID0gc2NoZWRfcGN0Y3B1KHRkKTsKIAkJfSBlbHNlIHsKIAkJCWtwLT5r aV9vbmNwdSA9IC0xOwogCQkJa3AtPmtpX2xhc3RjcHUgPSAtMTsKSW5kZXg6 IHN5cy9rZXJuL2tlcm5fc3dpdGNoLmMKPT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PQpSQ1MgZmlsZTogL3JlcG9zL3Byb2plY3RzL21pcnJvcmVkL2ZyZWVic2Qv c3JjL3N5cy9rZXJuL2tlcm5fc3dpdGNoLmMsdgpyZXRyaWV2aW5nIHJldmlz aW9uIDEuNTcKZGlmZiAtdSAtcjEuNTcga2Vybl9zd2l0Y2guYwotLS0gc3lz L2tlcm4va2Vybl9zd2l0Y2guYwkyMDAzLzA0LzAyIDIzOjUzOjI5CTEuNTcK KysrIHN5cy9rZXJuL2tlcm5fc3dpdGNoLmMJMjAwMy8wNC8xNyAwNjozNDo1 NQpAQCAtOTUsNiArOTUsNyBAQAogI2luY2x1ZGUgPHN5cy9rdHIuaD4KICNp bmNsdWRlIDxzeXMvbG9jay5oPgogI2luY2x1ZGUgPHN5cy9tdXRleC5oPgor I2luY2x1ZGUgPHN5cy9rc2V2YXIuaD4KICNpbmNsdWRlIDxzeXMvcHJvYy5o PgogI2luY2x1ZGUgPHN5cy9xdWV1ZS5oPgogI2luY2x1ZGUgPHN5cy9zY2hl ZC5oPgpJbmRleDogc3lzL2tlcm4va2Vybl90aHIuYwo9PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09ClJDUyBmaWxlOiAvcmVwb3MvcHJvamVjdHMvbWlycm9yZWQv ZnJlZWJzZC9zcmMvc3lzL2tlcm4va2Vybl90aHIuYyx2CnJldHJpZXZpbmcg cmV2aXNpb24gMS41CmRpZmYgLXUgLXIxLjUga2Vybl90aHIuYwotLS0gc3lz L2tlcm4va2Vybl90aHIuYwkyMDAzLzA0LzExIDE5OjI0OjM3CTEuNQorKysg c3lzL2tlcm4va2Vybl90aHIuYwkyMDAzLzA0LzE3IDA2OjM0OjU1CkBAIC01 MiwxMyArNTIsMTEgQEAKIHsKIAlzdHJ1Y3Qga3NlZ3JwICprZzsKIAlzdHJ1 Y3QgdGhyZWFkICp0ZDsKLQlzdHJ1Y3Qga3NlICprZTsKIAlzdHJ1Y3QgcHJv YyAqcDsKIAogCXRkID0gY3VydGhyZWFkOwogCXAgPSB0ZC0+dGRfcHJvYzsK IAlrZyA9IHRkLT50ZF9rc2VncnA7Ci0Ja2UgPSB0ZC0+dGRfa3NlOwogCiAJ bXR4X2Fzc2VydCgmc2NoZWRfbG9jaywgTUFfT1dORUQpOwogCVBST0NfTE9D S19BU1NFUlQocCwgTUFfT1dORUQpOwpAQCAtODAsMTggKzc4LDkgQEAKIAog CS8qIENsZWFuIHVwIGNwdSByZXNvdXJjZXMuICovCiAJY3B1X3RocmVhZF9l eGl0KHRkKTsKKwl0aHJlYWRfdW5saW5rKHRkKTsKKwlzY2hlZF90aHJfZXhp dCh0ZCk7CiAKLQkvKiBYWFggbWFrZSB0aHJlYWRfdW5saW5rKCkgKi8KLQlU QUlMUV9SRU1PVkUoJnAtPnBfdGhyZWFkcywgdGQsIHRkX3BsaXN0KTsKLQlw LT5wX251bXRocmVhZHMtLTsKLQlUQUlMUV9SRU1PVkUoJmtnLT5rZ190aHJl YWRzLCB0ZCwgdGRfa2dsaXN0KTsKLQlrZy0+a2dfbnVtdGhyZWFkcy0tOwot Ci0Ja2UtPmtlX3N0YXRlID0gS0VTX1VOUVVFVUVEOwotCWtlLT5rZV90aHJl YWQgPSBOVUxMOwotCWtzZV91bmxpbmsoa2UpOwotCXNjaGVkX2V4aXRfa3Nl KFRBSUxRX05FWFQoa2UsIGtlX2tnbGlzdCksIGtlKTsKLQogCS8qCiAJICog SWYgd2Ugd2VyZSBzdG9wcGVkIHdoaWxlIHdhaXRpbmcgZm9yIGFsbCB0aHJl YWRzIHRvIGV4aXQgYW5kIHRoaXMKIAkgKiBpcyB0aGUgbGFzdCB0aHJlYWQg d2FrZXVwIHRoZSBleGl0aW5nIHRocmVhZC4KQEAgLTEwMSw3ICs5MCw2IEBA CiAJCQl0aHJlYWRfdW5zdXNwZW5kX29uZShwLT5wX3NpbmdsZXRocmVhZCk7 CiAKIAlQUk9DX1VOTE9DSyhwKTsKLQl0ZC0+dGRfa3NlID0gTlVMTDsKIAl0 ZC0+dGRfc3RhdGUgPSBURFNfSU5BQ1RJVkU7CiAjaWYgMAogCXRkLT50ZF9w cm9jID0gTlVMTDsKQEAgLTEyNyw3ICsxMTUsNiBAQAogdGhyX2NyZWF0ZShz dHJ1Y3QgdGhyZWFkICp0ZCwgc3RydWN0IHRocl9jcmVhdGVfYXJncyAqdWFw KQogICAgIC8qIHVjb250ZXh0X3QgKmN0eCwgdGhyX2lkX3QgKmlkLCBpbnQg ZmxhZ3MgKi8KIHsKLQlzdHJ1Y3Qga3NlICprZTA7CiAJc3RydWN0IHRocmVh ZCAqdGQwOwogCXVjb250ZXh0X3QgY3R4OwogCWludCBlcnJvcjsKQEAgLTE1 NywzMiArMTQ0LDI3IEBACiAJYmNvcHkodGQtPnRkX2ZyYW1lLCB0ZDAtPnRk X2ZyYW1lLCBzaXplb2Yoc3RydWN0IHRyYXBmcmFtZSkpOwogCXRkMC0+dGRf dWNyZWQgPSBjcmhvbGQodGQtPnRkX3VjcmVkKTsKIAotCS8qIEluaXRpYWxp emUgb3VyIGtzZSBzdHJ1Y3R1cmUuICovCi0Ja2UwID0ga3NlX2FsbG9jKCk7 Ci0JYnplcm8oJmtlMC0+a2Vfc3RhcnR6ZXJvLAotCSAgICBSQU5HRU9GKHN0 cnVjdCBrc2UsIGtlX3N0YXJ0emVybywga2VfZW5kemVybykpOwogCiAJLyog U2V0IHVwIG91ciBtYWNoaW5lIGNvbnRleHQuICovCiAJY3B1X3NldF91cGNh bGwodGQwLCB0ZC0+dGRfcGNiKTsKIAllcnJvciA9IHNldF9tY29udGV4dCh0 ZDAsICZjdHgudWNfbWNvbnRleHQpOwogCWlmIChlcnJvciAhPSAwKSB7Ci0J CWtzZV9mcmVlKGtlMCk7CiAJCXRocmVhZF9mcmVlKHRkMCk7CiAJCWdvdG8g b3V0OwogCX0gCiAKLQkvKiBMaW5rIHRoZSB0aHJlYWQgYW5kIGtzZSBpbnRv IHRoZSBrc2VncnAgYW5kIG1ha2UgaXQgcnVubmFibGUuICovCiAJbXR4X2xv Y2tfc3Bpbigmc2NoZWRfbG9jayk7CiAKIAl0aHJlYWRfbGluayh0ZDAsIHRk LT50ZF9rc2VncnApOwotCWtzZV9saW5rKGtlMCwgdGQtPnRkX2tzZWdycCk7 Ci0KLQkvKiBCaW5kIHRoaXMgdGhyZWFkIGFuZCBrc2UgdG9nZXRoZXIuICov Ci0JdGQwLT50ZF9rc2UgPSBrZTA7Ci0Ja2UwLT5rZV90aHJlYWQgPSB0ZDA7 CiAKLQlzY2hlZF9mb3JrX2tzZSh0ZC0+dGRfa3NlLCBrZTApOwotCXNjaGVk X2ZvcmtfdGhyZWFkKHRkLCB0ZDApOworCS8qIERvIHdoYXRldmVyIGlzIG5l ZWRlZCBmb3IgdGhlIHNjaGVkdWxlciAqLworCWVycm9yID0gc2NoZWRfdGhy ZWFkX2ZvcmsodGQsIHRkMCk7CisJaWYgKGVycm9yICE9IDApIHsKKwkJdGhy ZWFkX3VubGluayh0ZCk7CisJCW10eF91bmxvY2tfc3Bpbigmc2NoZWRfbG9j ayk7CisJCXRocmVhZF9mcmVlKHRkMCk7CisJCWdvdG8gb3V0OworCX0KIAog CVREX1NFVF9DQU5fUlVOKHRkMCk7CiAJaWYgKCh1YXAtPmZsYWdzICYgVEhS X1NVU1BFTkRFRCkgPT0gMCkKSW5kZXg6IHN5cy9rZXJuL2tlcm5fdGhyZWFk LmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PQpSQ1MgZmlsZTogL3JlcG9zL3By b2plY3RzL21pcnJvcmVkL2ZyZWVic2Qvc3JjL3N5cy9rZXJuL2tlcm5fdGhy ZWFkLmMsdgpyZXRyaWV2aW5nIHJldmlzaW9uIDEuMTE0CmRpZmYgLXUgLXIx LjExNCBrZXJuX3RocmVhZC5jCi0tLSBzeXMva2Vybi9rZXJuX3RocmVhZC5j CTIwMDMvMDQvMTAgMTc6MzU6NDQJMS4xMTQKKysrIHN5cy9rZXJuL2tlcm5f dGhyZWFkLmMJMjAwMy8wNC8xNyAwNjozNDo1NQpAQCAtMzQsNiArMzQsNyBA QAogI2luY2x1ZGUgPHN5cy9sb2NrLmg+CiAjaW5jbHVkZSA8c3lzL21hbGxv Yy5oPgogI2luY2x1ZGUgPHN5cy9tdXRleC5oPgorI2luY2x1ZGUgPHN5cy9r c2V2YXIuaD4KICNpbmNsdWRlIDxzeXMvcHJvYy5oPgogI2luY2x1ZGUgPHN5 cy9zbXAuaD4KICNpbmNsdWRlIDxzeXMvc3lzY3RsLmg+CkBAIC02MSw3ICs2 Miw2IEBACiAgKiBLU0VHUlAgcmVsYXRlZCBzdG9yYWdlLgogICovCiBzdGF0 aWMgdW1hX3pvbmVfdCBrc2VncnBfem9uZTsKLXN0YXRpYyB1bWFfem9uZV90 IGtzZV96b25lOwogc3RhdGljIHVtYV96b25lX3QgdGhyZWFkX3pvbmU7CiBz dGF0aWMgdW1hX3pvbmVfdCB1cGNhbGxfem9uZTsKIApAQCAtMjAzLDE4ICsy MDMsNiBAQAogfQogCiAvKgotICogSW5pdGlhbGl6ZSB0eXBlLXN0YWJsZSBw YXJ0cyBvZiBhIGtzZSAod2hlbiBuZXdseSBjcmVhdGVkKS4KLSAqLwotc3Rh dGljIHZvaWQKLWtzZV9pbml0KHZvaWQgKm1lbSwgaW50IHNpemUpCi17Ci0J c3RydWN0IGtzZQkqa2U7Ci0KLQlrZSA9IChzdHJ1Y3Qga3NlICopbWVtOwot CWtlLT5rZV9zY2hlZCA9IChzdHJ1Y3Qga2Vfc2NoZWQgKikma2VbMV07Ci19 Ci0KLS8qCiAgKiBJbml0aWFsaXplIHR5cGUtc3RhYmxlIHBhcnRzIG9mIGEg a3NlZ3JwICh3aGVuIG5ld2x5IGNyZWF0ZWQpLgogICovCiBzdGF0aWMgdm9p ZApAQCAtMjI2LDQ0ICsyMTQsNiBAQAogCWtnLT5rZ19zY2hlZCA9IChzdHJ1 Y3Qga2dfc2NoZWQgKikma2dbMV07CiB9CiAKLS8qIAotICogS1NFIGlzIGxp bmtlZCBpbnRvIGtzZSBncm91cC4KLSAqLwotdm9pZAota3NlX2xpbmsoc3Ry dWN0IGtzZSAqa2UsIHN0cnVjdCBrc2VncnAgKmtnKQotewotCXN0cnVjdCBw cm9jICpwID0ga2ctPmtnX3Byb2M7Ci0KLQlUQUlMUV9JTlNFUlRfSEVBRCgm a2ctPmtnX2tzZXEsIGtlLCBrZV9rZ2xpc3QpOwotCWtnLT5rZ19rc2VzKys7 Ci0Ja2UtPmtlX3N0YXRlCT0gS0VTX1VOUVVFVUVEOwotCWtlLT5rZV9wcm9j CT0gcDsKLQlrZS0+a2Vfa3NlZ3JwCT0ga2c7Ci0Ja2UtPmtlX3RocmVhZAk9 IE5VTEw7Ci0Ja2UtPmtlX29uY3B1CT0gTk9DUFU7Ci0Ja2UtPmtlX2ZsYWdz CT0gMDsKLX0KLQotdm9pZAota3NlX3VubGluayhzdHJ1Y3Qga3NlICprZSkK LXsKLQlzdHJ1Y3Qga3NlZ3JwICprZzsKLQotCW10eF9hc3NlcnQoJnNjaGVk X2xvY2ssIE1BX09XTkVEKTsKLQlrZyA9IGtlLT5rZV9rc2VncnA7Ci0JVEFJ TFFfUkVNT1ZFKCZrZy0+a2dfa3NlcSwga2UsIGtlX2tnbGlzdCk7Ci0JaWYg KGtlLT5rZV9zdGF0ZSA9PSBLRVNfSURMRSkgewotCQlUQUlMUV9SRU1PVkUo JmtnLT5rZ19pcSwga2UsIGtlX2tncmxpc3QpOwotCQlrZy0+a2dfaWRsZV9r c2VzLS07Ci0JfQotCWlmICgtLWtnLT5rZ19rc2VzID09IDApCi0JCWtzZWdy cF91bmxpbmsoa2cpOwotCS8qCi0JICogQWdncmVnYXRlIHN0YXRzIGZyb20g dGhlIEtTRQotCSAqLwotCWtzZV9zdGFzaChrZSk7Ci19Ci0KIHZvaWQKIGtz ZWdycF9saW5rKHN0cnVjdCBrc2VncnAgKmtnLCBzdHJ1Y3QgcHJvYyAqcCkK IHsKQEAgLTM2NCw4ICszMTQsNyBAQAogICogbGluayB1cCBhbGwgdGhlIHN0 cnVjdHVyZXMgYW5kIGl0cyBpbml0aWFsIHRocmVhZHMgZXRjLgogICovCiB2 b2lkCi1wcm9jX2xpbmt1cChzdHJ1Y3QgcHJvYyAqcCwgc3RydWN0IGtzZWdy cCAqa2csCi0JICAgIHN0cnVjdCBrc2UgKmtlLCBzdHJ1Y3QgdGhyZWFkICp0 ZCkKK3Byb2NfbGlua3VwKHN0cnVjdCBwcm9jICpwLCBzdHJ1Y3Qga3NlZ3Jw ICprZywgc3RydWN0IHRocmVhZCAqdGQpCiB7CiAKIAlUQUlMUV9JTklUKCZw LT5wX2tzZWdycHMpOwkgICAgIC8qIGFsbCBrc2VncnBzIGluIHByb2MgKi8K QEAgLTM3NSw3ICszMjQsNiBAQAogCXAtPnBfbnVtdGhyZWFkcyA9IDA7CiAK IAlrc2VncnBfbGluayhrZywgcCk7Ci0Ja3NlX2xpbmsoa2UsIGtnKTsKIAl0 aHJlYWRfbGluayh0ZCwga2cpOwogfQogCkBAIC04MDUsOSArNzUzLDYgQEAK IAlrc2VncnBfem9uZSA9IHVtYV96Y3JlYXRlKCJLU0VHUlAiLCBzY2hlZF9z aXplb2Zfa3NlZ3JwKCksCiAJICAgIE5VTEwsIE5VTEwsIGtzZWdycF9pbml0 LCBOVUxMLAogCSAgICBVTUFfQUxJR05fQ0FDSEUsIDApOwotCWtzZV96b25l ID0gdW1hX3pjcmVhdGUoIktTRSIsIHNjaGVkX3NpemVvZl9rc2UoKSwKLQkg ICAgTlVMTCwgTlVMTCwga3NlX2luaXQsIE5VTEwsCi0JICAgIFVNQV9BTElH Tl9DQUNIRSwgMCk7CiAJdXBjYWxsX3pvbmUgPSB1bWFfemNyZWF0ZSgiVVBD QUxMIiwgc2l6ZW9mKHN0cnVjdCBrc2VfdXBjYWxsKSwKIAkgICAgTlVMTCwg TlVMTCwgTlVMTCwgTlVMTCwgVU1BX0FMSUdOX0NBQ0hFLCAwKTsKIH0KQEAg LTkyNSwxNSArODcwLDYgQEAKIH0KIAogLyoKLSAqIEFsbG9jYXRlIGEga3Nl LgotICovCi1zdHJ1Y3Qga3NlICoKLWtzZV9hbGxvYyh2b2lkKQotewotCXJl dHVybiAodW1hX3phbGxvYyhrc2Vfem9uZSwgTV9XQUlUT0spKTsKLX0KLQot LyoKICAqIEFsbG9jYXRlIGEgdGhyZWFkLgogICovCiBzdHJ1Y3QgdGhyZWFk ICoKQEAgLTk1MywxNSArODg5LDYgQEAKIH0KIAogLyoKLSAqIERlYWxsb2Nh dGUgYSBrc2UuCi0gKi8KLXZvaWQKLWtzZV9mcmVlKHN0cnVjdCBrc2UgKnRk KQotewotCXVtYV96ZnJlZShrc2Vfem9uZSwgdGQpOwotfQotCi0vKgogICog RGVhbGxvY2F0ZSBhIHRocmVhZC4KICAqLwogdm9pZApAQCAtMTIwMywxMCAr MTEzMCw3IEBACiAJCSAqIHdvdWxkIG9ubHkgYmUgY2FsbGVkIGZyb20gaGVy ZSAoSSB0aGluaykgc28gaXQgd291bGQKIAkJICogYmUgYSB3YXN0ZS4gKG1p Z2h0IGJlIHVzZWZ1bCBmb3IgcHJvY19maW5pKCkgYXMgd2VsbC4pCiAgCQkg Ki8KLQkJVEFJTFFfUkVNT1ZFKCZwLT5wX3RocmVhZHMsIHRkLCB0ZF9wbGlz dCk7Ci0JCXAtPnBfbnVtdGhyZWFkcy0tOwotCQlUQUlMUV9SRU1PVkUoJmtn LT5rZ190aHJlYWRzLCB0ZCwgdGRfa2dsaXN0KTsKLQkJa2ctPmtnX251bXRo cmVhZHMtLTsKKwkJdGhyZWFkX3VubGluayh0ZCk7CiAJCWlmIChwLT5wX21h eHRocndhaXRzKQogCQkJd2FrZXVwKCZwLT5wX251bXRocmVhZHMpOwogCQkv KgpAQCAtMTIzMSwxNSArMTE1NSwxMSBAQAogCQlpZiAodGQtPnRkX3VwY2Fs bCkKIAkJCXVwY2FsbF9yZW1vdmUodGQpOwogCQotCQlrZS0+a2Vfc3RhdGUg PSBLRVNfVU5RVUVVRUQ7Ci0JCWtlLT5rZV90aHJlYWQgPSBOVUxMOwogCQkv KiAKLQkJICogRGVjaWRlIHdoYXQgdG8gZG8gd2l0aCB0aGUgS1NFIGF0dGFj aGVkIHRvIHRoaXMgdGhyZWFkLgorCQkgKiBMZXQgdGhlIHNjaGVkdWxlciBk byBhbnl0aGluZyBpdCBuZWVkcyB0byBjbGVhbiB1cC4KIAkJICovCi0JCWlm IChrZS0+a2VfZmxhZ3MgJiBLRUZfRVhJVCkKLQkJCWtzZV91bmxpbmsoa2Up OwotCQllbHNlCi0JCQlrc2VfcmVhc3NpZ24oa2UpOworCQlzY2hlZF90ZGV4 aXQodGQpOworCiAJCVBST0NfVU5MT0NLKHApOwogCQl0ZC0+dGRfa3NlCT0g TlVMTDsKIAkJdGQtPnRkX3N0YXRlCT0gVERTX0lOQUNUSVZFOwpAQCAtMTI0 Nyw3ICsxMTY3LDYgQEAKIAkJdGQtPnRkX3Byb2MJPSBOVUxMOwogI2VuZGlm CiAJCXRkLT50ZF9rc2VncnAJPSBOVUxMOwotCQl0ZC0+dGRfbGFzdF9rc2UJ PSBOVUxMOwogCQlQQ1BVX1NFVChkZWFkdGhyZWFkLCB0ZCk7CiAJfSBlbHNl IHsKIAkJUFJPQ19VTkxPQ0socCk7CkBAIC0xMzExLDYgKzEyMzAsMTggQEAK IAlUQUlMUV9JTlNFUlRfSEVBRCgma2ctPmtnX3RocmVhZHMsIHRkLCB0ZF9r Z2xpc3QpOwogCXAtPnBfbnVtdGhyZWFkcysrOwogCWtnLT5rZ19udW10aHJl YWRzKys7Cit9CisKK3ZvaWQKK3RocmVhZF91bmxpbmsoc3RydWN0IHRocmVh ZCAqdGQpCit7CisJCXN0cnVjdCBwcm9jICpwID0gdGQtPnRkX3Byb2M7CisJ CXN0cnVjdCBrc2VncnAgKmtnID0gdGQtPnRkX2tzZWdycDsKKworCQlUQUlM UV9SRU1PVkUoJnAtPnBfdGhyZWFkcywgdGQsIHRkX3BsaXN0KTsKKwkJcC0+ cF9udW10aHJlYWRzLS07CisJCVRBSUxRX1JFTU9WRSgma2ctPmtnX3RocmVh ZHMsIHRkLCB0ZF9rZ2xpc3QpOworCQlrZy0+a2dfbnVtdGhyZWFkcy0tOwog fQogCiAvKgpJbmRleDogc3lzL2tlcm4vc2NoZWRfNGJzZC5jCj09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT0KUkNTIGZpbGU6IC9yZXBvcy9wcm9qZWN0cy9taXJy b3JlZC9mcmVlYnNkL3NyYy9zeXMva2Vybi9zY2hlZF80YnNkLmMsdgpyZXRy aWV2aW5nIHJldmlzaW9uIDEuMTUKZGlmZiAtdSAtcjEuMTUgc2NoZWRfNGJz ZC5jCi0tLSBzeXMva2Vybi9zY2hlZF80YnNkLmMJMjAwMy8wNC8xMSAwMzoz OTo0OAkxLjE1CisrKyBzeXMva2Vybi9zY2hlZF80YnNkLmMJMjAwMy8wNC8x NyAwNjozNDo1NQpAQCAtNDQsNiArNDQsNyBAQAogI2luY2x1ZGUgPHN5cy9r dHIuaD4KICNpbmNsdWRlIDxzeXMvbG9jay5oPgogI2luY2x1ZGUgPHN5cy9t dXRleC5oPgorI2luY2x1ZGUgPHN5cy9rc2V2YXIuaD4KICNpbmNsdWRlIDxz eXMvcHJvYy5oPgogI2luY2x1ZGUgPHN5cy9yZXNvdXJjZXZhci5oPgogI2lu Y2x1ZGUgPHN5cy9zY2hlZC5oPgpAQCAtNDM4LDEzICs0MzksMTMgQEAKICAq IHJ1biBtdWNoIHJlY2VudGx5LCBhbmQgdG8gcm91bmQtcm9iaW4gYW1vbmcg b3RoZXIgcHJvY2Vzc2VzLgogICovCiB2b2lkCi1zY2hlZF9jbG9jayhzdHJ1 Y3Qga3NlICprZSkKK3NjaGVkX2Nsb2NrKHN0cnVjdCB0aHJlYWQgKnRkKQog ewogCXN0cnVjdCBrc2VncnAgKmtnOwotCXN0cnVjdCB0aHJlYWQgKnRkOwor CXN0cnVjdCBrc2UgKmtlOwogCi0Ja2cgPSBrZS0+a2Vfa3NlZ3JwOwotCXRk ID0ga2UtPmtlX3RocmVhZDsKKwlrZyA9IHRkLT50ZF9rc2VncnA7CisJa2Ug PSB0ZC0+dGRfa3NlOwogCiAJa2UtPmtlX3NjaGVkLT5za2VfY3B0aWNrcysr OwogCWtnLT5rZ19lc3RjcHUgPSBFU1RDUFVMSU0oa2ctPmtnX2VzdGNwdSAr IDEpOwpAQCAtNDg3LDMxICs0ODgsNiBAQAogfQogCiB2b2lkCi1zY2hlZF9m b3JrKHN0cnVjdCBwcm9jICpwLCBzdHJ1Y3QgcHJvYyAqcDEpCi17Ci0Jc2No ZWRfZm9ya19rc2UoRklSU1RfS1NFX0lOX1BST0MocCksIEZJUlNUX0tTRV9J Tl9QUk9DKHAxKSk7Ci0Jc2NoZWRfZm9ya19rc2VncnAoRklSU1RfS1NFR1JQ X0lOX1BST0MocCksIEZJUlNUX0tTRUdSUF9JTl9QUk9DKHAxKSk7Ci0Jc2No ZWRfZm9ya190aHJlYWQoRklSU1RfVEhSRUFEX0lOX1BST0MocCksIEZJUlNU X1RIUkVBRF9JTl9QUk9DKHAxKSk7Ci19Ci0KLXZvaWQKLXNjaGVkX2Zvcmtf a3NlKHN0cnVjdCBrc2UgKmtlLCBzdHJ1Y3Qga3NlICpjaGlsZCkKLXsKLQlj aGlsZC0+a2Vfc2NoZWQtPnNrZV9jcHRpY2tzID0gMDsKLX0KLQotdm9pZAot c2NoZWRfZm9ya19rc2VncnAoc3RydWN0IGtzZWdycCAqa2csIHN0cnVjdCBr c2VncnAgKmNoaWxkKQotewotCWNoaWxkLT5rZ19lc3RjcHUgPSBrZy0+a2df ZXN0Y3B1OwotfQotCi12b2lkCi1zY2hlZF9mb3JrX3RocmVhZChzdHJ1Y3Qg dGhyZWFkICp0ZCwgc3RydWN0IHRocmVhZCAqY2hpbGQpCi17Ci19Ci0KLXZv aWQKIHNjaGVkX25pY2Uoc3RydWN0IGtzZWdycCAqa2csIGludCBuaWNlKQog ewogCWtnLT5rZ19uaWNlID0gbmljZTsKQEAgLTY5NSw3ICs2NzEsMTMgQEAK IH0KIAogZml4cHRfdAotc2NoZWRfcGN0Y3B1KHN0cnVjdCBrc2UgKmtlKQor c2NoZWRfcGN0Y3B1KHN0cnVjdCB0aHJlYWQgKnRkKQogewotCXJldHVybiAo a2UtPmtlX3BjdGNwdSk7CisJc3RydWN0IGtzZSAqa2U7CisKKwlrZSA9IHRk LT50ZF9rc2U7CisJaWYgKGtlKQorCQlyZXR1cm4gKGtlLT5rZV9wY3RjcHUp OworCWVsc2UKKwkJcmV0dXJuICgoZml4cHRfdCkwKTsKIH0KSW5kZXg6IHN5 cy9rZXJuL3NjaGVkX3VsZS5jCj09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KUkNT IGZpbGU6IC9yZXBvcy9wcm9qZWN0cy9taXJyb3JlZC9mcmVlYnNkL3NyYy9z eXMva2Vybi9zY2hlZF91bGUuYyx2CnJldHJpZXZpbmcgcmV2aXNpb24gMS4y OApkaWZmIC11IC1yMS4yOCBzY2hlZF91bGUuYwotLS0gc3lzL2tlcm4vc2No ZWRfdWxlLmMJMjAwMy8wNC8xMiAyMjozMzoyNAkxLjI4CisrKyBzeXMva2Vy bi9zY2hlZF91bGUuYwkyMDAzLzA0LzE3IDA2OjM0OjU1CkBAIC04NTUsMTEg Kzg1NSwxMSBAQAogfQogCiB2b2lkCi1zY2hlZF9jbG9jayhzdHJ1Y3Qga3Nl ICprZSkKK3NjaGVkX2Nsb2NrKHN0cnVjdCB0aHJlYWQgKnRkKQogewogCXN0 cnVjdCBrc2VxICprc2VxOwogCXN0cnVjdCBrc2VncnAgKmtnOwotCXN0cnVj dCB0aHJlYWQgKnRkOworCXN0cnVjdCBrc2UgKmtlCiAjaWYgMAogCXN0cnVj dCBrc2UgKm5rZTsKICNlbmRpZgpAQCAtODgwLDggKzg4MCw4IEBACiAJCQl0 aWNraW5jciA9IDE7CiAJfQogCi0JdGQgPSBrZS0+a2VfdGhyZWFkOwotCWtn ID0ga2UtPmtlX2tzZWdycDsKKwlrZSA9IHRkLT50ZF9rc2U7CisJa2cgPSB0 ZC0+dGRfa3NlZ3JwOwogCiAJbXR4X2Fzc2VydCgmc2NoZWRfbG9jaywgTUFf T1dORUQpOwogCUtBU1NFUlQoKHRkICE9IE5VTEwpLCAoInNjaGVkY2xvY2s6 IG51bGwgdGhyZWFkIHBvaW50ZXIiKSk7CkBAIC04OTQsNyArODk0LDcgQEAK IAlpZiAoa2UtPmtlX2Z0aWNrICsgU0NIRURfQ1BVX1RJQ0tTICsgaHogPCBr ZS0+a2VfbHRpY2spCiAJCXNjaGVkX3BjdGNwdV91cGRhdGUoa2UpOwogCi0J aWYgKHRkLT50ZF9rc2UtPmtlX2ZsYWdzICYgS0VGX0lETEVLU0UpCisJaWYg KHRkLT50ZF9mbGFncyAmIFREX0lETEVURCkKIAkJcmV0dXJuOwogCiAJQ1RS NChLVFJfVUxFLCAiVGljayBrc2UgJXAgKHNsaWNlOiAlZCwgc2xwdGltZTog JWQsIHJ1bnRpbWU6ICVkKSIsCkluZGV4OiBzeXMva2Vybi9zdWJyX3RyYXAu Ywo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09ClJDUyBmaWxlOiAvcmVwb3MvcHJv amVjdHMvbWlycm9yZWQvZnJlZWJzZC9zcmMvc3lzL2tlcm4vc3Vicl90cmFw LmMsdgpyZXRyaWV2aW5nIHJldmlzaW9uIDEuMjUwCmRpZmYgLXUgLXIxLjI1 MCBzdWJyX3RyYXAuYwotLS0gc3lzL2tlcm4vc3Vicl90cmFwLmMJMjAwMy8w My8zMSAyMjo0OToxNgkxLjI1MAorKysgc3lzL2tlcm4vc3Vicl90cmFwLmMJ MjAwMy8wNC8xNyAwNjozNDo1NwpAQCAtMTM5LDcgKzEzOSw2IEBACiB7CiAJ c3RydWN0IHRocmVhZCAqdGQ7CiAJc3RydWN0IHByb2MgKnA7Ci0Jc3RydWN0 IGtzZSAqa2U7CiAJc3RydWN0IGtzZWdycCAqa2c7CiAJc3RydWN0IHJsaW1p dCAqcmxpbTsKIAl1X2ludCBwcnRpY2tzLCBzdGlja3M7CkBAIC0xNzAsNyAr MTY5LDYgQEAKIAkgKiBhc3QoKSB3aWxsIGJlIGNhbGxlZCBhZ2Fpbi4KIAkg Ki8KIAltdHhfbG9ja19zcGluKCZzY2hlZF9sb2NrKTsKLQlrZSA9IHRkLT50 ZF9rc2U7CiAJc3RpY2tzID0gdGQtPnRkX3N0aWNrczsKIAlmbGFncyA9IHRk LT50ZF9mbGFnczsKIAlzZmxhZyA9IHAtPnBfc2ZsYWc7CkluZGV4OiBzeXMv c3lzL3Byb2MuaAo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09ClJDUyBmaWxlOiAv cmVwb3MvcHJvamVjdHMvbWlycm9yZWQvZnJlZWJzZC9zcmMvc3lzL3N5cy9w cm9jLmgsdgpyZXRyaWV2aW5nIHJldmlzaW9uIDEuMzEzCmRpZmYgLXUgLXIx LjMxMyBwcm9jLmgKLS0tIHN5cy9zeXMvcHJvYy5oCTIwMDMvMDQvMTMgMjE6 Mjk6MTEJMS4zMTMKKysrIHN5cy9zeXMvcHJvYy5oCTIwMDMvMDQvMTcgMDY6 MzU6MDEKQEAgLTIwMCwyMCArMjAwLDYgQEAKIHN0cnVjdCB0aHJlYWQ7CiAK IC8qCi0gKiBUaGUgc2Vjb25kIHN0cnVjdHVyZSBpcyB0aGUgS2VybmVsIFNj aGVkdWxhYmxlIEVudGl0eS4gKEtTRSkKLSAqIEl0IHJlcHJlc2VudHMgdGhl IGFiaWxpdHkgdG8gdGFrZSBhIHNsb3QgaW4gdGhlIHNjaGVkdWxlciBxdWV1 ZS4KLSAqIEFzIGxvbmcgYXMgdGhpcyBpcyBzY2hlZHVsZWQsIGl0IGNvdWxk IGNvbnRpbnVlIHRvIHJ1biBhbnkgdGhyZWFkcyB0aGF0Ci0gKiBhcmUgYXNz aWduZWQgdG8gdGhlIEtTRUdSUCAoc2VlIGxhdGVyKSB1bnRpbCBlaXRoZXIg aXQgcnVucyBvdXQKLSAqIG9mIHJ1bm5hYmxlIHRocmVhZHMgb2YgaGlnaCBl bm91Z2ggcHJpb3JpdHksIG9yIENQVS4KLSAqIEl0IHJ1bnMgb24gb25lIENQ VSBhbmQgaXMgYXNzaWduZWQgYSBxdWFudHVtIG9mIHRpbWUuIFdoZW4gYSB0 aHJlYWQgaXMKLSAqIGJsb2NrZWQsIFRoZSBLU0UgY29udGludWVzIHRvIHJ1 biBhbmQgd2lsbCBzZWFyY2ggZm9yIGFub3RoZXIgdGhyZWFkCi0gKiBpbiBh IHJ1bm5hYmxlIHN0YXRlIGFtb25nc3QgdGhvc2UgaXQgaGFzLiBJdCBNYXkg ZGVjaWRlIHRvIHJldHVybiB0byB1c2VyCi0gKiBtb2RlIHdpdGggYSBuZXcg J2VtcHR5JyB0aHJlYWQgaWYgdGhlcmUgYXJlIG5vIHJ1bm5hYmxlIHRocmVh ZHMuCi0gKiBUaHJlYWRzIGFyZSB0ZW1wb3JhcmlseSBhc3NvY2lhdGVkIHdp dGggYSBLU0UgZm9yIHNjaGVkdWxpbmcgcmVhc29ucy4KLSAqLwotc3RydWN0 IGtzZTsKLQotLyoKICAqIFRoZSBLU0VHUlAgaXMgYWxsb2NhdGVkIHJlc291 cmNlcyBhY3Jvc3MgYSBudW1iZXIgb2YgQ1BVcy4KICAqIChJbmNsdWRpbmcg YSBudW1iZXIgb2YgQ1BVeFFVQU5UQS4gSXQgcGFyY2VscyB0aGVzZSBRVUFO VEEgdXAgYW1vbmcKICAqIGl0cyBLU0VzLCBlYWNoIG9mIHdoaWNoIHNob3Vs ZCBiZSBydW5uaW5nIGluIGEgZGlmZmVyZW50IENQVS4KQEAgLTIzMywyNSAr MjE5LDEzIEBACiAgKiBmb3JrZWQgcHJvY2VzcyBjbHVzdGVyIGJ5IHNwYXdu aW5nIHNldmVyYWwgS1NFR1JQcy4KICAqLwogc3RydWN0IHByb2M7Ci0KLS8q KioqKioqKioqKioqKioKLSAqIEluIHBpY3R1cmVzOgotIFdpdGggYSBzaW5n bGUgcnVuIHF1ZXVlIHVzZWQgYnkgYWxsIHByb2Nlc3NvcnM6Ci0KLSBSVU5R OiAtLS0+S1NFLS0tS1NFLS0uLi4gICAgICAgICAgICAgICBTTEVFUFE6W10t LS1USFJFQUQtLS1USFJFQUQtLS1USFJFQUQKLQkgICB8ICAgLyAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICBbXS0tLVRIUkVBRAotCSAgIEtTRUct LS1USFJFQUQtLVRIUkVBRC0tVEhSRUFEICAgICAgIFtdCi0JCQkJCSAgICAg ICBbXS0tLVRIUkVBRC0tLVRIUkVBRAotCi0gIChwcm9jZXNzb3JzIHJ1biBU SFJFQURzIGZyb20gdGhlIEtTRUcgdW50aWwgdGhleSBhcmUgZXhoYXVzdGVk IG9yCi0gIHRoZSBLU0VHIGV4aGF1c3RzIGl0cyBxdWFudHVtKQotCi1XaXRo IFBFUi1DUFUgcnVuIHF1ZXVlczoKLUtTRXMgb24gdGhlIHNlcGFyYXRlIHJ1 biBxdWV1ZXMgZGlyZWN0bHkKLVRoZXkgd291bGQgYmUgZ2l2ZW4gcHJpb3Jp dGllcyBjYWxjdWxhdGVkIGZyb20gdGhlIEtTRUcuCiAKLSAqCi0gKioqKioq KioqKioqKioqKiovCitzdHJ1Y3QgZHVtbXkxIAoreworfTsKKyNpZm5kZWYg X1NZU19LU0VWQVJfSF8KKyNkZWZpbmUgS1NFREVGIGR1bW15MQorI2VuZGlm CiAKIC8qCiAgKiBLZXJuZWwgcnVubmFibGUgY29udGV4dCAodGhyZWFkKS4K QEAgLTI3OSw4ICsyNTMsOCBAQAogI2RlZmluZQl0ZF9zdGFydHplcm8gdGRf ZmxhZ3MKIAlpbnQJCXRkX2ZsYWdzOwkvKiAoaikgVERGXyogZmxhZ3MuICov CiAJaW50CQl0ZF9pbmhpYml0b3JzOwkvKiAoaikgV2h5IGNhbiBub3QgcnVu ICovCi0Jc3RydWN0IGtzZQkqdGRfbGFzdF9rc2U7CS8qIChqKSBQcmV2aW91 cyB2YWx1ZSBvZiB0ZF9rc2UgKi8KLQlzdHJ1Y3Qga3NlCSp0ZF9rc2U7CS8q IChqKSBDdXJyZW50IEtTRSBpZiBydW5uaW5nLiAqLworCXN0cnVjdCBLU0VE RUYJKnRkX2xhc3Rfa3NlOwkvKiAoaikgUHJldmlvdXMgdmFsdWUgb2YgdGRf a3NlICovCisJc3RydWN0IEtTRURFRgkqdGRfa3NlOwkvKiAoaikgQ3VycmVu dCBLU0UgaWYgcnVubmluZy4gKi8KIAlpbnQJCXRkX2R1cGZkOwkvKiAoaykg UmV0IHZhbHVlIGZyb20gZmRvcGVuLiBYWFggKi8KIAl2b2lkCQkqdGRfd2No YW47CS8qIChqKSBTbGVlcCBhZGRyZXNzLiAqLwogCWNvbnN0IGNoYXIJKnRk X3dtZXNnOwkvKiAoaikgUmVhc29uIGZvciBzbGVlcC4gKi8KQEAgLTM0OSw2 ICszMjMsNyBAQAogI2RlZmluZQlUREZfQ0FOX1VOQklORAkweDAwMDAwNCAv KiBPbmx5IHRlbXBvcmFyaWx5IGJvdW5kLiAqLwogI2RlZmluZQlUREZfU0lO VFIJMHgwMDAwMDggLyogU2xlZXAgaXMgaW50ZXJydXB0aWJsZS4gKi8KICNk ZWZpbmUJVERGX1RJTUVPVVQJMHgwMDAwMTAgLyogVGltaW5nIG91dCBkdXJp bmcgc2xlZXAuICovCisjZGVmaW5lCVRERl9JRExFVEQJMHgwMDAwNDAgLyog VGhpcyBpcyBhbiBpZGxlIHRocmVhZCAqLwogI2RlZmluZQlUREZfU0VMRUNU CTB4MDAwMDQwIC8qIFNlbGVjdGluZzsgd2FrZXVwL3dhaXRpbmcgZGFuZ2Vy LiAqLwogI2RlZmluZQlUREZfQ1ZXQUlUUQkweDAwMDA4MCAvKiBUaHJlYWQg aXMgb24gYSBjdl93YWl0cSAobm90IHNscHEpLiAqLwogI2RlZmluZQlUREZf VVBDQUxMSU5HCTB4MDAwMTAwIC8qIFRoaXMgdGhyZWFkIGlzIGRvaW5nIGFu IHVwY2FsbC4gKi8KQEAgLTQxOCw0MyArMzkzLDYgQEAKIH0gd2hpbGUgKDAp CiAKIC8qCi0gKiBUaGUgc2NoZWR1bGFibGUgZW50aXR5IHRoYXQgY2FuIGJl IGdpdmVuIGEgY29udGV4dCB0byBydW4uCi0gKiBBIHByb2Nlc3MgbWF5IGhh dmUgc2V2ZXJhbCBvZiB0aGVzZS4gUHJvYmFibHkgb25lIHBlciBwcm9jZXNz b3IKLSAqIGJ1dCBwb3NpYmx5IGEgZmV3IG1vcmUuIEluIHRoaXMgdW5pdmVy c2UgdGhleSBhcmUgZ3JvdXBlZAotICogd2l0aCBhIEtTRUcgdGhhdCBjb250 YWlucyB0aGUgcHJpb3JpdHkgYW5kIG5pY2VuZXNzCi0gKiBmb3IgdGhlIGdy b3VwLgotICovCi1zdHJ1Y3Qga3NlIHsKLQlzdHJ1Y3QgcHJvYwkqa2VfcHJv YzsJLyogQXNzb2NpYXRlZCBwcm9jZXNzLiAqLwotCXN0cnVjdCBrc2VncnAJ KmtlX2tzZWdycDsJLyogQXNzb2NpYXRlZCBLU0VHLiAqLwotCVRBSUxRX0VO VFJZKGtzZSkga2Vfa2dsaXN0OwkvKiBRdWV1ZSBvZiBhbGwgS1NFcyBpbiBr ZV9rc2VncnAuICovCi0JVEFJTFFfRU5UUlkoa3NlKSBrZV9rZ3JsaXN0Owkv KiBRdWV1ZSBvZiBhbGwgS1NFcyBpbiB0aGlzIHN0YXRlLiAqLwotCVRBSUxR X0VOVFJZKGtzZSkga2VfcHJvY3E7CS8qIChqKSBSdW4gcXVldWUuICovCi0K LSNkZWZpbmUJa2Vfc3RhcnR6ZXJvIGtlX2ZsYWdzCi0JaW50CQlrZV9mbGFn czsJLyogKGopIEtFRl8qIGZsYWdzLiAqLwotCXN0cnVjdCB0aHJlYWQJKmtl X3RocmVhZDsJLyogQWN0aXZlIGFzc29jaWF0ZWQgdGhyZWFkLiAqLwotCWZp eHB0X3QJCWtlX3BjdGNwdTsgICAgIC8qIChqKSAlY3B1IGR1cmluZyBwX3N3 dGltZS4gKi8KLQl1X2NoYXIJCWtlX29uY3B1OwkvKiAoaikgV2hpY2ggY3B1 IHdlIGFyZSBvbi4gKi8KLQljaGFyCQlrZV9ycWluZGV4OwkvKiAoaikgUnVu IHF1ZXVlIGluZGV4LiAqLwotCWVudW0gewotCQlLRVNfVU5VU0VEID0gMHgw LAotCQlLRVNfSURMRSwKLQkJS0VTX09OUlVOUSwKLQkJS0VTX1VOUVVFVUVE LAkJLyogaW4gdHJhbnNpdCAqLwotCQlLRVNfVEhSRUFECQkvKiBzbGF2ZWQg dG8gdGhyZWFkIHN0YXRlICovCi0JfSBrZV9zdGF0ZTsJCQkvKiAoaikgUyog cHJvY2VzcyBzdGF0dXMuICovCi0jZGVmaW5lCWtlX2VuZHplcm8ga2VfZHVt bXkKLQl1X2NoYXIJCWtlX2R1bW15OwotCXN0cnVjdCBrZV9zY2hlZAkqa2Vf c2NoZWQ7CS8qIFNjaGVkdWxlciBzcGVjaWZpYyBkYXRhICovCi19OwotCi0v KiBmbGFncyBrZXB0IGluIGtlX2ZsYWdzICovCi0jZGVmaW5lCUtFRl9JRExF S1NFCTB4MDAwMDQJLyogQSAnUGVyIENQVSBpZGxlIHByb2Nlc3MnLi4gaGFz IG9uZSB0aHJlYWQgKi8KLSNkZWZpbmUJS0VGX0RJRFJVTgkweDAyMDAwCS8q IEtTRSBhY3R1YWxseSByYW4uICovCi0jZGVmaW5lCUtFRl9FWElUCTB4MDQw MDAJLyogS1NFIGlzIGJlaW5nIGtpbGxlZC4gKi8KLQotLyoKICAqIFRoZSB1 cGNhbGwgbWFuYWdlbWVudCBzdHJ1Y3R1cmUuCiAgKiBUaGUgdXBjYWxsIGlz IHVzZWQgd2hlbiByZXR1cm5pbmcgdG8gdXNlcmxhbmQuICBJZiBhIHRocmVh ZCBkb2VzIG5vdCBoYXZlCiAgKiBhbiB1cGNhbGwgb24gcmV0dXJuIHRvIHVz ZXJsYW5kIHRoZSB0aHJlYWQgZXhwb3J0cyBpdHMgY29udGV4dCBhbmQgZXhp dHMuCkBAIC00NzksOCArNDE3LDggQEAKIHN0cnVjdCBrc2VncnAgewogCXN0 cnVjdCBwcm9jCSprZ19wcm9jOwkvKiBQcm9jZXNzIHRoYXQgY29udGFpbnMg dGhpcyBLU0VHLiAqLwogCVRBSUxRX0VOVFJZKGtzZWdycCkga2dfa3NlZ3Jw OwkvKiBRdWV1ZSBvZiBLU0VHcyBpbiBrZ19wcm9jLiAqLwotCVRBSUxRX0hF QUQoLCBrc2UpIGtnX2tzZXE7CS8qIChrZV9rZ2xpc3QpIEFsbCBLU0VzLiAq LwotCVRBSUxRX0hFQUQoLCBrc2UpIGtnX2lxOwkvKiAoa2Vfa2dybGlzdCkg QWxsIGlkbGUgS1NFcy4gKi8KKwlUQUlMUV9IRUFEKCwgS1NFREVGKSBrZ19r c2VxOwkvKiAoa2Vfa2dsaXN0KSBBbGwgS1NFcy4gKi8KKwlUQUlMUV9IRUFE KCwgS1NFREVGKSBrZ19pcTsJLyogKGtlX2tncmxpc3QpIEFsbCBpZGxlIEtT RXMuICovCiAJVEFJTFFfSEVBRCgsIHRocmVhZCkga2dfdGhyZWFkczsvKiAo dGRfa2dsaXN0KSBBbGwgdGhyZWFkcy4gKi8KIAlUQUlMUV9IRUFEKCwgdGhy ZWFkKSBrZ19ydW5xOwkvKiAodGRfcnVucSkgd2FpdGluZyBSVU5OQUJMRSB0 aHJlYWRzICovCiAJVEFJTFFfSEVBRCgsIHRocmVhZCkga2dfc2xwcTsJLyog KHRkX3J1bnEpIE5PTlJVTk5BQkxFIHRocmVhZHMuICovCkBAIC01MjYsOSAr NDY0LDYgQEAKIAlzdHJ1Y3Qgdm1fb2JqZWN0ICpwX3VwYWdlc19vYmo7IC8q IChhKSBVcGFnZXMgb2JqZWN0LiAqLwogCXN0cnVjdCBwcm9jc2lnCSpwX3By b2NzaWc7CS8qIChjKSBTaWduYWwgYWN0aW9ucywgc3RhdGUgKENQVSkuICov CiAKLQkvKnN0cnVjdCBrc2VncnAJcF9rc2VncnA7Ci0Jc3RydWN0IGtzZQlw X2tzZTsgKi8KLQogCS8qCiAJICogVGhlIGZvbGxvd2luZyBkb24ndCBtYWtl IHRvbyBtdWNoIHNlbnNlLi4KIAkgKiBTZWUgdGhlIHRkXyBvciBrZV8gdmVy c2lvbnMgb2YgdGhlIHNhbWUgZmxhZ3MKQEAgLTgyOSw3ICs3NjQsNiBAQAog ZXh0ZXJuIHN0cnVjdCBwcm9jIHByb2MwOwkJLyogUHJvY2VzcyBzbG90IGZv ciBzd2FwcGVyLiAqLwogZXh0ZXJuIHN0cnVjdCB0aHJlYWQgdGhyZWFkMDsJ CS8qIFByaW1hcnkgdGhyZWFkIGluIHByb2MwICovCiBleHRlcm4gc3RydWN0 IGtzZWdycCBrc2VncnAwOwkJLyogUHJpbWFyeSBrc2VncnAgaW4gcHJvYzAg Ki8KLWV4dGVybiBzdHJ1Y3Qga3NlIGtzZTA7CQkJLyogUHJpbWFyeSBrc2Ug aW4gcHJvYzAgKi8KIGV4dGVybiBzdHJ1Y3Qgdm1zcGFjZSB2bXNwYWNlMDsJ CS8qIFZNIHNwYWNlIGZvciBwcm9jMC4gKi8KIGV4dGVybiBpbnQgaG9ndGlj a3M7CQkJLyogTGltaXQgb24ga2VybmVsIGNwdSBob2dzLiAqLwogZXh0ZXJu IGludCBucHJvY3MsIG1heHByb2M7CQkvKiBDdXJyZW50IGFuZCBtYXggbnVt YmVyIG9mIHByb2NzLiAqLwpAQCAtODgwLDggKzgxNCw3IEBACiB2b2lkCXBh cmdzX2hvbGQoc3RydWN0IHBhcmdzICpwYSk7CiB2b2lkCXByb2Npbml0KHZv aWQpOwogdm9pZAl0aHJlYWRpbml0KHZvaWQpOwotdm9pZAlwcm9jX2xpbmt1 cChzdHJ1Y3QgcHJvYyAqcCwgc3RydWN0IGtzZWdycCAqa2csCi0JICAgIHN0 cnVjdCBrc2UgKmtlLCBzdHJ1Y3QgdGhyZWFkICp0ZCk7Cit2b2lkCXByb2Nf bGlua3VwKHN0cnVjdCBwcm9jICpwLCBzdHJ1Y3Qga3NlZ3JwICprZywgc3Ry dWN0IHRocmVhZCAqdGQpOwogdm9pZAlwcm9jX3JlcGFyZW50KHN0cnVjdCBw cm9jICpjaGlsZCwgc3RydWN0IHByb2MgKm5ld3BhcmVudCk7CiBpbnQJc2Vj dXJlbGV2ZWxfZ2Uoc3RydWN0IHVjcmVkICpjciwgaW50IGxldmVsKTsKIGlu dAlzZWN1cmVsZXZlbF9ndChzdHJ1Y3QgdWNyZWQgKmNyLCBpbnQgbGV2ZWwp OwpAQCAtOTEyLDE3ICs4NDUsMTEgQEAKIHN0cnVjdAlrc2VncnAgKmtzZWdy cF9hbGxvYyh2b2lkKTsKIHZvaWQJa3NlZ3JwX2ZyZWUoc3RydWN0IGtzZWdy cCAqa2cpOwogdm9pZAlrc2VncnBfc3Rhc2goc3RydWN0IGtzZWdycCAqa2cp Owotc3RydWN0CWtzZSAqa3NlX2FsbG9jKHZvaWQpOwotdm9pZAlrc2VfZnJl ZShzdHJ1Y3Qga3NlICprZSk7Ci12b2lkCWtzZV9zdGFzaChzdHJ1Y3Qga3Nl ICprZSk7CiB2b2lkCWNwdV9zZXRfdXBjYWxsKHN0cnVjdCB0aHJlYWQgKnRk LCB2b2lkICpwY2IpOwogdm9pZAljcHVfc2V0X3VwY2FsbF9rc2Uoc3RydWN0 IHRocmVhZCAqdGQsIHN0cnVjdCBrc2VfdXBjYWxsICprdSk7CiB2b2lkCWNw dV90aHJlYWRfY2xlYW4oc3RydWN0IHRocmVhZCAqKTsKIHZvaWQJY3B1X3Ro cmVhZF9leGl0KHN0cnVjdCB0aHJlYWQgKik7CiB2b2lkCWNwdV90aHJlYWRf c2V0dXAoc3RydWN0IHRocmVhZCAqdGQpOwotdm9pZAlrc2VfcmVhc3NpZ24o c3RydWN0IGtzZSAqa2UpOwotdm9pZAlrc2VfbGluayhzdHJ1Y3Qga3NlICpr ZSwgc3RydWN0IGtzZWdycCAqa2cpOwotdm9pZAlrc2VfdW5saW5rKHN0cnVj dCBrc2UgKmtlKTsKIHZvaWQJa3NlZ3JwX2xpbmsoc3RydWN0IGtzZWdycCAq a2csIHN0cnVjdCBwcm9jICpwKTsKIHZvaWQJa3NlZ3JwX3VubGluayhzdHJ1 Y3Qga3NlZ3JwICprZyk7CiB2b2lkCXRocmVhZF9zaWduYWxfYWRkKHN0cnVj dCB0aHJlYWQgKnRkLCBpbnQgc2lnKTsKQEAgLTk0Myw2ICs4NzAsNyBAQAog dm9pZAl0aHJlYWRfc3Rhc2goc3RydWN0IHRocmVhZCAqdGQpOwogaW50CXRo cmVhZF9zdXNwZW5kX2NoZWNrKGludCBob3cpOwogdm9pZAl0aHJlYWRfc3Vz cGVuZF9vbmUoc3RydWN0IHRocmVhZCAqdGQpOwordm9pZAl0aHJlYWRfdW5s aW5rKHN0cnVjdCB0aHJlYWQgKnRkKTsKIHZvaWQJdGhyZWFkX3Vuc3VzcGVu ZChzdHJ1Y3QgcHJvYyAqcCk7CiB2b2lkCXRocmVhZF91bnN1c3BlbmRfb25l KHN0cnVjdCB0aHJlYWQgKnRkKTsKIGludAl0aHJlYWRfdXNlcnJldChzdHJ1 Y3QgdGhyZWFkICp0ZCwgc3RydWN0IHRyYXBmcmFtZSAqZnJhbWUpOwpAQCAt OTU0LDcgKzg4Miw3IEBACiB2b2lkCXVwY2FsbF9saW5rKHN0cnVjdCBrc2Vf dXBjYWxsICprdSwgc3RydWN0IGtzZWdycCAqa2cpOwogdm9pZAl1cGNhbGxf dW5saW5rKHN0cnVjdCBrc2VfdXBjYWxsICprdSk7CiB2b2lkCXVwY2FsbF9y ZW1vdmUoc3RydWN0IHRocmVhZCAqdGQpOwotdm9pZAl1cGNhbGxfc3Rhc2go c3RydWN0IGtzZV91cGNhbGwgKmtlKTsKK3ZvaWQJdXBjYWxsX3N0YXNoKHN0 cnVjdCBrc2VfdXBjYWxsICprdSk7CiB2b2lkCXRocmVhZF9zYW5pdHlfY2hl Y2soc3RydWN0IHRocmVhZCAqdGQsIGNoYXIgKik7CiB2b2lkCXRocmVhZF9z dG9wcGVkKHN0cnVjdCBwcm9jICpwKTsKIHZvaWQJdGhyZWFkX3N3aXRjaG91 dChzdHJ1Y3QgdGhyZWFkICp0ZCk7CkluZGV4OiBzeXMvc3lzL3NjaGVkLmgK PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PQpSQ1MgZmlsZTogL3JlcG9zL3Byb2pl Y3RzL21pcnJvcmVkL2ZyZWVic2Qvc3JjL3N5cy9zeXMvc2NoZWQuaCx2CnJl dHJpZXZpbmcgcmV2aXNpb24gMS40CmRpZmYgLXUgLXIxLjQgc2NoZWQuaAot LS0gc3lzL3N5cy9zY2hlZC5oCTIwMDMvMDQvMTEgMDM6Mzk6MDYJMS40Cisr KyBzeXMvc3lzL3NjaGVkLmgJMjAwMy8wNC8xNyAwNjozNTowMQpAQCAtMzks NyArMzksNyBAQAogICogUHJvYyByZWxhdGVkIHNjaGVkdWxpbmcgaG9va3Mu CiAgKi8KIHZvaWQJc2NoZWRfZXhpdChzdHJ1Y3QgcHJvYyAqcCwgc3RydWN0 IHByb2MgKmNoaWxkKTsKLXZvaWQJc2NoZWRfZm9yayhzdHJ1Y3QgcHJvYyAq cCwgc3RydWN0IHByb2MgKmNoaWxkKTsKK3ZvaWQJc2NoZWRfZm9yayhzdHJ1 Y3QgdGhyZWFkICp0ZCwgc3RydWN0IHByb2MgKmNoaWxkKTsKIAogLyoKICAq IEtTRSBHcm91cHMgY29udGFpbiBzY2hlZHVsaW5nIHByaW9yaXR5IGluZm9y bWF0aW9uLiAgVGhleSByZWNvcmQgdGhlCkBAIC02MywxMiArNjMsMTUgQEAK IHZvaWQJc2NoZWRfdXNlcnJldChzdHJ1Y3QgdGhyZWFkICp0ZCk7CiB2b2lk CXNjaGVkX3dha2V1cChzdHJ1Y3QgdGhyZWFkICp0ZCk7CiAKK2ludAlzY2hl ZF90aHJlYWRfZm9yayhzdHJ1Y3QgdGhyZWFkICp0ZDEsIHN0cnVjdCB0aHJl YWQgKnRkMik7CiAvKgogICogS1NFcyBhcmUgbW92ZWQgb24gYW5kIG9mZiBv ZiBydW4gcXVldWVzLgogICovCitzdHJ1Y3Qga3NlOworCiB2b2lkCXNjaGVk X2FkZChzdHJ1Y3Qga3NlICprZSk7CiBzdHJ1Y3Qga3NlICpzY2hlZF9jaG9v c2Uodm9pZCk7Ci12b2lkCXNjaGVkX2Nsb2NrKHN0cnVjdCBrc2UgKmtlKTsK K3ZvaWQJc2NoZWRfY2xvY2soc3RydWN0IHRocmVhZCAqdGQpOwogdm9pZAlz Y2hlZF9leGl0X2tzZShzdHJ1Y3Qga3NlICprZSwgc3RydWN0IGtzZSAqY2hp bGQpOwogdm9pZAlzY2hlZF9mb3JrX2tzZShzdHJ1Y3Qga3NlICprZSwgc3Ry dWN0IGtzZSAqY2hpbGQpOwogdm9pZAlzY2hlZF9yZW0oc3RydWN0IGtzZSAq a2UpOwpAQCAtNzYsNyArNzksNyBAQAogLyoKICAqIGFuZCB0aGV5IHVzZSB1 cCBjcHUgdGltZS4KICAqLwotZml4cHRfdAlzY2hlZF9wY3RjcHUoc3RydWN0 IGtzZSAqa2UpOworZml4cHRfdAlzY2hlZF9wY3RjcHUoc3RydWN0IHRocmVh ZCAqdGQpOwogCiAvKgogICogVGhlc2UgcHJvY2VkdXJlcyB0ZWxsIHRoZSBw cm9jZXNzIGRhdGEgc3RydWN0dXJlIGFsbG9jYXRpb24gY29kZSBob3cKQEAg LTkyLDQgKzk1LDEyIEBACiBleHRlcm4gc3RydWN0IHBfc2NoZWQgKnByb2Mw X3NjaGVkOwogZXh0ZXJuIHN0cnVjdCB0ZF9zY2hlZCAqdGhyZWFkMF9zY2hl ZDsKIAorCisvKiB0ZW1wIGhhY2tzICovCitpbnQJc2NoZWRfbmV3cHJvYyhz dHJ1Y3QgcHJvYyAqcCwgc3RydWN0IGtzZWdycCAqa2csIHN0cnVjdCB0aHJl YWQgKnRkKTsKK3ZvaWQJc2NoZWRfZGJfcHJpbnQoc3RydWN0IHRocmVhZCAq dGQpOwordm9pZAlzY2hlZGluaXQodm9pZCk7Cit2b2lkCXNjaGVkX3RkZXhp dChzdHJ1Y3QgdGhyZWFkICp0ZCk7Cit2b2lkCXNjaGVkX3Rocl9leGl0KHN0 cnVjdCB0aHJlYWQgKnRkKTsKK3ZvaWQJc2NoZWRfZGVzdHJveXByb2Moc3Ry dWN0IHByb2MgKnAsIHN0cnVjdCBrc2VncnAgKmtnLCBzdHJ1Y3QgdGhyZWFk ICp0ZCk7CiAjZW5kaWYgLyogIV9TWVNfU0NIRURfSF8gKi8KAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAc3lzL2tlcm4v c2NoZWRfNGJzZF9rc2UuYwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA ADAxMDA2NDQAMDAwMDAwMAAwMDAwMDAwADAwMDAwMDA3Njc2ADA3NjQ3Mzc2 MTQzADAxNDUyNAAgMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB1c3RhciAgAHJvb3QAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAd2hlZWwAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAKI2luY2x1ZGUgPHN5cy9wYXJhbS5oPgoKI2luY2x1ZGUgPHN5 cy9zeXN0bS5oPgojaW5jbHVkZSA8c3lzL2tlcm5lbC5oPgojaW5jbHVkZSA8 c3lzL2xvY2suaD4KI2luY2x1ZGUgPHN5cy9tYWxsb2MuaD4KI2luY2x1ZGUg PHN5cy9tdXRleC5oPgojaW5jbHVkZSA8c3lzL2tzZXZhci5oPgojaW5jbHVk ZSA8c3lzL3Byb2MuaD4KI2luY2x1ZGUgPHN5cy9zbXAuaD4KI2luY2x1ZGUg PHN5cy9zeXNjdGwuaD4KI2luY2x1ZGUgPHN5cy9zeXNwcm90by5oPgojaW5j bHVkZSA8c3lzL2ZpbGVkZXNjLmg+CiNpbmNsdWRlIDxzeXMvc2NoZWQuaD4K I2luY2x1ZGUgPHN5cy9zaWduYWx2YXIuaD4KI2luY2x1ZGUgPHN5cy9zeC5o PgojaW5jbHVkZSA8c3lzL3R0eS5oPgojaW5jbHVkZSA8c3lzL3VzZXIuaD4K I2luY2x1ZGUgPHN5cy9qYWlsLmg+CiNpbmNsdWRlIDxzeXMva3NlLmg+CiNp bmNsdWRlIDxzeXMva3RyLmg+CiNpbmNsdWRlIDxzeXMvdWNvbnRleHQuaD4K CiNpbmNsdWRlIDx2bS92bS5oPgojaW5jbHVkZSA8dm0vdm1fb2JqZWN0Lmg+ CiNpbmNsdWRlIDx2bS9wbWFwLmg+CiNpbmNsdWRlIDx2bS91bWEuaD4KI2lu Y2x1ZGUgPHZtL3ZtX21hcC5oPgoKCnN0cnVjdCAga3NlIGtzZTA7CnN0YXRp YyB1bWFfem9uZV90IGtzZV96b25lOwpzdGF0aWMgdm9pZCBrc2VfaW5pdCh2 b2lkICptZW0sIGludCBzaXplKTsKCnZvaWQKc2NoZWRpbml0KHZvaWQpCnsK CS8qCgkgKiBTZXQgdXAgdGhlIHNjaGVkdWxlciBzcGVjaWZpYyBwYXJ0cyBv ZiBwcm9jMC4KCSAqLwoJa3NlX2xpbmsoJmtzZTAsICZrc2VncnAwKTsKCWtz ZTAua2Vfc2NoZWQgPSBrc2UwX3NjaGVkOwoJa3NlZ3JwMC5rZ19zY2hlZCA9 IGtzZWdycDBfc2NoZWQ7Cglwcm9jMC5wX3NjaGVkID0gcHJvYzBfc2NoZWQ7 Cgl0aHJlYWQwLnRkX3NjaGVkID0gdGhyZWFkMF9zY2hlZDsKCWtzZTAua2Vf c3RhdGUgPSBLRVNfVEhSRUFEOwoJa3NlMC5rZV90aHJlYWQgPSAmdGhyZWFk MDsKCXRocmVhZDAudGRfa3NlID0gJmtzZTA7IC8qIHdlIGFyZSBydW5uaW5n ICovCgoKCWtzZV96b25lID0gdW1hX3pjcmVhdGUoIktTRSIsIHNjaGVkX3Np emVvZl9rc2UoKSwKCSAgICBOVUxMLCBOVUxMLCBrc2VfaW5pdCwgTlVMTCwg VU1BX0FMSUdOX0NBQ0hFLCAwKTsKfQoKLyoKICogSW5pdGlhbGl6ZSB0eXBl LXN0YWJsZSBwYXJ0cyBvZiBhIGtzZSAod2hlbiBuZXdseSBjcmVhdGVkKS4K ICovCnN0YXRpYyB2b2lkCmtzZV9pbml0KHZvaWQgKm1lbSwgaW50IHNpemUp CnsKCXN0cnVjdCBrc2UgICAgICAqa2U7CgoJa2UgPSAoc3RydWN0IGtzZSAq KW1lbTsKCWtlLT5rZV9zY2hlZCA9IChzdHJ1Y3Qga2Vfc2NoZWQgKikma2Vb MV07Cn0KCi8qCiAqIEtTRSBpcyBsaW5rZWQgaW50byBrc2UgZ3JvdXAuICAg CiAqLwp2b2lkCmtzZV9saW5rKHN0cnVjdCBrc2UgKmtlLCBzdHJ1Y3Qga3Nl Z3JwICprZykKewoJc3RydWN0IHByb2MgKnAgPSBrZy0+a2dfcHJvYzsKCQoJ VEFJTFFfSU5TRVJUX0hFQUQoJmtnLT5rZ19rc2VxLCBrZSwga2Vfa2dsaXN0 KTsKCWtnLT5rZ19rc2VzKys7CglrZS0+a2Vfc3RhdGUgICAgPSBLRVNfVU5R VUVVRUQ7CglrZS0+a2VfcHJvYyAgICAgPSBwOwoJa2UtPmtlX2tzZWdycCAg ID0ga2c7CglrZS0+a2VfdGhyZWFkICAgPSBOVUxMOwoJa2UtPmtlX29uY3B1 ICAgID0gTk9DUFU7CglrZS0+a2VfZmxhZ3MgICAgPSAwOwp9CgovKgogKiBB bGxvY2F0ZSBhIGtzZS4KICovCnN0cnVjdCBrc2UgKgprc2VfYWxsb2Modm9p ZCkKewoJcmV0dXJuICh1bWFfemFsbG9jKGtzZV96b25lLCBNX1dBSVRPSykp Owp9CgovKgogKiBEZWFsbG9jYXRlIGEga3NlLgogKi8Kdm9pZAprc2VfZnJl ZShzdHJ1Y3Qga3NlICp0ZCkKewoJdW1hX3pmcmVlKGtzZV96b25lLCB0ZCk7 Cn0KCmludApzY2hlZF9uZXdwcm9jKHN0cnVjdCBwcm9jICpwLCBzdHJ1Y3Qg a3NlZ3JwICprZywgc3RydWN0IHRocmVhZCAqdGQpCnsKCXN0cnVjdCBrc2Ug KmtlOwoKCWtlID0ga3NlX2FsbG9jKCk7CglpZiAoa2UpIHsKCQlrc2VfbGlu ayhrZSwga2cpOwoJCXJldHVybiAoMCk7Cgl9CglyZXR1cm4gKEVOT01FTSk7 Cn0KCnZvaWQKc2NoZWRfZGVzdHJveXByb2Moc3RydWN0IHByb2MgKnAsIHN0 cnVjdCBrc2VncnAgKmtnLCBzdHJ1Y3QgdGhyZWFkICp0ZCkKewoJc3RydWN0 IGtzZSAqa2U7CgoJa2UgPSB0ZC0+dGRfa3NlOwoJaWYgKGtlKQoJCWtzZV9m cmVlKGtlKTsKfQoKdm9pZAprc2VfdW5saW5rKHN0cnVjdCBrc2UgKmtlKQp7 CglzdHJ1Y3Qga3NlZ3JwICprZzsKCgltdHhfYXNzZXJ0KCZzY2hlZF9sb2Nr LCBNQV9PV05FRCk7CglrZyA9IGtlLT5rZV9rc2VncnA7CglUQUlMUV9SRU1P VkUoJmtnLT5rZ19rc2VxLCBrZSwga2Vfa2dsaXN0KTsKCWlmIChrZS0+a2Vf c3RhdGUgPT0gS0VTX0lETEUpIHsKCQlUQUlMUV9SRU1PVkUoJmtnLT5rZ19p cSwga2UsIGtlX2tncmxpc3QpOwoJCWtnLT5rZ19pZGxlX2tzZXMtLTsKCX0K CWlmICgtLWtnLT5rZ19rc2VzID09IDApCgkJa3NlZ3JwX3VubGluayhrZyk7 CgkvKgoJICogQWdncmVnYXRlIHN0YXRzIGZyb20gdGhlIEtTRQoJICovCglr c2Vfc3Rhc2goa2UpOwp9CgoKCiNkZWZpbmUgUkFOR0VPRih0eXBlLCBzdGFy dCwgZW5kKSAob2Zmc2V0b2YodHlwZSwgZW5kKSAtIG9mZnNldG9mKHR5cGUs IHN0YXJ0KSkKCi8qIG5ldyB2ZXJzaW9uIG9mIHNjaGVkLWZvcmsoKSAqLwp2 b2lkCnNjaGVkX2Zvcmsoc3RydWN0IHRocmVhZCAqdGQsIHN0cnVjdCBwcm9j ICpwMikKewoJc3RydWN0IGtzZSAqa2UyOwoJc3RydWN0IGtzZWdycCAqa2cy OwoJc3RydWN0IHRocmVhZCAqdGQyOwoKICAgICAgICBrZTIgPSBGSVJTVF9L U0VfSU5fUFJPQyhwMik7CiAgICAgICAgdGQyID0gRklSU1RfVEhSRUFEX0lO X1BST0MocDIpOwogICAgICAgIGtnMiA9IEZJUlNUX0tTRUdSUF9JTl9QUk9D KHAyKTsKICAgICAgICBiemVybygma2UyLT5rZV9zdGFydHplcm8sCiAgICAg ICAgICAgICh1bnNpZ25lZCkgUkFOR0VPRihzdHJ1Y3Qga3NlLCBrZV9zdGFy dHplcm8sIGtlX2VuZHplcm8pKTsKICAgICAgICBrZTItPmtlX3N0YXRlID0g S0VTX1RIUkVBRDsKICAgICAgICBrZTItPmtlX3RocmVhZCA9IHRkMjsKCXRk Mi0+dGRfa3NlID0ga2UyOwogICAgICAgIC8qa2UyLT5rZV9zY2hlZC0+c2tl X2NwdGlja3MgPSAwOyAqLyAvKiBYWFggRklYISAqLwogICAgICAgIGtnMi0+ a2dfZXN0Y3B1ID0gdGQtPnRkX2tzZWdycC0+a2dfZXN0Y3B1Owp9CgppbnQK c2NoZWRfdGhyZWFkX2Zvcmsoc3RydWN0IHRocmVhZCAqdGQxLCBzdHJ1Y3Qg dGhyZWFkICp0ZDIpCnsKCXN0cnVjdCBrc2UgKmtlMjsKCS8qIEluaXRpYWxp emUgb3VyIGtzZSBzdHJ1Y3R1cmUuICovCglrZTIgPSBrc2VfYWxsb2MoKTsK CWlmIChrZTIgPT0gTlVMTCkKCQlyZXR1cm4gKEVOT01FTSk7CgliemVybygm a2UyLT5rZV9zdGFydHplcm8sCgkgICAgUkFOR0VPRihzdHJ1Y3Qga3NlLCBr ZV9zdGFydHplcm8sIGtlX2VuZHplcm8pKTsKCWtzZV9saW5rKGtlMiwgdGQy LT50ZF9rc2VncnApOwoJdGQyLT50ZF9rc2UgPSBrZTI7CiAgICAgICAga2Uy LT5rZV90aHJlYWQgPSB0ZDI7CglyZXR1cm4gKDApOwp9Cgp2b2lkCnNjaGVk X3RkZXhpdChzdHJ1Y3QgdGhyZWFkICp0ZCkKewoJc3RydWN0IGtzZSAqa2U7 CgoJa2UgPSB0ZC0+dGRfa3NlOwoJa2UtPmtlX3N0YXRlID0gS0VTX1VOUVVF VUVEOwoJa2UtPmtlX3RocmVhZCA9IE5VTEw7CglpZiAoa2UtPmtlX2ZsYWdz ICYgS0VGX0VYSVQpCgkJa3NlX3VubGluayhrZSk7CgllbHNlCgkJa3NlX3Jl YXNzaWduKGtlKTsKCXRkLT50ZF9rc2UJPSBOVUxMOwoJdGQtPnRkX2xhc3Rf a3NlCT0gTlVMTDsKfQoKdm9pZApzY2hlZF90aHJfZXhpdChzdHJ1Y3QgdGhy ZWFkICp0ZCkKewoJc3RydWN0IGtzZSAqa2U7CgoJa2UgPSB0ZC0+dGRfa3Nl OwoJa2UtPmtlX3N0YXRlID0gS0VTX1VOUVVFVUVEOwoJa2UtPmtlX3RocmVh ZCA9IE5VTEw7Cglrc2VfdW5saW5rKGtlKTsKCXNjaGVkX2V4aXRfa3NlKFRB SUxRX05FWFQoa2UsIGtlX2tnbGlzdCksIGtlKTsKfQoKAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAc3lzL3N5cy9rc2V2YXIuaAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAADAxMDA2NDQAMDAwMDAwMAAwMDAwMDAwADAw MDAwMDExNzA1ADA3NjQ3MzY3MDc3ADAxMzAzMQAgMAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB1c3Rh ciAgAHJvb3QAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAd2hlZWwAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAvKgogKiBDb3B5cmlnaHQgKEMp IDIwMDEgSnVsaWFuIEVsaXNjaGVyIDxqdWxpYW5AZnJlZWJzZC5vcmc+CiAq IGZvciB0aGUgRnJlZUJTRCBGb3VuZGF0aW9uLgogKgogKiAgQWxsIHJpZ2h0 cyByZXNlcnZlZC4KICoKICogUmVkaXN0cmlidXRpb24gYW5kIHVzZSBpbiBz b3VyY2UgYW5kIGJpbmFyeSBmb3Jtcywgd2l0aCBvciB3aXRob3V0CiAqIG1v ZGlmaWNhdGlvbiwgYXJlIHBlcm1pdHRlZCBwcm92aWRlZCB0aGF0IHRoZSBm b2xsb3dpbmcgY29uZGl0aW9ucwogKiBhcmUgbWV0OgogKiAxLiBSZWRpc3Ry aWJ1dGlvbnMgb2Ygc291cmNlIGNvZGUgbXVzdCByZXRhaW4gdGhlIGFib3Zl IGNvcHlyaWdodAogKiAgICBub3RpY2UocyksIHRoaXMgbGlzdCBvZiBjb25k aXRpb25zIGFuZCB0aGUgZm9sbG93aW5nIGRpc2NsYWltZXIgYXMKICogICAg dGhlIGZpcnN0IGxpbmVzIG9mIHRoaXMgZmlsZSB1bm1vZGlmaWVkIG90aGVy IHRoYW4gdGhlIHBvc3NpYmxlIAogKiAgICBhZGRpdGlvbiBvZiBvbmUgb3Ig bW9yZSBjb3B5cmlnaHQgbm90aWNlcy4KICogMi4gUmVkaXN0cmlidXRpb25z IGluIGJpbmFyeSBmb3JtIG11c3QgcmVwcm9kdWNlIHRoZSBhYm92ZSBjb3B5 cmlnaHQKICogICAgbm90aWNlKHMpLCB0aGlzIGxpc3Qgb2YgY29uZGl0aW9u cyBhbmQgdGhlIGZvbGxvd2luZyBkaXNjbGFpbWVyIGluIHRoZQogKiAgICBk b2N1bWVudGF0aW9uIGFuZC9vciBvdGhlciBtYXRlcmlhbHMgcHJvdmlkZWQg d2l0aCB0aGUgZGlzdHJpYnV0aW9uLgogKgogKiBUSElTIFNPRlRXQVJFIElT IFBST1ZJREVEIEJZIFRIRSBDT1BZUklHSFQgSE9MREVSKFMpIGBgQVMgSVMn JyBBTkQgQU5ZCiAqIEVYUFJFU1MgT1IgSU1QTElFRCBXQVJSQU5USUVTLCBJ TkNMVURJTkcsIEJVVCBOT1QgTElNSVRFRCBUTywgVEhFIElNUExJRUQKICog V0FSUkFOVElFUyBPRiBNRVJDSEFOVEFCSUxJVFkgQU5EIEZJVE5FU1MgRk9S IEEgUEFSVElDVUxBUiBQVVJQT1NFIEFSRQogKiBESVNDTEFJTUVELiAgSU4g Tk8gRVZFTlQgU0hBTEwgVEhFIENPUFlSSUdIVCBIT0xERVIoUykgQkUgTElB QkxFIEZPUiBBTlkKICogRElSRUNULCBJTkRJUkVDVCwgSU5DSURFTlRBTCwg U1BFQ0lBTCwgRVhFTVBMQVJZLCBPUiBDT05TRVFVRU5USUFMIERBTUFHRVMK ICogKElOQ0xVRElORywgQlVUIE5PVCBMSU1JVEVEIFRPLCBQUk9DVVJFTUVO VCBPRiBTVUJTVElUVVRFIEdPT0RTIE9SCiAqIFNFUlZJQ0VTOyBMT1NTIE9G IFVTRSwgREFUQSwgT1IgUFJPRklUUzsgT1IgQlVTSU5FU1MgSU5URVJSVVBU SU9OKSBIT1dFVkVSCiAqIENBVVNFRCBBTkQgT04gQU5ZIFRIRU9SWSBPRiBM SUFCSUxJVFksIFdIRVRIRVIgSU4gQ09OVFJBQ1QsIFNUUklDVAogKiBMSUFC SUxJVFksIE9SIFRPUlQgKElOQ0xVRElORyBORUdMSUdFTkNFIE9SIE9USEVS V0lTRSkgQVJJU0lORyBJTiBBTlkgV0FZCiAqIE9VVCBPRiBUSEUgVVNFIE9G IFRISVMgU09GVFdBUkUsIEVWRU4gSUYgQURWSVNFRCBPRiBUSEUgUE9TU0lC SUxJVFkgT0YgU1VDSAogKiBEQU1BR0UuCiAqCiAqICRGcmVlQlNEOiBzcmMv c3lzL3N5cy9rc2UuaCx2IDEuMTIgMjAwMy8wMy8xOSAwNTo0OTozNyBkYXZp ZHh1IEV4cCAkCiAqLwoKI2lmbmRlZiBfU1lTX0tTRVZBUl9IXwojZGVmaW5l IF9TWVNfS1NFVkFSX0hfCiNpZm5kZWYgX0tFUk5FTAojZXJyb3IgIk5vdCBm b3IgdXNlIG91dHNpZGUgdGhlIGtlcm5lbCIKI2Vsc2UKI2RlZmluZSBLU0VE RUYga3NlCi8qCiAqIFRoZSBzZWNvbmQgc3RydWN0dXJlIGlzIHRoZSBLZXJu ZWwgU2NoZWR1bGFibGUgRW50aXR5LiAoS1NFKQogKiBJdCByZXByZXNlbnRz IHRoZSBhYmlsaXR5IHRvIHRha2UgYSBzbG90IGluIHRoZSBzY2hlZHVsZXIg cXVldWUuCiAqIEFzIGxvbmcgYXMgdGhpcyBpcyBzY2hlZHVsZWQsIGl0IGNv dWxkIGNvbnRpbnVlIHRvIHJ1biBhbnkgdGhyZWFkcyB0aGF0CiAqIGFyZSBh c3NpZ25lZCB0byB0aGUgS1NFR1JQIChzZWUgbGF0ZXIpIHVudGlsIGVpdGhl ciBpdCBydW5zIG91dAogKiBvZiBydW5uYWJsZSB0aHJlYWRzIG9mIGhpZ2gg ZW5vdWdoIHByaW9yaXR5LCBvciBDUFUuCiAqIEl0IHJ1bnMgb24gb25lIENQ VSBhbmQgaXMgYXNzaWduZWQgYSBxdWFudHVtIG9mIHRpbWUuIFdoZW4gYSB0 aHJlYWQgaXMKICogYmxvY2tlZCwgVGhlIEtTRSBjb250aW51ZXMgdG8gcnVu IGFuZCB3aWxsIHNlYXJjaCBmb3IgYW5vdGhlciB0aHJlYWQKICogaW4gYSBy dW5uYWJsZSBzdGF0ZSBhbW9uZ3N0IHRob3NlIGl0IGhhcy4gSXQgTWF5IGRl Y2lkZSB0byByZXR1cm4gdG8gdXNlcgogKiBtb2RlIHdpdGggYSBuZXcgJ2Vt cHR5JyB0aHJlYWQgaWYgdGhlcmUgYXJlIG5vIHJ1bm5hYmxlIHRocmVhZHMu CiAqIFRocmVhZHMgYXJlIHRlbXBvcmFyaWx5IGFzc29jaWF0ZWQgd2l0aCBh IEtTRSBmb3Igc2NoZWR1bGluZyByZWFzb25zLgogKi8Kc3RydWN0IGtzZTsK Ci8qKioqKioqKioqKioqKioKICogSW4gcGljdHVyZXM6CiBXaXRoIGEgc2lu Z2xlIHJ1biBxdWV1ZSB1c2VkIGJ5IGFsbCBwcm9jZXNzb3JzOgoKIFJVTlE6 IC0tLT5LU0UtLS1LU0UtLS4uLiAgICAgICAgICAgICAgIFNMRUVQUTpbXS0t LVRIUkVBRC0tLVRIUkVBRC0tLVRIUkVBRAoJICAgfCAgIC8gICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgW10tLS1USFJFQUQKCSAgIEtTRUctLS1U SFJFQUQtLVRIUkVBRC0tVEhSRUFEICAgICAgIFtdCgkJCQkJICAgICAgIFtd LS0tVEhSRUFELS0tVEhSRUFECgogIChwcm9jZXNzb3JzIHJ1biBUSFJFQURz IGZyb20gdGhlIEtTRUcgdW50aWwgdGhleSBhcmUgZXhoYXVzdGVkIG9yCiAg dGhlIEtTRUcgZXhoYXVzdHMgaXRzIHF1YW50dW0pCgpXaXRoIFBFUi1DUFUg cnVuIHF1ZXVlczoKS1NFcyBvbiB0aGUgc2VwYXJhdGUgcnVuIHF1ZXVlcyBk aXJlY3RseQpUaGV5IHdvdWxkIGJlIGdpdmVuIHByaW9yaXRpZXMgY2FsY3Vs YXRlZCBmcm9tIHRoZSBLU0VHLgoKICoKICoqKioqKioqKioqKioqKioqLwoK LyoKICogVGhlIHNjaGVkdWxhYmxlIGVudGl0eSB0aGF0IGNhbiBiZSBnaXZl biBhIGNvbnRleHQgdG8gcnVuLgogKiBBIHByb2Nlc3MgbWF5IGhhdmUgc2V2 ZXJhbCBvZiB0aGVzZS4gUHJvYmFibHkgb25lIHBlciBwcm9jZXNzb3IKICog YnV0IHBvc2libHkgYSBmZXcgbW9yZS4gSW4gdGhpcyB1bml2ZXJzZSB0aGV5 IGFyZSBncm91cGVkCiAqIHdpdGggYSBLU0VHIHRoYXQgY29udGFpbnMgdGhl IHByaW9yaXR5IGFuZCBuaWNlbmVzcwogKiBmb3IgdGhlIGdyb3VwLgogKi8K c3RydWN0IGtzZSB7CglzdHJ1Y3QgcHJvYwkqa2VfcHJvYzsJLyogQXNzb2Np YXRlZCBwcm9jZXNzLiAqLwoJc3RydWN0IGtzZWdycAkqa2Vfa3NlZ3JwOwkv KiBBc3NvY2lhdGVkIEtTRUcuICovCglUQUlMUV9FTlRSWShrc2UpIGtlX2tn bGlzdDsJLyogUXVldWUgb2YgYWxsIEtTRXMgaW4ga2Vfa3NlZ3JwLiAqLwoJ VEFJTFFfRU5UUlkoa3NlKSBrZV9rZ3JsaXN0OwkvKiBRdWV1ZSBvZiBhbGwg S1NFcyBpbiB0aGlzIHN0YXRlLiAqLwoJVEFJTFFfRU5UUlkoa3NlKSBrZV9w cm9jcTsJLyogKGopIFJ1biBxdWV1ZS4gKi8KCiNkZWZpbmUJa2Vfc3RhcnR6 ZXJvIGtlX2ZsYWdzCglpbnQJCWtlX2ZsYWdzOwkvKiAoaikgS0VGXyogZmxh Z3MuICovCglzdHJ1Y3QgdGhyZWFkCSprZV90aHJlYWQ7CS8qIEFjdGl2ZSBh c3NvY2lhdGVkIHRocmVhZC4gKi8KCWZpeHB0X3QJCWtlX3BjdGNwdTsgICAg IC8qIChqKSAlY3B1IGR1cmluZyBwX3N3dGltZS4gKi8KCXVfY2hhcgkJa2Vf b25jcHU7CS8qIChqKSBXaGljaCBjcHUgd2UgYXJlIG9uLiAqLwoJY2hhcgkJ a2VfcnFpbmRleDsJLyogKGopIFJ1biBxdWV1ZSBpbmRleC4gKi8KCWVudW0g ewoJCUtFU19VTlVTRUQgPSAweDAsCgkJS0VTX0lETEUsCgkJS0VTX09OUlVO USwKCQlLRVNfVU5RVUVVRUQsCQkvKiBpbiB0cmFuc2l0ICovCgkJS0VTX1RI UkVBRAkJLyogc2xhdmVkIHRvIHRocmVhZCBzdGF0ZSAqLwoJfSBrZV9zdGF0 ZTsJCQkvKiAoaikgUyogcHJvY2VzcyBzdGF0dXMuICovCiNkZWZpbmUJa2Vf ZW5kemVybyBrZV9kdW1teQoJdV9jaGFyCQlrZV9kdW1teTsKCXN0cnVjdCBr ZV9zY2hlZAkqa2Vfc2NoZWQ7CS8qIFNjaGVkdWxlciBzcGVjaWZpYyBkYXRh ICovCn07CgovKiBmbGFncyBrZXB0IGluIGtlX2ZsYWdzICovCiNkZWZpbmUJ S0VGX0lETEVLU0UJMHgwMDAwNAkvKiBBICdQZXIgQ1BVIGlkbGUgcHJvY2Vz cycuLiBoYXMgb25lIHRocmVhZCAqLwojZGVmaW5lCUtFRl9ESURSVU4JMHgw MjAwMAkvKiBLU0UgYWN0dWFsbHkgcmFuLiAqLwojZGVmaW5lCUtFRl9FWElU CTB4MDQwMDAJLyogS1NFIGlzIGJlaW5nIGtpbGxlZC4gKi8KCiNkZWZpbmUJ Rk9SRUFDSF9LU0VfSU5fR1JPVVAoa2csIGtlKQkJCQkJXAoJVEFJTFFfRk9S RUFDSCgoa2UpLCAmKGtnKS0+a2dfa3NlcSwga2Vfa2dsaXN0KQoKLyogWFhY S1NFIHRoZSBsaW5lcyBiZWxvdyBzaG91bGQgcHJvYmFibHkgb25seSBiZSB1 c2VkIGluIDE6MSBjb2RlICovCiNkZWZpbmUJRklSU1RfS1NFX0lOX0tTRUdS UChrZykgVEFJTFFfRklSU1QoJmtnLT5rZ19rc2VxKQojZGVmaW5lCUZJUlNU X0tTRV9JTl9QUk9DKHApIEZJUlNUX0tTRV9JTl9LU0VHUlAoRklSU1RfS1NF R1JQX0lOX1BST0MocCkpCgpleHRlcm4gc3RydWN0IGtzZSBrc2UwOwkJCS8q IFByaW1hcnkga3NlIGluIHByb2MwICovCgovKiB2b2lkCXByb2NfbGlua3Vw KHN0cnVjdCBwcm9jICpwLCBzdHJ1Y3Qga3NlZ3JwICprZywKCSAgICBzdHJ1 Y3Qga3NlICprZSwgc3RydWN0IHRocmVhZCAqdGQpOyAqLwoKLyogTmV3IGlu IEtTRS4gKi8Kc3RydWN0CWtzZSAqa3NlX2FsbG9jKHZvaWQpOwp2b2lkCWtz ZV9mcmVlKHN0cnVjdCBrc2UgKmtlKTsKdm9pZAlrc2Vfc3Rhc2goc3RydWN0 IGtzZSAqa2UpOwp2b2lkCWtzZV9yZWFzc2lnbihzdHJ1Y3Qga3NlICprZSk7 CnZvaWQJa3NlX2xpbmsoc3RydWN0IGtzZSAqa2UsIHN0cnVjdCBrc2VncnAg KmtnKTsKdm9pZAlrc2VfdW5saW5rKHN0cnVjdCBrc2UgKmtlKTsKCiNlbmRp ZgkvKiAhX0tFUk5FTCAqLwoKI2VuZGlmCS8qICFfU1lTX0tTRVZBUl9IXyAq LwoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= --0-499056429-1050562550=:94222-- From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 00:39:00 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E45E137B404 for ; Thu, 17 Apr 2003 00:39:00 -0700 (PDT) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4C1BE43FBF for ; Thu, 17 Apr 2003 00:39:00 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0126.cvx22-bradley.dialup.earthlink.net ([209.179.198.126] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 1963yo-0005Xe-00; Thu, 17 Apr 2003 00:38:46 -0700 Message-ID: <3E9E59B8.455970F2@mindspring.com> Date: Thu, 17 Apr 2003 00:37:28 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Daniel Eischen References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4f24b8e6c39d4a9f11ac934cea6b21f87387f7b89c61deb1d350badd9bab72f9c350badd9bab72f9c cc: David Xu cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 07:39:01 -0000 Daniel Eischen wrote: > The critical section is to prevent the thread from being > swapped out by the kernel and sent to another KSE. That's > it; it's not meant to do the same thing as locking. > There's per-kse stuff that needs to be accessed that > isn't correct if the thread gets run on another KSE, > regardless of whether or not a lock protects it. I guess the next question is why thread-lending (ala NT, MACH, etc.) isn't supportable. > If it wasn't possible for the kernel to send completed > threads from one KSE to another (within the same KSE > group), we probably wouldn't need critical sections > (at least as currently implemented). Is there a particular performance/other reason this is allowed? -- Terry From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 01:24:19 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2064D37B401; Thu, 17 Apr 2003 01:24:19 -0700 (PDT) Received: from freebsd.org.ru (www.freebsd.org.ru [194.84.67.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1FA3C43FA3; Thu, 17 Apr 2003 01:24:18 -0700 (PDT) (envelope-from osa@freebsd.org.ru) Received: by freebsd.org.ru (Postfix, from userid 1000) id AC05193; Thu, 17 Apr 2003 12:24:12 +0400 (MSD) Date: Thu, 17 Apr 2003 12:24:12 +0400 From: "Sergey A. Osokin" To: John Polstra Message-ID: <20030417082412.GA67305@freebsd.org.ru> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.4i cc: deischen@freebsd.org cc: jeff@freebsd.org cc: freebsd-threads@freebsd.org Subject: Re: FWD: Re: May I add pthread_[gs]etconcurrency to the threads libr X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: osa@FreeBSD.org.ru List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 08:24:19 -0000 On Wed, Apr 16, 2003 at 09:44:09AM -0700, John Polstra wrote: > Sergey, > > FYI -- Dan Eischen asked me not to commit your changes to > libpthread. I then told him he should at least try to use your man > page and credit you appropriately. > > I also told him that he's wrong about returning ENOTSUP, according > to the standards. > > John So, I can't understand Daniel's position. SUSV2 and SUSV3 says: If an implementation does not support multiplexing of user threads on top of several kernel-scheduled entities, the pthread_setconcurrency() and pthread_getconcurrency() functions are provided for source code compatibility but they shall have no effect when called. AFAIK at this time real implementation of that fuctions not yet avaliable. If it not yet avaliable - I think we must use this implementation. In near future, when the other implementations to be avaliable, somebody immediatly replace old fake implementation with new real one. > Date: Wed, 16 Apr 2003 12:35:57 -0400 (EDT) > From: Daniel Eischen > To: John Polstra > Subject: Re: May I add pthread_[gs]etconcurrency to the threads libraries? > Cc: deischen@freebsd.org, jeff@freebsd.org, freebsd-threads@freebsd.org > > On Wed, 16 Apr 2003, John Polstra wrote: > > > Sergey Osokin sent me patches to add the standard > > pthread_[gs]etconcurrency functions to our various threads > > libraries. I reviewed them and they're OK. The functions don't do > > anything significant, but they fill the need for this part of the > > API. > > > > OK if I commit them this weekend? The changes don't change anything > > else. They just add stuff. > > I'm about to implement them for real in libpthread. I'd appreciate > you not adding them to that. I've got a slew of other changes that > I want add to it very soon. > > They don't seem to make sense for libthr and libc_r unless it > returns ENOTSUP. libthr is 1:1, so it is meaning less there > as well as libc_r. > > -- > Dan Eischen -- Rgdz, /"\ ASCII RIBBON CAMPAIGN Sergey Osokin aka oZZ, \ / AGAINST HTML MAIL http://ozz.pp.ru/ X AND NEWS / \ From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 01:31:23 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B70FC37B401 for ; Thu, 17 Apr 2003 01:31:23 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id DE67743F75 for ; Thu, 17 Apr 2003 01:31:22 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h3H8VIO57781; Thu, 17 Apr 2003 04:31:18 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Thu, 17 Apr 2003 04:31:18 -0400 (EDT) From: Jeff Roberson To: Julian Elischer In-Reply-To: Message-ID: <20030417042350.X76635-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: Patches for threads/scheduler abstraction. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 08:31:24 -0000 On Wed, 16 Apr 2003, Julian Elischer wrote: > > > > > > > > My plan is: > > > eventually, sched_4bsd.c will only do the original BSD > > > scheduling (except with threads instead of procsses) > > > i.e. unfair, but simple and easy to benchmark on non-threaded apps. > > > (or on threaded apps with no competing processes) > > > > > > sched_4bsd_kse.c will suck in the current contents of kern_switch.c > > > and the KSE'd version of sched_4bsd.c. > > > > > > this will act like the current 4bsd scheduler WITH KSEs > > > > > > > There is a better way to do this. If you would listen to my tiered > > approach you could end up with one copy of sched_4bsd and not two. > > The kse specific logic could just sit in kern_kse which would not > > tell sched_4bsd about threads until they owned the KSE. We could > > even do this without the kse structure as a go between. > > I think that is an overcomplication. > the two BSD based files would be very different. It's not an overcomplication. The two tasks are independent. > For a start the simple one would be queueing threads on the run queues. > A system compiled with that scheduelr would have no KSEs anywhere > in the entire kernel. > The kse one would be queueing KSEs. I don't see how you can do this > with a shared file. You're missing the point. The scheduler shouldn't be tied to the threading implementation. The threading implementation can pick a KSE to run, assign that thread to the KSE, and then set that thread runnable with the scheduler. If it later decides that the KSE should be running something else it can remove the thread currently assigned to it from the scheduler and then reassign the KSE and add the new thread to the scheduler, which places it on the run queue. This way you will not duplicate code and you will keep the two tasks independant. Essentially the sched_*.c files decide system scope contention while the threading implementation determines the process scope contention which may include some concurrency limits imposed by KSE or some other structure. Do you see? This way we could get KSEs out of the entire kernel other than kern_kse.c and still support them with the sched_4bsd and sched_ule scheduler. Otherwise we're going to have a copy of each scheduler for each threading implementation and we wont be able to support two threading implementations simultaneously. > Anyhow, the following hack (totaly unoptimised.... notice the I think this describes the whole project so far. > existance of sched_td_exit, sched_exit_thread, and sched_thr_exit.. > I just haven't got to cleaning it) is not so advanced that > the question of tiered schedulers is relevant yet.. > this patch just shows that it is possible to get the KSEs > out of the rst of the system. > > > > > > switch.c will go away (unless you want it as part of ULE). > > > > > > kern_thread.c will forget all about KSEs > > > (but its not there yet). > > > > > > I'll send the patches in an hour or so.. > > > My wife's calling :-/ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Julian > > > > > > > > > > > > > > > > > > > > > > > > > > From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 02:24:41 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DEF5E37B401 for ; Thu, 17 Apr 2003 02:24:41 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 466E943FB1 for ; Thu, 17 Apr 2003 02:24:41 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3H9OeBg006029; Thu, 17 Apr 2003 05:24:40 -0400 (EDT) Received: from localhost (eischen@localhost)h3H9OclW006024; Thu, 17 Apr 2003 05:24:38 -0400 (EDT) Date: Thu, 17 Apr 2003 05:24:38 -0400 (EDT) From: Daniel Eischen To: Terry Lambert In-Reply-To: <3E9E59B8.455970F2@mindspring.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: David Xu cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 09:24:42 -0000 On Thu, 17 Apr 2003, Terry Lambert wrote: > Daniel Eischen wrote: > > The critical section is to prevent the thread from being > > swapped out by the kernel and sent to another KSE. That's > > it; it's not meant to do the same thing as locking. > > There's per-kse stuff that needs to be accessed that > > isn't correct if the thread gets run on another KSE, > > regardless of whether or not a lock protects it. > > I guess the next question is why thread-lending (ala NT, > MACH, etc.) isn't supportable. It might still be possible; it's just not as easy. > > If it wasn't possible for the kernel to send completed > > threads from one KSE to another (within the same KSE > > group), we probably wouldn't need critical sections > > (at least as currently implemented). > > Is there a particular performance/other reason this is allowed? I dunno. The original Jasone Evans paper didn't allow this, but the Anderson SA paper did (although our version of KSE's is a bit different than SA). -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 02:45:55 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7D42237B401 for ; Thu, 17 Apr 2003 02:45:55 -0700 (PDT) Received: from exchhz01.viatech.com.cn (ip-167-164-97-218.anlai.com [218.97.164.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id BA88F43F75 for ; Thu, 17 Apr 2003 02:45:53 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from davidw2k (ip-240-1-168-192.rev.dyxnet.com [192.168.1.240]) by exchhz01.viatech.com.cn with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id HLDQQR7L; Thu, 17 Apr 2003 17:31:32 +0800 Message-ID: <001d01c304c6$60b13dd0$f001a8c0@davidw2k> From: "David Xu" To: "Daniel Eischen" , "Terry Lambert" References: Date: Thu, 17 Apr 2003 17:47:36 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 09:45:55 -0000 ----- Original Message -----=20 From: "Daniel Eischen" To: "Terry Lambert" Cc: "David Xu" ; Sent: Thursday, April 17, 2003 5:24 PM Subject: Re: libpthread patch > On Thu, 17 Apr 2003, Terry Lambert wrote: >=20 > > Daniel Eischen wrote: > > > The critical section is to prevent the thread from being > > > swapped out by the kernel and sent to another KSE. That's > > > it; it's not meant to do the same thing as locking. > > > There's per-kse stuff that needs to be accessed that > > > isn't correct if the thread gets run on another KSE, > > > regardless of whether or not a lock protects it. > >=20 > > I guess the next question is why thread-lending (ala NT, > > MACH, etc.) isn't supportable. >=20 > It might still be possible; it's just not as easy. >=20 > > > If it wasn't possible for the kernel to send completed > > > threads from one KSE to another (within the same KSE > > > group), we probably wouldn't need critical sections > > > (at least as currently implemented). > >=20 > > Is there a particular performance/other reason this is allowed? >=20 > I dunno. The original Jasone Evans paper didn't allow this, but > the Anderson SA paper did (although our version of KSE's is a bit > different than SA). >=20 Because there is no benefit for kernel side, whether you use one completed list per-ksegrp or one completed list per-upcall context, there is always competion between threads trying to export their=20 contexts, so using one completed list is simple way to go, if we use completed list per-upcall context, there is other things we would worry about. > --=20 > Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 02:46:10 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EB89437B401; Thu, 17 Apr 2003 02:46:10 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1D37543F85; Thu, 17 Apr 2003 02:46:10 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3H9k8Bg009084; Thu, 17 Apr 2003 05:46:08 -0400 (EDT) Received: from localhost (eischen@localhost)h3H9k7ow009077; Thu, 17 Apr 2003 05:46:08 -0400 (EDT) Date: Thu, 17 Apr 2003 05:46:07 -0400 (EDT) From: Daniel Eischen To: "Sergey A. Osokin" In-Reply-To: <20030417082412.GA67305@freebsd.org.ru> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: deischen@freebsd.org cc: jeff@freebsd.org cc: freebsd-threads@freebsd.org cc: John Polstra Subject: Re: FWD: Re: May I add pthread_[gs]etconcurrency to the threads libr X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 09:46:11 -0000 On Thu, 17 Apr 2003, Sergey A. Osokin wrote: > On Wed, Apr 16, 2003 at 09:44:09AM -0700, John Polstra wrote: > > Sergey, > > > > FYI -- Dan Eischen asked me not to commit your changes to > > libpthread. I then told him he should at least try to use your man > > page and credit you appropriately. > > > > I also told him that he's wrong about returning ENOTSUP, according > > to the standards. > > > > John > > So, I can't understand Daniel's position. Because usually POSIX functions are either fully implemented or not implemented at all. I took a look at the POSIX spec and this is not the case with pthread_[gs]setconcurrency; they do match what you say. > AFAIK at this time real implementation of that fuctions not yet avaliable. > If it not yet avaliable - I think we must use this implementation. > In near future, when the other implementations to be avaliable, > somebody immediatly replace old fake implementation with new real one. You missed my response that said I am implementing them "for real" (not fake) and that I had a slew of other changes. I am a day or so away from commiting my changes (this is for libpthread, not libc_r or libthr). -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 02:52:23 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EC8D937B401; Thu, 17 Apr 2003 02:52:23 -0700 (PDT) Received: from freebsd.org.ru (www.freebsd.org.ru [194.84.67.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0AD5043FB1; Thu, 17 Apr 2003 02:52:23 -0700 (PDT) (envelope-from osa@freebsd.org.ru) Received: by freebsd.org.ru (Postfix, from userid 1000) id 2336793; Thu, 17 Apr 2003 13:52:21 +0400 (MSD) Date: Thu, 17 Apr 2003 13:52:21 +0400 From: "Sergey A. Osokin" To: Daniel Eischen Message-ID: <20030417095221.GC67305@freebsd.org.ru> References: <20030417082412.GA67305@freebsd.org.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.4i cc: deischen@freebsd.org cc: jeff@freebsd.org cc: freebsd-threads@freebsd.org cc: John Polstra Subject: Re: FWD: Re: May I add pthread_[gs]etconcurrency to the threads libr X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: osa@FreeBSD.org.ru List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 09:52:24 -0000 On Thu, Apr 17, 2003 at 05:46:07AM -0400, Daniel Eischen wrote: > On Thu, 17 Apr 2003, Sergey A. Osokin wrote: > > On Wed, Apr 16, 2003 at 09:44:09AM -0700, John Polstra wrote: > > > > > > FYI -- Dan Eischen asked me not to commit your changes to > > > libpthread. I then told him he should at least try to use your man > > > page and credit you appropriately. > > > > > > I also told him that he's wrong about returning ENOTSUP, according > > > to the standards. > > > > So, I can't understand Daniel's position. > > Because usually POSIX functions are either fully implemented or not > implemented at all. I took a look at the POSIX spec and this is not > the case with pthread_[gs]setconcurrency; they do match what you > say. > > > AFAIK at this time real implementation of that fuctions not yet avaliable. > > If it not yet avaliable - I think we must use this implementation. > > In near future, when the other implementations to be avaliable, > > somebody immediatly replace old fake implementation with new real one. > > You missed my response that said I am implementing them "for real" > (not fake) and that I had a slew of other changes. I am a day or > so away from commiting my changes (this is for libpthread, not > libc_r or libthr). Thanks, i see. So, what about other (not libpthread) implementations of threads (like libc_r and libthr)? -- Rgdz, /"\ ASCII RIBBON CAMPAIGN Sergey Osokin aka oZZ, \ / AGAINST HTML MAIL http://ozz.pp.ru/ X AND NEWS / \ From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 06:37:03 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4A05937B401; Thu, 17 Apr 2003 06:37:03 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8C68543FD7; Thu, 17 Apr 2003 06:37:02 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3HDb1Bg009882; Thu, 17 Apr 2003 09:37:01 -0400 (EDT) Received: from localhost (eischen@localhost)h3HDb0vx009848; Thu, 17 Apr 2003 09:37:00 -0400 (EDT) Date: Thu, 17 Apr 2003 09:37:00 -0400 (EDT) From: Daniel Eischen To: "Sergey A. Osokin" In-Reply-To: <20030417095221.GC67305@freebsd.org.ru> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: deischen@freebsd.org cc: jeff@freebsd.org cc: freebsd-threads@freebsd.org cc: John Polstra Subject: Re: FWD: Re: May I add pthread_[gs]etconcurrency to the threads libr X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 13:37:03 -0000 On Thu, 17 Apr 2003, Sergey A. Osokin wrote: > On Thu, Apr 17, 2003 at 05:46:07AM -0400, Daniel Eischen wrote: > > On Thu, 17 Apr 2003, Sergey A. Osokin wrote: > > > On Wed, Apr 16, 2003 at 09:44:09AM -0700, John Polstra wrote: > > > > > > > > FYI -- Dan Eischen asked me not to commit your changes to > > > > libpthread. I then told him he should at least try to use your man > > > > page and credit you appropriately. > > > > > > > > I also told him that he's wrong about returning ENOTSUP, according > > > > to the standards. > > > > > > So, I can't understand Daniel's position. > > > > Because usually POSIX functions are either fully implemented or not > > implemented at all. I took a look at the POSIX spec and this is not > > the case with pthread_[gs]setconcurrency; they do match what you > > say. > > > > > AFAIK at this time real implementation of that fuctions not yet avaliable. > > > If it not yet avaliable - I think we must use this implementation. > > > In near future, when the other implementations to be avaliable, > > > somebody immediatly replace old fake implementation with new real one. > > > > You missed my response that said I am implementing them "for real" > > (not fake) and that I had a slew of other changes. I am a day or > > so away from commiting my changes (this is for libpthread, not > > libc_r or libthr). > > Thanks, i see. So, what about other (not libpthread) implementations > of threads (like libc_r and libthr)? It's OK for those; libc_r will never need anything other than a fake one, and Jeff said it was OK for libthr (Jeff, correct me if I'm wrong). -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 09:16:42 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7A76037B401 for ; Thu, 17 Apr 2003 09:16:42 -0700 (PDT) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id CE21E43F3F for ; Thu, 17 Apr 2003 09:16:41 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0064.cvx21-bradley.dialup.earthlink.net ([209.179.192.64] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 196C3u-0001Ic-00; Thu, 17 Apr 2003 09:16:35 -0700 Message-ID: <3E9ED311.1BC9610D@mindspring.com> Date: Thu, 17 Apr 2003 09:15:13 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Jeff Roberson References: <20030417042350.X76635-100000@mail.chesapeake.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a412eac2785dba42fd9dc9824dbe5d4599350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: Julian Elischer cc: freebsd-threads@freebsd.org Subject: Re: Patches for threads/scheduler abstraction. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 16:16:42 -0000 Jeff Roberson wrote: > On Wed, 16 Apr 2003, Julian Elischer wrote: > > For a start the simple one would be queueing threads on the run queues. > > A system compiled with that scheduelr would have no KSEs anywhere > > in the entire kernel. > > The kse one would be queueing KSEs. I don't see how you can do this > > with a shared file. > > You're missing the point. The scheduler shouldn't be tied to the > threading implementation. I think you will lose CPU affinity and negaffinity, if you do this. I agree that the scheduler shouldn't know about threads, but it has to know about scheduling entities, given that it's, well, a scheduler, after all. Right now, there are too many locks taken in the scheduler path, as it is, and I don't see how concurrency will be improved by doing what you suggest. There is also no clustering support -- for migration of a process from one node to another -- something that can't be done with a scheduler that snoops shared memory, since the memory in question isn't shared. > This way you will not duplicate code and you will keep the two tasks > independant. Essentially the sched_*.c files decide system scope > contention while the threading implementation determines the process scope > contention which may include some concurrency limits imposed by KSE or > some other structure. > > Do you see? This way we could get KSEs out of the entire kernel other > than kern_kse.c and still support them with the sched_4bsd and sched_ule > scheduler. Otherwise we're going to have a copy of each scheduler for > each threading implementation and we wont be able to support two threading > implementations simultaneously. I think this is premature optimization; you're complaining about a multiplicy problem, but the most glaringly obvious multiplicy problem in the scheduling context is the fact that the 4BSD and ULE schedulers can't coexist in the same kernel. I know ULE is your baby, but what is needed is a cleaner abstraction than you are currently suggesting. I think if you go forward with a half-abstraction, which is going to end up setting in concrete the non-coexistance of schedulers, that it would be just as big a mistake as not doing the part that you are talking about. If you want to rename the terminology, you should go ahead and rename it. Mach calls the container abstraction for a process for a scheduler a "task". If you want to call it that instead of "KSE" (or "KSEGRP", which I personally don't like), then go ahead -- BUT there needs to be some type of container, and it needs to be common to all scheduler implementations, or a given implementation won't be able to provide CPU affinity and negaffinity for all the objects in the container class. > > Anyhow, the following hack (totaly unoptimised.... notice the > > I think this describes the whole project so far. 8-) 8-). -- Terry From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 09:21:02 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 87A3237B401; Thu, 17 Apr 2003 09:21:02 -0700 (PDT) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id C4A7F43F75; Thu, 17 Apr 2003 09:21:01 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0064.cvx21-bradley.dialup.earthlink.net ([209.179.192.64] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 196C8C-0002H0-00; Thu, 17 Apr 2003 09:21:01 -0700 Message-ID: <3E9ED420.41E89DDA@mindspring.com> Date: Thu, 17 Apr 2003 09:19:44 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: David Xu References: <001d01c304c6$60b13dd0$f001a8c0@davidw2k> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a412eac2785dba42fded3c9d55f46d4d73350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: Daniel Eischen cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 16:21:02 -0000 David Xu wrote: > > > > If it wasn't possible for the kernel to send completed > > > > threads from one KSE to another (within the same KSE > > > > group), we probably wouldn't need critical sections > > > > (at least as currently implemented). > > > > > > Is there a particular performance/other reason this is allowed? > > > > I dunno. The original Jasone Evans paper didn't allow this, but > > the Anderson SA paper did (although our version of KSE's is a bit > > different than SA). > > Because there is no benefit for kernel side, whether you use one > completed list per-ksegrp or one completed list per-upcall context, > there is always competion between threads trying to export their > contexts, so using one completed list is simple way to go, if we > use completed list per-upcall context, there is other things we > would worry about. So dual-map the pages containing them, into both kernel and user space, and be done with it. Add a seperate read-only mapping to a read/write mapping, and point to one in the other, if you're concerned about protecting some kernel data from the user space process. This doesn't need to be so hard. -- Terry From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 10:19:45 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1DD8737B401 for ; Thu, 17 Apr 2003 10:19:43 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4DBC843FE3 for ; Thu, 17 Apr 2003 10:19:42 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h3HHJco28615; Thu, 17 Apr 2003 13:19:38 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Thu, 17 Apr 2003 13:19:38 -0400 (EDT) From: Jeff Roberson To: Terry Lambert In-Reply-To: <3E9ED311.1BC9610D@mindspring.com> Message-ID: <20030417131733.D76635-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Julian Elischer cc: freebsd-threads@freebsd.org Subject: Re: Patches for threads/scheduler abstraction. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 17:19:46 -0000 Terry, you're frustrating me. So many of these comments are off the mark that I'm not going to address them. You continue to demonstrate that you don't know what you're talking about and yet you feel the need to tell me how it is. I know you're smart enough to get it but right now you're not getting it. Please, go write some code. On Thu, 17 Apr 2003, Terry Lambert wrote: > Jeff Roberson wrote: > > On Wed, 16 Apr 2003, Julian Elischer wrote: > > > For a start the simple one would be queueing threads on the run queues. > > > A system compiled with that scheduelr would have no KSEs anywhere > > > in the entire kernel. > > > The kse one would be queueing KSEs. I don't see how you can do this > > > with a shared file. > > > > You're missing the point. The scheduler shouldn't be tied to the > > threading implementation. > > I think you will lose CPU affinity and negaffinity, if you do this. > > I agree that the scheduler shouldn't know about threads, but it > has to know about scheduling entities, given that it's, well, a > scheduler, after all. > > Right now, there are too many locks taken in the scheduler path, > as it is, and I don't see how concurrency will be improved by > doing what you suggest. > > There is also no clustering support -- for migration of a process > from one node to another -- something that can't be done with a > scheduler that snoops shared memory, since the memory in question > isn't shared. > > > This way you will not duplicate code and you will keep the two tasks > > independant. Essentially the sched_*.c files decide system scope > > contention while the threading implementation determines the process scope > > contention which may include some concurrency limits imposed by KSE or > > some other structure. > > > > Do you see? This way we could get KSEs out of the entire kernel other > > than kern_kse.c and still support them with the sched_4bsd and sched_ule > > scheduler. Otherwise we're going to have a copy of each scheduler for > > each threading implementation and we wont be able to support two threading > > implementations simultaneously. > > I think this is premature optimization; you're complaining about > a multiplicy problem, but the most glaringly obvious multiplicy > problem in the scheduling context is the fact that the 4BSD and > ULE schedulers can't coexist in the same kernel. I know ULE is > your baby, but what is needed is a cleaner abstraction than you > are currently suggesting. > > I think if you go forward with a half-abstraction, which is going > to end up setting in concrete the non-coexistance of schedulers, > that it would be just as big a mistake as not doing the part that > you are talking about. > > If you want to rename the terminology, you should go ahead and > rename it. Mach calls the container abstraction for a process > for a scheduler a "task". If you want to call it that instead > of "KSE" (or "KSEGRP", which I personally don't like), then go > ahead -- BUT there needs to be some type of container, and it > needs to be common to all scheduler implementations, or a given > implementation won't be able to provide CPU affinity and negaffinity > for all the objects in the container class. > > > > > Anyhow, the following hack (totaly unoptimised.... notice the > > > > I think this describes the whole project so far. > > 8-) 8-). > > -- Terry > From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 11:00:35 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3442137B41E for ; Thu, 17 Apr 2003 11:00:35 -0700 (PDT) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id B9A2C43FDD for ; Thu, 17 Apr 2003 11:00:33 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0064.cvx21-bradley.dialup.earthlink.net ([209.179.192.64] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 196DgR-0007Ke-00; Thu, 17 Apr 2003 11:00:28 -0700 Message-ID: <3E9EEB64.A193DB99@mindspring.com> Date: Thu, 17 Apr 2003 10:59:00 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Jeff Roberson References: <20030417131733.D76635-100000@mail.chesapeake.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4c67aa86b005786abdc878b51e6e19fa03ca473d225a0f487350badd9bab72f9c350badd9bab72f9c cc: Julian Elischer cc: freebsd-threads@freebsd.org Subject: Re: Patches for threads/scheduler abstraction. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 18:00:35 -0000 Jeff Roberson wrote: > Terry, you're frustrating me. So many of these comments are off the mark > that I'm not going to address them. You continue to demonstrate that you > don't know what you're talking about and yet you feel the need to tell me > how it is. > > I know you're smart enough to get it but right now you're not getting it. You are complaining about having a scheduler per threading model as a result of having the KSE stuff around in the kernel, and having to swap out files as a result, and getting a multiplicity of kernels. Fine. I agree. Let me think out loud, state my conclusions, and show my work so that there will be crystal clarity, and neither you nor I will end up frustrated. If my understanding is flawed, then I will accept corection willingly. Right now, there are two schedulers and two threading models. Worst case, this gives you 4 kernels. You want to make this 2 kernels, by making the threading models coexist, and the schedulers not coexist. It's also possible to make this 2 kernels by making the schedulers coexist and the threading models not coexist. Now what I'm saying, which is frustrating you because you do not want to hear it is that it's possible to have 1 kernel, by making both the threading models and the schedulers coexist. For this to work you can't pass around "processes", you have to pass around these abstract things that contain a scheduler specific part and a threading model specific part, and which are opaque to the scheduler API, as far as the threading model goes, and opaque to the threading API, as far as the scheduling model goes. I can understand your point of view here: you want a cleaner API. Have I made myself clear so far? I don't want any misunderstanding. Now to me, this means that when you create a process, you end up with something attached to it that is threading model specific, and something else attached to it that is scheduler specific. Historically, for the SMPng stuff that supports an N:M model, we've called these things "KSE" and "KSEGRP", specifically, and we've *assumed* a scheduler implementation that operates on the basis of "KSEGRP". I don't like the abstraction, but I can see the necessity for supporting multiple threading models, if that's what you want to be able to do. Personally, I don't care *what* they are called any more; the issue that's the problem here is that the thing you want to pass around in the scheduler API pretty much doesn't permit per-threading model information in the container object. Perhaps my understanding of your proposal is flawed; if that's the case, then I'd like to be educated, rather than dismissed with no explanation as to what I can do to correct my understanding. Thanks, -- Terry From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 15:29:31 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AE1F237B401 for ; Thu, 17 Apr 2003 15:29:31 -0700 (PDT) Received: from sccrmhc02.attbi.com (sccrmhc02.attbi.com [204.127.202.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id AB90143FCB for ; Thu, 17 Apr 2003 15:29:30 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by sccrmhc02.attbi.com (sccrmhc02) with ESMTP id <2003041722292900200aqg7le>; Thu, 17 Apr 2003 22:29:29 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id PAA55653; Thu, 17 Apr 2003 15:29:28 -0700 (PDT) Date: Thu, 17 Apr 2003 15:29:26 -0700 (PDT) From: Julian Elischer To: Jeff Roberson In-Reply-To: <20030417042350.X76635-100000@mail.chesapeake.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: Patches for threads/scheduler abstraction. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2003 22:29:32 -0000 On Thu, 17 Apr 2003, Jeff Roberson wrote: > > On Wed, 16 Apr 2003, Julian Elischer wrote: > > > > > > > > > > > > > My plan is: > > > > eventually, sched_4bsd.c will only do the original BSD > > > > scheduling (except with threads instead of procsses) > > > > i.e. unfair, but simple and easy to benchmark on non-threaded apps. > > > > (or on threaded apps with no competing processes) > > > > > > > > sched_4bsd_kse.c will suck in the current contents of kern_switch.c > > > > and the KSE'd version of sched_4bsd.c. > > > > > > > > this will act like the current 4bsd scheduler WITH KSEs > > > > > > > > > > There is a better way to do this. If you would listen to my tiered > > > approach you could end up with one copy of sched_4bsd and not two. > > > The kse specific logic could just sit in kern_kse which would not > > > tell sched_4bsd about threads until they owned the KSE. We could > > > even do this without the kse structure as a go between. > > > > I think that is an overcomplication. > > the two BSD based files would be very different. > > It's not an overcomplication. The two tasks are independent. I'm missing something here. I think our problem is one of definitions.. in my vocb. the threading system: has an API to allow user space to create/delete threads and specify behaviour of threads.. it knows about processes, KSEGRPS (subprocesses?, sched_groups?) and threads. it doesn't know about KSEs. the scheduler: Takes requests from the general OS to make threads runnable, and takes requests to find a new thread to run (Bfrom switch()). It also has some entrypoints for such things as clock tick notification, and entrypoints for events such as a fork, exit, thread_creation, thread destruction. It contains code to recalculate priorities and to change them. It may, internally use KSEs or it may not. That is a decision for the writer. It only communicatges with the higher layers only in terms of threads, ksegrps (sched_groups or whatever they get called) and processes. It has private storage in all those structures which may be used to implement all sorts of data structures (e.g. run queues, or priority lists or whatever it wants). Are you saying that you want the threading system to know about KSEs? if so, WHY? especially when some schedulers may NOT know about KSEs. > > > For a start the simple one would be queueing threads on the run queues. > > A system compiled with that scheduelr would have no KSEs anywhere > > in the entire kernel. > > The kse one would be queueing KSEs. I don't see how you can do this > > with a shared file. > > You're missing the point. The scheduler shouldn't be tied to the > threading implementation. It wouldn't be if the threading implementation didn't need to know about KSEs. Or are you saying that you want KSEs to be what the lower end of the threading system uses and passes to teh scheduler, even if the scheduler has no other need for them except to extract a thread out of it and ignore the KSE? > > The threading implementation can pick a KSE to run, assign that thread to > the KSE, and then set that thread runnable with the scheduler. If it > later decides that the KSE should be running something else it can remove > the thread currently assigned to it from the scheduler and then reassign > the KSE and add the new thread to the scheduler, which places it on the > run queue. I think I'm seeing the problem here.. We hav different ideas of where the boundaries of the threading and scheduling modules are. You are saying that the threading implementation takes part in the scheduling.. I on the other hand don't think that is right.. The SCHEDULER takes part in the scheduling. the threading system just exists in order to allow the system to create and destroy threads. The scheduling system is responsible for taking requests to make a thread runnable (from many parts of the system) and for deciding what thread is returned to switch() when it asks for a new thread context to load and run. > > This way you will not duplicate code and you will keep the two tasks > independant. Essentially the sched_*.c files decide system scope > contention while the threading implementation determines the process scope > contention which may include some concurrency limits imposed by KSE or > some other structure. Ok, so you want to schedule a KSE and once you have decided what KSE is run , you allow the threading system to map that to a thread, or, maybe a sequence of threads.. is that right? The difference is that I don't see that as part of the threading system. I see that as part of the scheduling system.. If it makes things better, I can agree that there is a posibility that the scheduler could be implemented as two parts.. The basic scheduler that decides what ksegrp(sched_group, subproc would be better names) is going to be allowed to get time to run threads, and a thread scheduler that decides which threads within that larger entity are going to be run on its behalf. The difference is that I don't think that that lower part is part of the thread system. I think it is part of the scheduler. > > Do you see? This way we could get KSEs out of the entire kernel other > than kern_kse.c and still support them with the sched_4bsd and sched_ule > scheduler. Otherwise we're going to have a copy of each scheduler for > each threading implementation and we wont be able to support two threading > implementations simultaneously. I'm afraid we have such a wide terminology difference here that the only way we are going to be able to truely understand what we both mean is with pictures and examples. > > > Anyhow, the following hack (totaly unoptimised.... notice the > > > existance of sched_td_exit, sched_exit_thread, and sched_thr_exit.. > > I just haven't got to cleaning it) is not so advanced that > > the question of tiered schedulers is relevant yet.. > > this patch just shows that it is possible to get the KSEs > > out of the rst of the system. Which you have agreed is not a bad idea. From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 22:22:16 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CB64A37B401 for ; Thu, 17 Apr 2003 22:22:16 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2BD6E43FDF for ; Thu, 17 Apr 2003 22:22:16 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3I5MFBg026864; Fri, 18 Apr 2003 01:22:15 -0400 (EDT) Received: from localhost (eischen@localhost)h3I5MDoo026859; Fri, 18 Apr 2003 01:22:15 -0400 (EDT) Date: Fri, 18 Apr 2003 01:22:13 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: <004c01c30470$9e36ddf0$0701a8c0@tiger> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 05:22:17 -0000 On Thu, 17 Apr 2003, David Xu wrote: > > ----- Original Message ----- > From: "Daniel Eischen" > To: "David Xu" > Cc: > Sent: Thursday, April 17, 2003 5:05 AM > Subject: Re: libpthread patch > > > > There's a new patch available at: > > > > http://people.freebsd.org/~deischen/kse/libpthread.diffs > > > > This passes all the ACE tests that libc_r passes, with the > > exception of Cached_Conn_Test. > > > > It also seems to work with KDE, konqueror, kwrite, kmail, etc. > > I don't have mozilla built (and am dreading trying to), but > > it would be interesting to see if it works with that. > > > > Cool! > > > If no-one has any objections, I'd like to commit this > > soon. I'll let David review and comment to it first. > > > > David, I didn't add critical regions to _thr_alloc() and > > _thr_free(). I think that whenever they are used, we > > are already in a critical region or operating on an upcall. > > > > Hmm, I don't like to put malloc calling under critical section, > it is better to put it under a lock, otherwise this would cause dead > lock. suppose that an user thread is calling malloc(), and heap manager > got malloc spinlock, then it does somethings and the thread is preempted > by upcall from kernel, now UTS switches to another thread, that thread > starts to call pthread_create, so UTS kernel enters a critical region first, > and calls malloc, this would cause dead lock, because UTS is under critical > region and no context switch could happen. > Also I don't like thr_free under critical region, I think a GC thread is still > needed to recycle zombie thread and free extra memory, UTS kernel > should't be blocked by user thread. Despite this, I think the patch should > be committed. I tried to rework this based on your idea of doing it at thread allocation time. I just committed it. There are a few issues we've got to go over, as well as looking closely at any locking order problems. One thing, which I think you agreed to some weeks/months ago, was to have the kernel set flag in the KSE mailbox in the kse_exit() system call. This is so the UTS can know that the KSE and it's stack is truly done. I've got some code in thr_kern.c that is commented out and is expecting that this will get done. Is that still something we can do? One other thing. Is there a way to wake up a KSE without having an upcall? For instance, in _kse_lock_wait(), we do something like this: /* * Enter a loop to wait until we get the lock. */ ts.tv_sec = 0; ts.tv_nsec = 1000000; /* 1 sec */ KSE_SET_WAIT(curkse); while (_LCK_BUSY(lu)) { /* * Yield the kse and wait to be notified when the lock * is granted. */ crit = _kse_critical_enter(); __sys_nanosleep(&ts, NULL); _kse_critical_leave(crit); /* * Make sure that the wait flag is set again in case * we wokeup without the lock being granted. */ KSE_SET_WAIT(curkse); } KSE_CLEAR_WAIT(curkse); Can it be woken with kse_wakeup() or possible kse_thr_interrupt() (on a KSE mailbox) so that the nanosleep just returns? I used nanosleep, because kse_release() will force a new upcall. Hmm, perhaps we can have a parameter on kse_release to specify whether we want a normal return or a new upcall. I know that you have a patch for KSEs that never want to have an upcall, but that wouldn't work for this case where we want the KSE to upcall normally. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 22:56:02 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 66BDE37B401 for ; Thu, 17 Apr 2003 22:56:02 -0700 (PDT) Received: from exchhz01.viatech.com.cn (ip-167-164-97-218.anlai.com [218.97.164.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5821B43FAF for ; Thu, 17 Apr 2003 22:56:00 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from davidw2k (ip-240-1-168-192.rev.dyxnet.com [192.168.1.240]) by exchhz01.viatech.com.cn with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id HLDQQVJH; Fri, 18 Apr 2003 13:41:49 +0800 Message-ID: <002f01c3056f$6d9b7840$f001a8c0@davidw2k> From: "David Xu" To: "Daniel Eischen" References: Date: Fri, 18 Apr 2003 13:57:43 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 05:56:02 -0000 ----- Original Message -----=20 From: "Daniel Eischen" To: "David Xu" Cc: Sent: Friday, April 18, 2003 1:22 PM Subject: Re: libpthread patch > I tried to rework this based on your idea of doing it at > thread allocation time. I just committed it. >=20 Good! > There are a few issues we've got to go over, as well as > looking closely at any locking order problems. >=20 I have ever tried to port some kernel code to userland (e.g mutex and witness), but now they were left there for several days without be touched. > One thing, which I think you agreed to some weeks/months ago, > was to have the kernel set flag in the KSE mailbox in the > kse_exit() system call. This is so the UTS can know that > the KSE and it's stack is truly done. I've got some code > in thr_kern.c that is commented out and is expecting that > this will get done. Is that still something we can do? >=20 I think you can use kse_wakeup for exiting kse, if the kse is no longer used by kernel, kse_wakeup will return ESRCH, it is not perfect, but should work. > One other thing. Is there a way to wake up a KSE without > having an upcall? For instance, in _kse_lock_wait(), we > do something like this: >=20 > /* > * Enter a loop to wait until we get the lock. > */ > ts.tv_sec =3D 0; > ts.tv_nsec =3D 1000000; /* 1 sec */ > KSE_SET_WAIT(curkse); > while (_LCK_BUSY(lu)) { > /* > * Yield the kse and wait to be notified when the lock > * is granted. > */ > crit =3D _kse_critical_enter(); > __sys_nanosleep(&ts, NULL); > _kse_critical_leave(crit); >=20 > /* > * Make sure that the wait flag is set again in case > * we wokeup without the lock being granted. > */ > KSE_SET_WAIT(curkse); > } > KSE_CLEAR_WAIT(curkse); >=20 > Can it be woken with kse_wakeup() or possible kse_thr_interrupt() > (on a KSE mailbox) so that the nanosleep just returns? I used > nanosleep, because kse_release() will force a new upcall. >=20 > Hmm, perhaps we can have a parameter on kse_release to specify > whether we want a normal return or a new upcall. >=20 > I know that you have a patch for KSEs that never want to > have an upcall, but that wouldn't work for this case where > we want the KSE to upcall normally. >=20 the patch http://people.freebsd.org/~davidxu/kse_release.diff may be what you want. there is two flags you can dynamically set/clear in km_flags, they are not static, it means you can change them at runtime. KMF_NOUPCALL tell kernel to not schedule an upcall for blocked syscall or for kse_release(), this too can be used to poll completed context without restarting from upcall entry. KMF_NOCOMPLETED tell kernel to not bring completed context back to userland when doing upcall. if you set KMF_NOUPCALL at same time, it can be used for a kse trying to get a lock in critical section where it can not process additional work. > --=20 > Dan Eischen David Xu From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 23:04:35 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 931) id 762EB37B401; Thu, 17 Apr 2003 23:04:35 -0700 (PDT) Date: Fri, 18 Apr 2003 01:04:35 -0500 From: Juli Mallett To: David Xu Message-ID: <20030418010435.A6251@FreeBSD.org> References: <002f01c3056f$6d9b7840$f001a8c0@davidw2k> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <002f01c3056f$6d9b7840$f001a8c0@davidw2k>; from davidxu@freebsd.org on Fri, Apr 18, 2003 at 01:57:43PM +0800 Organisation: The FreeBSD Project X-Alternate-Addresses: , , , , X-Towel: Yes X-Negacore: Yes X-Title: Code Maven X-Authentication-Warning: localhost: juli pwned teh intarweb cc: Daniel Eischen cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 06:04:35 -0000 * De: David Xu [ Data: 2003-04-18 ] [ Subjecte: Re: libpthread patch ] > > There are a few issues we've got to go over, as well as > > looking closely at any locking order problems. > > > I have ever tried to port some kernel code to userland (e.g > mutex and witness), but now they were left there for > several days without be touched. This seems like overkill, in fact, it is by definition. If you want some primitive private-locks-only mutex tracing/auditing, I've done a bit of that and could give you some hints. Using the casuptr facility introduced by thr may be a good idea, no? It is known to work, and is relatively un-complex? Or am I missing something? -- juli mallett. email: jmallett@freebsd.org; aim: bsdflata; efnet: juli; From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 23:07:45 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1297B37B401; Thu, 17 Apr 2003 23:07:45 -0700 (PDT) Received: from exchhz01.viatech.com.cn (ip-167-164-97-218.anlai.com [218.97.164.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id CEC8E43FCB; Thu, 17 Apr 2003 23:07:41 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from davidw2k (ip-240-1-168-192.rev.dyxnet.com [192.168.1.240]) by exchhz01.viatech.com.cn with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id HLDQQVKV; Fri, 18 Apr 2003 13:53:29 +0800 Message-ID: <003d01c30571$0edb1cf0$f001a8c0@davidw2k> From: "David Xu" To: "Juli Mallett" References: <002f01c3056f$6d9b7840$f001a8c0@davidw2k> <20030418010435.A6251@FreeBSD.org> Date: Fri, 18 Apr 2003 14:09:23 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 cc: Daniel Eischen cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 06:07:45 -0000 ----- Original Message -----=20 From: "Juli Mallett" To: "David Xu" Cc: "Daniel Eischen" ; = Sent: Friday, April 18, 2003 2:04 PM Subject: Re: libpthread patch > * De: David Xu [ Data: 2003-04-18 ] > [ Subjecte: Re: libpthread patch ] > > > There are a few issues we've got to go over, as well as > > > looking closely at any locking order problems. > > >=20 > > I have ever tried to port some kernel code to userland (e.g > > mutex and witness), but now they were left there for > > several days without be touched. >=20 > This seems like overkill, in fact, it is by definition. If you > want some primitive private-locks-only mutex tracing/auditing, > I've done a bit of that and could give you some hints. Using the > casuptr facility introduced by thr may be a good idea, no? It > is known to work, and is relatively un-complex? Or am I missing > something? I want to use code to detect LOR not just human eyes, I can accept any reasonable method. > --=20 > juli mallett. email: jmallett@freebsd.org; aim: bsdflata; efnet: juli; From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 23:12:25 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9965737B401; Thu, 17 Apr 2003 23:12:25 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9F34843F75; Thu, 17 Apr 2003 23:12:24 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3I6CNBg003282; Fri, 18 Apr 2003 02:12:23 -0400 (EDT) Received: from localhost (eischen@localhost)h3I6CNod003279; Fri, 18 Apr 2003 02:12:23 -0400 (EDT) Date: Fri, 18 Apr 2003 02:12:23 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: <003d01c30571$0edb1cf0$f001a8c0@davidw2k> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 06:12:25 -0000 On Fri, 18 Apr 2003, David Xu wrote: > > ----- Original Message ----- > From: "Juli Mallett" > To: "David Xu" > Cc: "Daniel Eischen" ; > Sent: Friday, April 18, 2003 2:04 PM > Subject: Re: libpthread patch > > > > * De: David Xu [ Data: 2003-04-18 ] > > [ Subjecte: Re: libpthread patch ] > > > > There are a few issues we've got to go over, as well as > > > > looking closely at any locking order problems. > > > > > > > I have ever tried to port some kernel code to userland (e.g > > > mutex and witness), but now they were left there for > > > several days without be touched. > > > > This seems like overkill, in fact, it is by definition. If you > > want some primitive private-locks-only mutex tracing/auditing, > > I've done a bit of that and could give you some hints. Using the > > casuptr facility introduced by thr may be a good idea, no? It > > is known to work, and is relatively un-complex? Or am I missing > > something? > > I want to use code to detect LOR not just human eyes, I can accept > any reasonable method. We can do that now with the locks that I have in place. Each consumer of a lock has a "lock user". Threads and KSEs have an array of 3 lock users; probably 2 is enough because I don't think we need more than a nesting of 2. When you decrement the lock user index when releasing a lock, you make sure that the lock being released matches the one owned. In fact, I implemented it this way so you couldn't possible have lock order reversals. The locks will not work if you reverse them. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 23:25:18 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8235137B401 for ; Thu, 17 Apr 2003 23:25:18 -0700 (PDT) Received: from exchhz01.viatech.com.cn (ip-167-164-97-218.anlai.com [218.97.164.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id D6C3743F85 for ; Thu, 17 Apr 2003 23:25:14 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from davidw2k (ip-240-1-168-192.rev.dyxnet.com [192.168.1.240]) by exchhz01.viatech.com.cn with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id HLDQQVML; Fri, 18 Apr 2003 14:11:01 +0800 Message-ID: <005b01c30573$821cf6a0$f001a8c0@davidw2k> From: "David Xu" To: "Daniel Eischen" References: Date: Fri, 18 Apr 2003 14:26:55 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 06:25:18 -0000 ----- Original Message -----=20 From: "Daniel Eischen" To: "David Xu" Cc: Sent: Friday, April 18, 2003 2:12 PM Subject: Re: libpthread patch > On Fri, 18 Apr 2003, David Xu wrote: >=20 > >=20 > > ----- Original Message -----=20 > > From: "Juli Mallett" > > To: "David Xu" > > Cc: "Daniel Eischen" ; = > > Sent: Friday, April 18, 2003 2:04 PM > > Subject: Re: libpthread patch > >=20 > >=20 > > > * De: David Xu [ Data: 2003-04-18 ] > > > [ Subjecte: Re: libpthread patch ] > > > > > There are a few issues we've got to go over, as well as > > > > > looking closely at any locking order problems. > > > > >=20 > > > > I have ever tried to port some kernel code to userland (e.g > > > > mutex and witness), but now they were left there for > > > > several days without be touched. > > >=20 > > > This seems like overkill, in fact, it is by definition. If you > > > want some primitive private-locks-only mutex tracing/auditing, > > > I've done a bit of that and could give you some hints. Using the > > > casuptr facility introduced by thr may be a good idea, no? It > > > is known to work, and is relatively un-complex? Or am I missing > > > something? > >=20 > > I want to use code to detect LOR not just human eyes, I can accept > > any reasonable method. >=20 > We can do that now with the locks that I have in place. > Each consumer of a lock has a "lock user". Threads and > KSEs have an array of 3 lock users; probably 2 is enough > because I don't think we need more than a nesting of 2. > When you decrement the lock user index when releasing > a lock, you make sure that the lock being released > matches the one owned. In fact, I implemented it this > way so you couldn't possible have lock order reversals. > The locks will not work if you reverse them. >=20 witeness in kernel records locking order, not lock instance. for example, at one time, code uses locking order mutex1 mutex2, and release them, next time if another code locks them in the order mutex2 mutex1, it would detect it. > --=20 > Dan Eischen >=20 > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to = "freebsd-threads-unsubscribe@freebsd.org" From owner-freebsd-threads@FreeBSD.ORG Thu Apr 17 23:28:34 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ABD7937B401; Thu, 17 Apr 2003 23:28:34 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id E431143FA3; Thu, 17 Apr 2003 23:28:33 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3I6SXBg005531; Fri, 18 Apr 2003 02:28:33 -0400 (EDT) Received: from localhost (eischen@localhost)h3I6SW5m005528; Fri, 18 Apr 2003 02:28:32 -0400 (EDT) Date: Fri, 18 Apr 2003 02:28:32 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: <005b01c30573$821cf6a0$f001a8c0@davidw2k> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 06:28:35 -0000 On Fri, 18 Apr 2003, David Xu wrote: > > ----- Original Message ----- > From: "Daniel Eischen" > To: "David Xu" > Cc: > Sent: Friday, April 18, 2003 2:12 PM > Subject: Re: libpthread patch > > > > On Fri, 18 Apr 2003, David Xu wrote: > > > > > > > > ----- Original Message ----- > > > From: "Juli Mallett" > > > To: "David Xu" > > > Cc: "Daniel Eischen" ; > > > Sent: Friday, April 18, 2003 2:04 PM > > > Subject: Re: libpthread patch > > > > > > > > > > * De: David Xu [ Data: 2003-04-18 ] > > > > [ Subjecte: Re: libpthread patch ] > > > > > > There are a few issues we've got to go over, as well as > > > > > > looking closely at any locking order problems. > > > > > > > > > > > I have ever tried to port some kernel code to userland (e.g > > > > > mutex and witness), but now they were left there for > > > > > several days without be touched. > > > > > > > > This seems like overkill, in fact, it is by definition. If you > > > > want some primitive private-locks-only mutex tracing/auditing, > > > > I've done a bit of that and could give you some hints. Using the > > > > casuptr facility introduced by thr may be a good idea, no? It > > > > is known to work, and is relatively un-complex? Or am I missing > > > > something? > > > > > > I want to use code to detect LOR not just human eyes, I can accept > > > any reasonable method. > > > > We can do that now with the locks that I have in place. > > Each consumer of a lock has a "lock user". Threads and > > KSEs have an array of 3 lock users; probably 2 is enough > > because I don't think we need more than a nesting of 2. > > When you decrement the lock user index when releasing > > a lock, you make sure that the lock being released > > matches the one owned. In fact, I implemented it this > > way so you couldn't possible have lock order reversals. > > The locks will not work if you reverse them. > > > > witeness in kernel records locking order, not lock instance. > for example, at one time, code uses locking order > mutex1 mutex2, and release them, next time if another > code locks them in the order mutex2 mutex1, it would detect > it. Ahh, OK. There aren't that many locks used by the library. We can probably come up with something that does what you want. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Fri Apr 18 00:23:29 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 931) id 4772A37B401; Fri, 18 Apr 2003 00:23:29 -0700 (PDT) Date: Fri, 18 Apr 2003 02:23:29 -0500 From: Juli Mallett To: Daniel Eischen Message-ID: <20030418022329.A11963@FreeBSD.org> References: <005b01c30573$821cf6a0$f001a8c0@davidw2k> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: ; from eischen@pcnet1.pcnet.com on Fri, Apr 18, 2003 at 02:28:32AM -0400 Organisation: The FreeBSD Project X-Alternate-Addresses: , , , , X-Towel: Yes X-Negacore: Yes X-Title: Code Maven X-Authentication-Warning: localhost: juli pwned teh intarweb cc: David Xu cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 07:23:29 -0000 * De: Daniel Eischen [ Data: 2003-04-18 ] [ Subjecte: Re: libpthread patch ] > On Fri, 18 Apr 2003, David Xu wrote: > > > > > ----- Original Message ----- > > From: "Daniel Eischen" > > To: "David Xu" > > Cc: > > Sent: Friday, April 18, 2003 2:12 PM > > Subject: Re: libpthread patch > > > > > > > On Fri, 18 Apr 2003, David Xu wrote: > > > > > > > > > > > ----- Original Message ----- > > > > From: "Juli Mallett" > > > > To: "David Xu" > > > > Cc: "Daniel Eischen" ; > > > > Sent: Friday, April 18, 2003 2:04 PM > > > > Subject: Re: libpthread patch > > > > > > > > > > > > > * De: David Xu [ Data: 2003-04-18 ] > > > > > [ Subjecte: Re: libpthread patch ] > > > > > > > There are a few issues we've got to go over, as well as > > > > > > > looking closely at any locking order problems. > > > > > > > > > > > > > I have ever tried to port some kernel code to userland (e.g > > > > > > mutex and witness), but now they were left there for > > > > > > several days without be touched. > > > > > > > > > > This seems like overkill, in fact, it is by definition. If you > > > > > want some primitive private-locks-only mutex tracing/auditing, > > > > > I've done a bit of that and could give you some hints. Using the > > > > > casuptr facility introduced by thr may be a good idea, no? It > > > > > is known to work, and is relatively un-complex? Or am I missing > > > > > something? > > > > > > > > I want to use code to detect LOR not just human eyes, I can accept > > > > any reasonable method. > > > > > > We can do that now with the locks that I have in place. > > > Each consumer of a lock has a "lock user". Threads and > > > KSEs have an array of 3 lock users; probably 2 is enough > > > because I don't think we need more than a nesting of 2. > > > When you decrement the lock user index when releasing > > > a lock, you make sure that the lock being released > > > matches the one owned. In fact, I implemented it this > > > way so you couldn't possible have lock order reversals. > > > The locks will not work if you reverse them. > > > > > > > witeness in kernel records locking order, not lock instance. > > for example, at one time, code uses locking order > > mutex1 mutex2, and release them, next time if another > > code locks them in the order mutex2 mutex1, it would detect > > it. > > Ahh, OK. There aren't that many locks used by the library. > We can probably come up with something that does what you > want. TAILQ makes this exceedingly trivial. -- juli mallett. email: jmallett@freebsd.org; aim: bsdflata; efnet: juli; From owner-freebsd-threads@FreeBSD.ORG Fri Apr 18 00:53:01 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 93D4437B401; Fri, 18 Apr 2003 00:53:01 -0700 (PDT) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id D35C843FBD; Fri, 18 Apr 2003 00:53:00 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0088.cvx40-bradley.dialup.earthlink.net ([216.244.42.88] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 196Qg7-00034f-00; Fri, 18 Apr 2003 00:53:00 -0700 Message-ID: <3E9FAE89.4D656F38@mindspring.com> Date: Fri, 18 Apr 2003 00:51:37 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: David Xu References: <005b01c30573$821cf6a0$f001a8c0@davidw2k> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4c4b46d059d7a7576bb03d0c9c27ddf4593caf27dac41a8fd350badd9bab72f9c350badd9bab72f9c cc: Daniel Eischen cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 07:53:01 -0000 David Xu wrote: > From: "Daniel Eischen" > > We can do that now with the locks that I have in place. > > Each consumer of a lock has a "lock user". Threads and > > KSEs have an array of 3 lock users; probably 2 is enough > > because I don't think we need more than a nesting of 2. > > When you decrement the lock user index when releasing > > a lock, you make sure that the lock being released > > matches the one owned. In fact, I implemented it this > > way so you couldn't possible have lock order reversals. > > The locks will not work if you reverse them. > > witeness in kernel records locking order, not lock instance. > for example, at one time, code uses locking order > mutex1 mutex2, and release them, next time if another > code locks them in the order mutex2 mutex1, it would detect > it. You are allowed to recurse on a mutex in the kernel, because there's some depth to code paths that shouldn't be there, but it's easier to allow recursion than it is to correct the code. Because of this, Witness is able to catch mutex2 mutex1, but it won't catch mutex1 mutex2 mutex1, which is technically a reversal, for all intents and purposes, if the code happens to unlock the first mutex1 first (this will be seen as an unlock of the second mutex1, and so won't trigger any alarm). Daniel's stuff works fine (better than witness!), if you disallow recursion. -- Terry From owner-freebsd-threads@FreeBSD.ORG Fri Apr 18 01:02:41 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2D57537B401; Fri, 18 Apr 2003 01:02:41 -0700 (PDT) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 84E8A43F3F; Fri, 18 Apr 2003 01:02:40 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0088.cvx40-bradley.dialup.earthlink.net ([216.244.42.88] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 196QpS-0003ks-00; Fri, 18 Apr 2003 01:02:39 -0700 Message-ID: <3E9FB0D0.57CF9343@mindspring.com> Date: Fri, 18 Apr 2003 01:01:20 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Daniel Eischen References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4c4b46d059d7a7576d2741988809111a6667c3043c0873f7e350badd9bab72f9c350badd9bab72f9c cc: David Xu cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 08:02:41 -0000 Daniel Eischen wrote: > Ahh, OK. There aren't that many locks used by the library. > We can probably come up with something that does what you > want. I don't think it's an issue (IMO -- see other post). However, if you guys are really concerned with this (in the kernel, too, if it comes to that), then add a "parent_lock" parameter to the mutex creation routine, and set it. Then, if you think there is an issue, you can instrument it by simply checking that the lock is held by the same entity all the way to the root, when you acquire a lock. This beats the witness approach all to heck for diagnostics: you can collect/check all the information you want about the locking entities (thread id, process id, program counter at which the lock was asserted, whatever). Also, at some point, this structure would allow you to have both intention mode locking ("SIX" locks), and allow you to lock partial lists instead of complete lists, to reduce overall contention (e.g. divide a list into 8 parts, one lock per part, create another lock parent, assign it as the parent of all 8 locks, and then get a SIX lock on the parent when you are planning on traversing the list, and then rotor through an X lock on all the inferior locks as you traverse each range -- yields a concurrency of 8, with 1/8 the stall length for write locks). -- Terry From owner-freebsd-threads@FreeBSD.ORG Fri Apr 18 03:49:59 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BA85A37B401; Fri, 18 Apr 2003 03:49:59 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id DC94343F85; Fri, 18 Apr 2003 03:49:58 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.12.9/8.12.9) with SMTP id h3IAoDua039356; Fri, 18 Apr 2003 06:50:14 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Fri, 18 Apr 2003 06:50:12 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Terry Lambert In-Reply-To: <3E9FAE89.4D656F38@mindspring.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org cc: David Xu cc: Daniel Eischen Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 10:50:00 -0000 On Fri, 18 Apr 2003, Terry Lambert wrote: > You are allowed to recurse on a mutex in the kernel, because there's > some depth to code paths that shouldn't be there, but it's easier to > allow recursion than it is to correct the code. By default, mutexes are not permitted to be recursed, and recursing them will cause a panic. Recursion is only permitted if the MTX_RECURSE flag is set at mutex initialization time, which is strongly discouraged. There are several mutex in the kernel where the flag is set, but I hope to see the list remain small (and go down). A quick grep of of kern shows four classes of mutexes with MTX_RECURSE set (one is Giant), and 57 without (approximately). The IP stack locking, on the other hand, has four mutex classes, of which two permit recursion (PCB locks). Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories From owner-freebsd-threads@FreeBSD.ORG Fri Apr 18 10:53:10 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 805B637B405; Fri, 18 Apr 2003 10:53:10 -0700 (PDT) Received: from stork.mail.pas.earthlink.net (stork.mail.pas.earthlink.net [207.217.120.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id DA90143FB1; Fri, 18 Apr 2003 10:53:09 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0577.cvx22-bradley.dialup.earthlink.net ([209.179.200.67] helo=mindspring.com) by stork.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 196a2u-0001jw-00; Fri, 18 Apr 2003 10:53:09 -0700 Message-ID: <3EA03B31.A6D1D83E@mindspring.com> Date: Fri, 18 Apr 2003 10:51:45 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Robert Watson References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4c525a484888e1861d438fdefa02e04ff350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: freebsd-threads@freebsd.org cc: David Xu cc: Daniel Eischen Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 17:53:10 -0000 Robert Watson wrote: > On Fri, 18 Apr 2003, Terry Lambert wrote: > > You are allowed to recurse on a mutex in the kernel, because there's > > some depth to code paths that shouldn't be there, but it's easier to > > allow recursion than it is to correct the code. > > By default, mutexes are not permitted to be recursed, and recursing them > will cause a panic. Recursion is only permitted if the MTX_RECURSE flag > is set at mutex initialization time, which is strongly discouraged. I should have said that "the potential is there, and it's used". The best way to discourage recursion, of course, is to remove the capability. I wish you had commented on the issue of introducing hierarchy into lock acquisition; witness effectively enforces the first hierarchy that occurs, thereafter, but only on a per locking entity basis, and not in the circumstances I described, e.g.: acquire: mutex1(a) mutex2 mutex1(b), release mutex1(a). -- Terry From owner-freebsd-threads@FreeBSD.ORG Fri Apr 18 15:45:25 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 87F6B37B401; Fri, 18 Apr 2003 15:45:25 -0700 (PDT) Received: from snark.ratmir.ru (snark.ratmir.ru [213.24.248.177]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6B80A43F85; Fri, 18 Apr 2003 15:45:24 -0700 (PDT) (envelope-from freebsd@snark.ratmir.ru) Received: from snark.ratmir.ru (freebsd@localhost [127.0.0.1]) by snark.ratmir.ru (8.12.9/8.12.9) with ESMTP id h3IMjNC2063383; Sat, 19 Apr 2003 02:45:23 +0400 (MSD) (envelope-from freebsd@snark.ratmir.ru) Received: (from freebsd@localhost) by snark.ratmir.ru (8.12.9/8.12.9/Submit) id h3IMjN42063382; Sat, 19 Apr 2003 02:45:23 +0400 (MSD) Date: Sat, 19 Apr 2003 02:45:22 +0400 From: Alex Semenyaka To: freebsd-threads@freebsd.org, freebsd-hackers@freebsd.org Message-ID: <20030418224522.GA63339@snark.ratmir.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.4i cc: David Xu cc: Daniel Eischen Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 22:45:25 -0000 On Wed, Apr 16, 2003 at 01:09:46PM -0400, Daniel Eischen wrote: > Found it! The library's version of sigaction was not allowing > a signal handler for SIGCHLD to be installed; it was leftover > from libc_r. By the way, in this connection could somebody check my follow-up to the PR bin/48856? I suggested a really small fix (3 lines in total) to the current implementation of the signal(). With the current one the call of signal(SIGCHLD, SIG_IGN) still allows zombie to be born. There are to places to fix actually, one is the standard libc and second is the pthreads (that's why I am sending this message to -threads as well). If that fix is bad somehow I would gladly receive any explanations. Thanks! SY, Alex From owner-freebsd-threads@FreeBSD.ORG Fri Apr 18 15:53:56 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 17CEB37B401; Fri, 18 Apr 2003 15:53:56 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5D02C43FA3; Fri, 18 Apr 2003 15:53:55 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3IMrsBg005358; Fri, 18 Apr 2003 18:53:54 -0400 (EDT) Received: from localhost (eischen@localhost)h3IMrsnO005354; Fri, 18 Apr 2003 18:53:54 -0400 (EDT) Date: Fri, 18 Apr 2003 18:53:54 -0400 (EDT) From: Daniel Eischen To: Alex Semenyaka In-Reply-To: <20030418224522.GA63339@snark.ratmir.ru> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-hackers@freebsd.org cc: David Xu cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 22:53:57 -0000 On Sat, 19 Apr 2003, Alex Semenyaka wrote: > On Wed, Apr 16, 2003 at 01:09:46PM -0400, Daniel Eischen wrote: > > Found it! The library's version of sigaction was not allowing > > a signal handler for SIGCHLD to be installed; it was leftover > > from libc_r. > > By the way, in this connection could somebody check my follow-up to > the PR bin/48856? I suggested a really small fix (3 lines in total) > to the current implementation of the signal(). With the current one > the call of signal(SIGCHLD, SIG_IGN) still allows zombie to be born. > There are to places to fix actually, one is the standard libc and > second is the pthreads (that's why I am sending this message to > -threads as well). If that fix is bad somehow I would gladly receive > any explanations. Thanks! I don't think you can always set SA_NOCLDWAIT because it could screw up wait() in programs that don't set SA_NOCLDWAIT. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Fri Apr 18 16:08:23 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E91C537B401; Fri, 18 Apr 2003 16:08:23 -0700 (PDT) Received: from snark.ratmir.ru (snark.ratmir.ru [213.24.248.177]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9CA0C43FE5; Fri, 18 Apr 2003 16:08:22 -0700 (PDT) (envelope-from alexs@snark.ratmir.ru) Received: from snark.ratmir.ru (alexs@localhost [127.0.0.1]) by snark.ratmir.ru (8.12.9/8.12.9) with ESMTP id h3IN8KC2063680; Sat, 19 Apr 2003 03:08:21 +0400 (MSD) (envelope-from alexs@snark.ratmir.ru) Received: (from alexs@localhost) by snark.ratmir.ru (8.12.9/8.12.9/Submit) id h3IN8J4M063675; Sat, 19 Apr 2003 03:08:19 +0400 (MSD) Date: Sat, 19 Apr 2003 03:08:19 +0400 From: Alex Semenyaka To: Daniel Eischen Message-ID: <20030418230818.GE3693@snark.ratmir.ru> References: <20030418224522.GA63339@snark.ratmir.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.4i cc: freebsd-hackers@freebsd.org cc: Alex Semenyaka cc: David Xu cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 23:08:24 -0000 On Fri, Apr 18, 2003 at 06:53:54PM -0400, Daniel Eischen wrote: > I don't think you can always set SA_NOCLDWAIT because it could > screw up wait() in programs that don't set SA_NOCLDWAIT. Sure, and I wrote it in that PR: zombie is the information for the wait, no zombie - no infromation. But are there programs which call signal(SIGCHLD, SIG_IGN) and then trying to wait()? I supposed that there are no such: suggested behaviour of signal() is the standard for a lot of systems. Yes, I know that most of them are SysV. But do we have such non-portable BSD-only products trying to use signal(SIGCHLD, SIG_IGN)/wait()? I may be wrong of course, but would anybody points me out the product that will be broken after such changes? Thanks in advance! SY, Alex From owner-freebsd-threads@FreeBSD.ORG Fri Apr 18 16:34:16 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E9B7637B401; Fri, 18 Apr 2003 16:34:16 -0700 (PDT) Received: from snark.ratmir.ru (snark.ratmir.ru [213.24.248.177]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9E5B943FE0; Fri, 18 Apr 2003 16:34:15 -0700 (PDT) (envelope-from alexs@snark.ratmir.ru) Received: from snark.ratmir.ru (alexs@localhost [127.0.0.1]) by snark.ratmir.ru (8.12.9/8.12.9) with ESMTP id h3INYEC2063991; Sat, 19 Apr 2003 03:34:14 +0400 (MSD) (envelope-from alexs@snark.ratmir.ru) Received: (from alexs@localhost) by snark.ratmir.ru (8.12.9/8.12.9/Submit) id h3INYEkR063990; Sat, 19 Apr 2003 03:34:14 +0400 (MSD) Date: Sat, 19 Apr 2003 03:34:14 +0400 From: Alex Semenyaka To: Daniel Eischen Message-ID: <20030418233414.GF3693@snark.ratmir.ru> References: <20030418224522.GA63339@snark.ratmir.ru> <20030418230818.GE3693@snark.ratmir.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030418230818.GE3693@snark.ratmir.ru> User-Agent: Mutt/1.5.4i cc: freebsd-hackers@freebsd.org cc: Alex Semenyaka cc: David Xu cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 23:34:17 -0000 On Sat, Apr 19, 2003 at 03:08:19AM +0400, Alex Semenyaka wrote: > But are there programs which call signal(SIGCHLD, SIG_IGN) and then trying > to wait()? I supposed that there are no such: suggested behaviour of signal() By the way... I just thought that it might be reasonable to allow user to choose that behaviour on the fly (like he can do it for the malloc(3)). Do anyone have the objections against this way? If there are no ones, I can do the patch that allows to signal() OPTIONALLY set SA_NOCLDWAIT flag if the environment variable (say, NOCHLD_NOWAIT) is set. Thanks again :) SY, Alex From owner-freebsd-threads@FreeBSD.ORG Fri Apr 18 17:49:13 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6BC8F37B401; Fri, 18 Apr 2003 17:49:13 -0700 (PDT) Received: from snark.ratmir.ru (snark.ratmir.ru [213.24.248.177]) by mx1.FreeBSD.org (Postfix) with ESMTP id EB0DA43FB1; Fri, 18 Apr 2003 17:49:11 -0700 (PDT) (envelope-from alexs@snark.ratmir.ru) Received: from snark.ratmir.ru (alexs@localhost [127.0.0.1]) by snark.ratmir.ru (8.12.9/8.12.9) with ESMTP id h3J0nAC2067873; Sat, 19 Apr 2003 04:49:10 +0400 (MSD) (envelope-from alexs@snark.ratmir.ru) Received: (from alexs@localhost) by snark.ratmir.ru (8.12.9/8.12.9/Submit) id h3J0nA0G067872; Sat, 19 Apr 2003 04:49:10 +0400 (MSD) Date: Sat, 19 Apr 2003 04:49:10 +0400 From: Alex Semenyaka To: Daniel Eischen Message-ID: <20030419004910.GG3693@snark.ratmir.ru> References: <20030418224522.GA63339@snark.ratmir.ru> <20030418230818.GE3693@snark.ratmir.ru> <20030418233414.GF3693@snark.ratmir.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030418233414.GF3693@snark.ratmir.ru> User-Agent: Mutt/1.5.4i cc: freebsd-hackers@freebsd.org cc: Alex Semenyaka cc: David Xu cc: freebsd-threads@freebsd.org Subject: Re: libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Apr 2003 00:49:13 -0000 On Sat, Apr 19, 2003 at 03:34:14AM +0400, Alex Semenyaka wrote: > By the way... I just thought that it might be reasonable to allow user to > choose that behaviour on the fly (like he can do it for the malloc(3)). Sorry for such amount of messages. This is the last one in this bunch. I found myself software that ignores SIGCHLD with signal(3) but wants then to wait() for a child. So, I must say: this behaviour might be done as an option, if user wants it, and sould be prohibited by default. So do anybody have objections against the patch which will allow such optional behaviour if some envvariable is set? > Thanks again :) And again, and sorry :) SY, Alex From owner-freebsd-threads@FreeBSD.ORG Sat Apr 19 18:14:36 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 31B9837B401; Sat, 19 Apr 2003 18:14:36 -0700 (PDT) Received: from snark.ratmir.ru (snark.ratmir.ru [213.24.248.177]) by mx1.FreeBSD.org (Postfix) with ESMTP id AC35143FE1; Sat, 19 Apr 2003 18:14:32 -0700 (PDT) (envelope-from freebsd@snark.ratmir.ru) Received: from snark.ratmir.ru (freebsd@localhost [127.0.0.1]) by snark.ratmir.ru (8.12.9/8.12.9) with ESMTP id h3K1ERC2052447; Sun, 20 Apr 2003 05:14:27 +0400 (MSD) (envelope-from freebsd@snark.ratmir.ru) Received: (from freebsd@localhost) by snark.ratmir.ru (8.12.9/8.12.9/Submit) id h3K1EOrL052446; Sun, 20 Apr 2003 05:14:25 +0400 (MSD) Date: Sun, 20 Apr 2003 05:14:24 +0400 From: Alex Semenyaka To: threads@freebsd.org, freebsd-hackers@freebsd.org Message-ID: <20030420011424.GA52428@snark.ratmir.ru> References: <20030408164614.GA7236@comp.chem.msu.su> <20030409044301.J628@gamplex.bde.org> <20030408194944.GA43822@nagual.pp.ru> <20030409112047.GB33265@snark.ratmir.ru> <3E940993.FBAFFD70@mindspring.com> <20030409123431.GD33144@snark.ratmir.ru> <3E94AB82.496681D5@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3E94AB82.496681D5@mindspring.com> User-Agent: Mutt/1.5.4i cc: "Andrey A. Chernov" cc: Yar Tikhiy Subject: Re: termios & non-blocking I/O X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2003 01:14:36 -0000 Hi! On Wed, Apr 09, 2003 at 04:23:46PM -0700, Terry Lambert wrote: >> when MIN>0 and TIME=0. Don't you think the behaviour in the (TIME=0, MIN>0) >> and (TIME>0, MIN=0) should be consistent somehow? > Consistent with what? With each other, see: > The cases are: > > MIN TIME > 1) >0 >0 Interbyte timer; after 1 byte received, timer > is started; if MIN bytes received before timer > is statisfied, return. If another byte is > received, timer is reset; if timer expires, > read is satisfied. > 2) >0 =0 Pending read is not satisfied until MIN bytes > received. > 3) =0 >0 NOT interbyte timer; read is satisfied on a > single byte received or timer expired. > 4) =0 =0 Polling read; available bytes up to number of > requested bytes is returned, and read is always > satisfied. The cases 2) and 3) should be consistent when there is no input data. Now thay are not. > unacceptable. I think it is impossible to implement case #1 correctly > with NBIO enabled, and would convert it to case #4, and implement the > TIME in user space in libc_r. Probably you are right, but that was not the subject of the discussion. > Case #2 is still not correctly handled at all by your proposed change; > libc_r work is required to avoid stalling until input is present; the > correct return on NBIO is pretty obviously "EAGAIN,-1"; HOWEVER, you > may have received *some* input data, but not MIN worth of input data; > returning some number < MIN violates the interface contract; so does > losing the data. That's right as well, of course. Let me remind you that we are discussing the case when there is no input data at all. > Case #3 is a full read timer for one byte; also not correctly handled > in the face of the proposed change. For a TIME=100, we are talking > about stalling all threads for 100 seconds; the correct return is You just left the rest empty, unfortunatelly. But the WHOLE SUBJECT of the discussion is HERE, and my idea is that the correct return is -1/EAGAIN if there is no data. > Case #4 is polling I/O; it's trivial to implement in libc_r. Sure. > Rule of thumb: If it adds latency, which is always annoying; if > it can add an arbitray number of seconds of latency, it's > unacceptable. I never, never, NEVER suggest to add the latency or to block the process in the kernel or something like this. I understand perfectly why is not acceptable. In several letters I wrote my point. Here it is: in the case TIME>0, MIN=0 we in non-blocking mode in the absence of data we should return immidiately, without any blocking (as now, just perfect), but with -1/EAGAIN instead of 0 (as we do now). Because of that 0 we cannot distinguish the legal EOF and just "no data yet" case behind the libc_r. That's it. >> No, sorry, you have missed my point. Sure I do NOT propose to block >> the input when TIME>0. I propose to return -1/EAGAIN (as in the >> MIN>0 case) instead of 0 (as it is done now). That is what I called >> "the second way". And in this case both libc_r and program can >> easely distinct between EOF and nothing-to-read situation and handle >> them properly. Also all other threads will not be stucked. > The problem with this is case #2 with some number of characters less > than MIN. For example, let us propose a MIN of 10, with 3 characters Well it is _another_ problem. The originator had no questions about incomptele block of data for the MIN>0 and waiting data case. It was ok with him. He told about the case MIN=0, TIME>0, no data, non-clocking mode. It is incorrect to return 0 in this case, told he. And I agree with him. ##1 TIME>0 I argue that this is impossible to implement > in the presence of NBIO, since you can not > return both the value of the short read, and > the fact that the timer would result in a > block, simultaneously. Best approach is to > cause an error, EAGAIN,-1. Right! You see? Exactly our point! But NOW it returns just 0. Should be fixed, I think, shouldn't it? :) SY, Alex