From owner-freebsd-hackers Tue Aug 15 9:30:31 2000 Delivered-To: freebsd-hackers@freebsd.org Received: from acl.lanl.gov (acl.lanl.gov [128.165.147.1]) by hub.freebsd.org (Postfix) with ESMTP id 8E37237B815 for ; Tue, 15 Aug 2000 09:30:26 -0700 (PDT) (envelope-from rminnich@lanl.gov) Received: from mini.acl.lanl.gov (root@mini.acl.lanl.gov [128.165.147.34]) by acl.lanl.gov (8.8.8/8.8.5) with ESMTP id KAA1472935 for ; Tue, 15 Aug 2000 10:30:25 -0600 (MDT) Received: from localhost (rminnich@localhost) by mini.acl.lanl.gov (8.9.3/8.8.8) with ESMTP id KAA24170 for ; Tue, 15 Aug 2000 10:30:25 -0600 X-Authentication-Warning: mini.acl.lanl.gov: rminnich owned process doing -bs Date: Tue, 15 Aug 2000 10:30:25 -0600 (MDT) From: Ronald G Minnich X-Sender: rminnich@mini.acl.lanl.gov To: hackers@FreeBSD.ORG Subject: Re: IPC, shared memory, syncronization AND threads... In-Reply-To: <200008151613.JAA04129@vashon.polstra.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG OK, here's a note from long ago, when this came up before. Dated: Tue Jul 2 10:48:16 1996 The idea is simple: tset is the fastest, but you only want to spin so long. Then you want to drop into the kernel, and wait for someone to wake you up. This thing was quite fast on freebsd, even four years ago. In fact I have yet to see anything faster, but I'm willing to be corrected. ------------------------------ Here's a simple test-and-set function for the 386 (tested and works): int tset(int *i, int lockval, int unlockval) { int j = 0; asm("movl 16(%ebp), %eax"); asm("movl 8(%ebp),%ecx"); asm("movl 12(%ebp),%edx"); asm("cmpxchg %edx, (%ecx)"); asm("jne failed"); asm("movl %eax, -4(%ebp)"); asm("jmp done"); asm("failed: movl %eax, -4(%ebp)"); asm("done:"); return j; } This will run a bit faster than a file lock system call :-). But what about contention? notice that this function, if it fails, fails. so we need a retry strategy. IF you fail, should you spin forever? NO! Do that, and you eat the processor doing nothing. You ought to have a reasonable way to, say, spin one or two more times, then go to a more heavyweight sleep. SO: here's the fastlock library call (#ifdef USEMOD is for LKM ) void fastlock(int *address, int lockval, int unlockval) { int testval; #ifdef USEMOD static int syscallnum = -1; if (syscallnum < 0) syscallnum = syscallfind("fastlock"); if (syscallnum < 0) { perror("fastlock syscallfind"); return; } #endif testval = tset(address, lockval, unlockval); if (testval == unlockval) { #ifdef FASTLOCKDEBUG printf("(%d)fastlock quickout\n", getpid()); #endif return; } /* attempt to lock failed. test then wait in kernel sleep() */ while (1) { /* set the high-order bit. This tells the unlocker to do the system * call and free up the lock. */ (void) tset(address, testval|0x80000000,testval); #ifdef FASTLOCKDEBUG printf("(%d)hang in there\n", getpid()); #endif /* we should be checking here to make sure that high-order bit is * set. But this second tset fails only * in the event of contention, in which case * someone else has set the high-order * bit too ... seems pointless, esp. given that fastlock has a timeout */ syscall(syscallnum, 1, address, unlockval); testval = tset(address, lockval, unlockval); if (testval == unlockval) return; } } So what are we doing? We're doing the tset. If it fails, then we do one more tset, to set the high order bit, then drop into the fastlock system call. Once we return from that, we try to tset the variable again. If that fails, we drop once again into the system call. Here's fastunlock: void fastunlock(int *address, int unlockval) { int dosyscall = 0; static int syscallnum = -1; /* this is really in the file */ #ifdef USEMOD if (syscallnum < 0) syscallnum = syscallfind("fastlock"); if (syscallnum < 0) { perror("fastunlock syscallfind"); return; } #endif if (*address & 0x80000000) dosyscall = 1; *address = unlockval; #ifdef FASTLOCKDEBUG printf("(%d)fastunlock dosyscall is %d\n", getpid(), dosyscall); if (dosyscall) printf("conflict %d\n", getpid()); fflush(stdout); #endif if (dosyscall) syscall(syscallnum, 0, address, unlockval); } Ok, this one tests to see if it needs to wake any sleepers, clears the memory variable, then drops into the kernel if needed (if (dosyscall) ...) Here's the system call. Note several things: 1) the definition of 'unlocked' is passed in to the system call for the final test, not assumed to be zero. 2) The 'address' argument does NOT NEED TO BE AN ADDRESS. it's a number that all the procs have to agree on, that is all. 3) if you accidently awake more than one sleeper, the loop in fastlock handles that properly 4) This system call handles both waking up and sleeping 5) For my measurements, this thing is a whole lot faster than anything else available on (e.g.) freebsd. Questions to me. int fastlock(p, uap, retval) struct proc *p; struct flu *uap; int retval[]; { extern int hz; retval[0] = 0; /* printf("fastlockunlock: com %d address 0x%x unlocked %d\n", uap->com, uap->address, uap->unlocked); */ if (uap->com == 0) /* unlock */ wakeup((void *) uap->address); else { /* last chance */ /* try one last time to see if it is unlocked */ int curval = fuword((void *) uap->address); if (curval == uap->unlocked) return; tsleep((void *) uap->address, PUSER, NULL, 10*hz); } return 0; } -------------------- questions to me, as it says in my old note. ron To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message