From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 5 05:51:22 2005 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B5E9316A4CE for ; Sat, 5 Feb 2005 05:51:22 +0000 (GMT) Received: from mail03.syd.optusnet.com.au (mail03.syd.optusnet.com.au [211.29.132.184]) by mx1.FreeBSD.org (Postfix) with ESMTP id D83D343D31 for ; Sat, 5 Feb 2005 05:51:21 +0000 (GMT) (envelope-from PeterJeremy@optushome.com.au) Received: from cirb503493.alcatel.com.au (c211-30-75-229.belrs2.nsw.optusnet.com.au [211.30.75.229]) j155pK12024352 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO) for ; Sat, 5 Feb 2005 16:51:20 +1100 Received: from cirb503493.alcatel.com.au (localhost.alcatel.com.au [127.0.0.1])j155pJ7l054506 for ; Sat, 5 Feb 2005 16:51:19 +1100 (EST) (envelope-from pjeremy@cirb503493.alcatel.com.au) Received: (from pjeremy@localhost) by cirb503493.alcatel.com.au (8.12.10/8.12.9/Submit) id j155pJ2V054505 for freebsd-hackers@freebsd.org; Sat, 5 Feb 2005 16:51:19 +1100 (EST) (envelope-from pjeremy) Date: Sat, 5 Feb 2005 16:51:19 +1100 From: Peter Jeremy To: freebsd-hackers@freebsd.org Message-ID: <20050205055119.GC44250@cirb503493.alcatel.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2i Subject: Sharing memory between ithread and userland X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Feb 2005 05:51:22 -0000 I have a device driver that uses a mmap'd region to share status information between the kernel driver and the userland application and am finding that some updates made by the interrupt handler don't seem to be visible to the application. I can't see anything I am doing wrong and hope that someone here can tell me what I've done wrong. I've tried disassembling both functions in question and as far as I can tell the machine code matches my expectations. The system is a single-CPU Athlon-XP running 5.3-RELEASEp4. Looking at the following code, my application is receiving SIGUSR1 signals with ev1ctr incrementing as expected but status, event1 and ev1sts all report no event has occurred. The signals are occurring at the expected rate and the actual data (which arrives via another mmap'd region) also looks correct (if I ignore event1 and just rely on ev1ctr, everything seems to work correctly). I'd suspect that something else was corrupting memory but is seems odd that the corruption affects non-adjacent words and always sets them to zero. Can anyone offer any suggestions? The shared area looks like: struct dev_status { uint32_t event1; uint32_t event2; uint32_t event3; ... uint32_t ev1ctr; ... uint32_t status; ... uint32_t ev1sts; ... }; My kernel interrupt handler looks like: static void dev_intr(void *arg) { struct dev_softc *sc = (struct dev_softc *)arg; struct dev_status *sts = sc->sc_sts; uint32_t irep, r; struct proc *p; irep = DEVICE_READ(INTERRUPT_REPORT); sts->ssts_irep = irep; sts->event1 = 0; sts->event2 = 0; sts->event3 = 0; if (irep & EVENT3) sts->event3 = 1; if (irep & EVENT2) { sts->event2 = 1; .... } if (irep & EVENT1) { r = DEVICE_READ(EVENT1_STATUS); sts->ev1sts = r; if (r & EVENT1_READY) { sts->event1 = 1; .... sts->ev1ctr++; } } if (sts->event1 || sts->event2 || sts->event3) { p = pfind(sc->sc_sigpid)) != NULL) { psignal(p, SIGUSR1); PROC_UNLOCK(p); } } The mmap handler looks like: dev_mmap(DPTYPE dev, vm_offset_t offset, vm_paddr_t *paddr, int nprot) { struct dev_softc *sc; sc = devclass_get_softc(dev_devclass, minor(dev)); if (offset == MAGIC_OFFSET) { *paddr = vtophys(sc->sc_sts); return 0; } ... } And my userland code looks like: volatile struct dev_status *sts; sts = mmap(NULL, STATUS_SIZE, PROT_READ, MAP_SHARED, device_fd, MAGIC_OFFSET); void handle_sigusr1(int sig) { static uint32_t buf1ctr; if (sts->event1) { while (bufctr < sts->ev1ctr) { ... bufctr++; } } if (sts->event2) { ... } if (sts->event3) { ... } } The following is a typical output from debugging code at the beginning of handle_sigusr1() - though the output is not identical from run to run: buf1ctr ev1ctr event1 status vptr:0x00000000 0x00000001 0x00000000 0x00000000 vptr:0x00000000 0x00000002 0x00000000 0x00000000 vptr:0x00000000 0x00000003 0x00000000 0x00000000 vptr:0x00000000 0x00000004 0x00000000 0x00000000 vptr:0x00000000 0x00000005 0x00000000 0x00000000 vptr:0x00000000 0x00000006 0x00000000 0x00000000 vptr:0x00000000 0x00000007 0x00000000 0x00000000 vptr:0x00000000 0x00000008 0x00000000 0x00000000 vptr:0x00000000 0x00000009 0x00000000 0x00000000 vptr:0x00000000 0x0000000A 0x00000001 0x00000001 vptr:0x0000000B 0x0000000B 0x00000000 0x00000000 vptr:0x0000000B 0x0000000C 0x00000000 0x00000000 vptr:0x0000000B 0x0000000D 0x00000001 0x00000001 vptr:0x0000000D 0x0000000E 0x00000000 0x00000000 vptr:0x0000000D 0x0000000F 0x00000001 0x00000001 vptr:0x0000000F 0x00000010 0x00000001 0x00000001 vptr:0x00000010 0x00000011 0x00000000 0x00000000 vptr:0x00000010 0x00000012 0x00000000 0x00000000 vptr:0x00000010 0x00000013 0x00000000 0x00000000 vptr:0x00000010 0x00000014 0x00000000 0x00000000 vptr:0x00000010 0x00000015 0x00000000 0x00000000 vptr:0x00000010 0x00000016 0x00000001 0x00000001 vptr:0x00000016 0x00000017 0x00000000 0x00000000 vptr:0x00000016 0x00000018 0x00000000 0x00000000 vptr:0x00000016 0x00000019 0x00000001 0x00000001 vptr:0x00000019 0x0000001A 0x00000001 0x00000001 vptr:0x0000001A 0x0000001B 0x00000001 0x00000001 vptr:0x0000001B 0x0000001C 0x00000001 0x00000001 vptr:0x0000001C 0x0000001D 0x00000001 0x00000001 vptr:0x0000001D 0x0000001E 0x00000001 0x00000001 vptr:0x0000001E 0x0000001F 0x00000001 0x00000001 vptr:0x0000001F 0x00000020 0x00000001 0x00000001 vptr:0x00000020 0x00000021 0x00000001 0x00000001 vptr:0x00000021 0x00000022 0x00000001 0x00000001 -- Peter Jeremy