From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 21 16:27:19 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6BA93E6A for ; Fri, 21 Mar 2014 16:27:19 +0000 (UTC) Received: from smtp.fagskolen.gjovik.no (smtp.fagskolen.gjovik.no [IPv6:2001:700:1100:1:200:ff:fe00:b]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CD771FC4 for ; Fri, 21 Mar 2014 16:27:18 +0000 (UTC) Received: from mail.fig.ol.no (localhost [127.0.0.1]) by mail.fig.ol.no (8.14.7/8.14.7) with ESMTP id s2LGRDuB018623 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 21 Mar 2014 17:27:13 +0100 (CET) (envelope-from trond@fagskolen.gjovik.no) Received: from localhost (trond@localhost) by mail.fig.ol.no (8.14.7/8.14.7/Submit) with ESMTP id s2LGRDAm018620; Fri, 21 Mar 2014 17:27:13 +0100 (CET) (envelope-from trond@fagskolen.gjovik.no) X-Authentication-Warning: mail.fig.ol.no: trond owned process doing -bs Date: Fri, 21 Mar 2014 17:27:13 +0100 (CET) From: =?ISO-8859-1?Q?Trond_Endrest=F8l?= Sender: Trond.Endrestol@fagskolen.gjovik.no To: Dmitry Sivachenko Subject: Re: madvise() vs posix_fadvise() In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) Organization: Fagskolen Innlandet OpenPGP: url=http://fig.ol.no/~trond/trond.key MIME-Version: 1.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mail.fig.ol.no Content-Type: TEXT/PLAIN; charset=ISO-8859-1 Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 16:27:19 -0000 On Fri, 21 Mar 2014 18:56+0400, Dmitry Sivachenko wrote: > Hello! > > I have a program which uses large data files (read-only, via mmap()). > > These machines have a bit more RAM that these files occupy, so it is > possible to have all these data in memory. > > What techniques should I use to promote this data not to be purged > from RAM: > > -- madvise(MADV_WILLNEED) > -- posix_fadvise(POSIX_FADV_WILLNEED) > -- both? Although a bit dangerous, mlock(2) might be your ticket. That system call prevents your memory region from being swapped/paged away from physical memory. > Some parts of this mmap()ed data is frequently accessed, some is > not, and I see that rarely accessed regions are purged from RAM. I > want disk cache (program writes large log files) to have lower > priority, so disk cache does not purge mmaped regions from memory. > > Thanks. -- +-------------------------------+------------------------------------+ | Vennlig hilsen, | Best regards, | | Trond Endrestøl, | Trond Endrestøl, | | IT-ansvarlig, | System administrator, | | Fagskolen Innlandet, | Gjøvik Technical College, Norway, | | tlf. mob. 952 62 567, | Cellular...: +47 952 62 567, | | sentralbord 61 14 54 00. | Switchboard: +47 61 14 54 00. | +-------------------------------+------------------------------------+ From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 22 22:48:01 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 81CDF899; Sat, 22 Mar 2014 22:48:01 +0000 (UTC) Received: from mail-oa0-x236.google.com (mail-oa0-x236.google.com [IPv6:2607:f8b0:4003:c02::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2B53DF10; Sat, 22 Mar 2014 22:48:01 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n16so4264812oag.27 for ; Sat, 22 Mar 2014 15:48:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=wpwiPtviH485jBxxbg6mb9qFEd2CIsFnVvcW6THRZWM=; b=f92Uq78d1Wu9ySJ4u7xoIOhdtJ2tqYBZ15CqkzQ0lW+O5RlmaAUS9a6Aw7f2UkRTSz s9TlMROnmHn/j6u46UXdUUwy2MC2fZKS/uiwVI7wrtVMN5BgykK5SHwxxlDmSzmP8kQb 0XYFfDk1BIm36b+FRTIx9aa25dMDi+eZUVOGGpH9sSaJOY5XvB5n0sGz2Pz5jf0Vpwpx 08FKzsa2wnD3zUEloYN/fZfMP3YniNYcRlOSX+TnOeVxLaqi+PDsilv+HMSa8XH8sfrX aVbYvF7NiSO+TfBbs9rSbIOviIKugRQAoXOj21wFmudYX0sYQybQ2tNPiAFj0QyuuGae Jy7Q== MIME-Version: 1.0 X-Received: by 10.60.172.70 with SMTP id ba6mr49287713oec.17.1395528480555; Sat, 22 Mar 2014 15:48:00 -0700 (PDT) Received: by 10.182.130.71 with HTTP; Sat, 22 Mar 2014 15:48:00 -0700 (PDT) In-Reply-To: <532BEC4C.6080201@FreeBSD.org> References: <53269B0E.5020904@FreeBSD.org> <532BEC4C.6080201@FreeBSD.org> Date: Sat, 22 Mar 2014 18:48:00 -0400 Message-ID: Subject: Re: Definition struct and int From: Joe Nosay To: Kevin Lo Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Hackers , Adrian Chadd X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 22:48:01 -0000 On Fri, Mar 21, 2014 at 3:37 AM, Kevin Lo wrote: > On 2014/03/18 08:05, Joe Nosay wrote: > >> On Mon, Mar 17, 2014 at 2:49 AM, Kevin Lo wrote: >> >> On 2014/03/17 14:07, Joe Nosay wrote: >>> >>> On Sun, Mar 16, 2014 at 1:07 AM, Joe Nosay >>>> wrote: >>>> >>>> >>>> On Fri, Mar 14, 2014 at 5:43 PM, Joe Nosay >>>>> wrote: >>>>> >>>>> >>>>> On Thu, Mar 13, 2014 at 9:26 PM, Adrian Chadd >>>>>> wrote: >>>>>> >>>>>> Is this -HEAD, or? >>>>>> >>>>>>> >>>>>>> -a >>>>>>> >>>>>>> >>>>>>> On 13 March 2014 18:13, Joe Nosay wrote: >>>>>>> >>>>>>> Testing of a patch for using UDP Lite on FreeBSD caused no >>>>>>>> compilation >>>>>>>> errors; however, after adding the options of "VIMAGE" and "MROUTING" >>>>>>>> to >>>>>>>> conf/kern?GENERIC, make buildkernel stops with: >>>>>>>> /usr/src/sys/netinet/udp_usrreq.c:1701:18: error: too few arguments >>>>>>>> to >>>>>>>> function >>>>>>>> call, expected 2, have 1 >>>>>>>> udp_discardcb(up); >>>>>>>> ~~~~~~~~~~~~~ ^ >>>>>>>> /usr/src/sys/netinet/udp_usrreq.c:274:1: note: 'udp_discardcb' >>>>>>>> >>>>>>>> declared here >>>>>>> >>>>>>> void >>>>>>>> ^ >>>>>>>> 1 error generated. >>>>>>>> *** Error code 1 >>>>>>>> >>>>>>>> The file in question of >>>>>>>> /usr/src/sys/netinet/udp_usrreq.c >>>>>>>> >>>>>>>> has the value of >>>>>>>> udp_discardcb(struct udpcb *up, int isudp) >>>>>>>> >>>>>>>> that is causing the problem. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I believe that the compiler is looking for a value to int isudp but >>>>>>>> >>>>>>>> that >>>>>>> >>>>>>> value does not exist. >>>>>>>> _______________________________________________ >>>>>>>> freebsd-hackers@freebsd.org mailing list >>>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>>>>>> To unsubscribe, send any mail to " >>>>>>>> >>>>>>>> freebsd-hackers-unsubscribe@freebsd.org" >>>>>>> >>>>>>> >>>>>>> No, it is 10.0 RELEASE #0 r262601 >>>>>> >>>>>> Last time, I had entered too many files. Here are the relative files. >>>>>> >>>>>> >>>>>> There was no problem in compiling as listed earlier. I have not >>>>>> studied >>>>>> C enough to solve this problem; however, I can see that int isudp >>>>>> happens >>>>>> once while the next closest account is int isudplite. >>>>>> >>>>>> >>>>>> I've just upgraded source to head. I have three patches for UDP >>>>>> Lite. >>>>>> >>>>> The >>>>> question is which one(s) should I use. >>>>> >>>>> The udp-v.diff only has a reference to udp_discardcb up, while patch >>>>> udplite and udplite.diff have the struct and int references. >>>>> >>>>> >>>>> Could someone possibly point me towards some online documentation >>>>> that >>>>> >>>> would allow me to learn of a solution to this problem? This would be >>>> much >>>> better because I would be solving and learning. >>>> >>>> Included are the three patches mentioned earlier. >>>> >>>> Yes, I will be looking at them again. >>>> >>>> Apologies for any noise and if I am coming off the wrong way. >>>> >>>> Hi Joe, >>> >>> As I mentioned, my udp-lite's diff no longer applies cleanly against >>> -HEAD. I've been busy lately and haven't got time to reply to your >>> messages. >>> Give me another day or two to cook up a revised patch for you to test. >>> Thanks! >>> >>> Kevin >>> >>> >> Alright. >> > > The revised patch is available at: > http://people.freebsd.org/~kevlo/udplite.diff > > Kevin > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > I have VPS, VIMAGE, and MROUTING enabled on GENERIC. I'll tell you how it goes for me.