Date: Fri, 21 Mar 2014 17:27:13 +0100 (CET) From: =?ISO-8859-1?Q?Trond_Endrest=F8l?= <Trond.Endrestol@fagskolen.gjovik.no> To: Dmitry Sivachenko <trtrmitya@gmail.com> Cc: hackers@freebsd.org Subject: Re: madvise() vs posix_fadvise() Message-ID: <alpine.BSF.2.00.1403211725140.56113@mail.fig.ol.no> In-Reply-To: <D6BD48AF-9522-495D-8D54-37854E53C272@gmail.com> References: <D6BD48AF-9522-495D-8D54-37854E53C272@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 21 Mar 2014 18:56+0400, Dmitry Sivachenko wrote: > Hello! > > I have a program which uses large data files (read-only, via mmap()). > > These machines have a bit more RAM that these files occupy, so it is > possible to have all these data in memory. > > What techniques should I use to promote this data not to be purged > from RAM: > > -- madvise(MADV_WILLNEED) > -- posix_fadvise(POSIX_FADV_WILLNEED) > -- both? Although a bit dangerous, mlock(2) might be your ticket. That system call prevents your memory region from being swapped/paged away from physical memory. > Some parts of this mmap()ed data is frequently accessed, some is > not, and I see that rarely accessed regions are purged from RAM. I > want disk cache (program writes large log files) to have lower > priority, so disk cache does not purge mmaped regions from memory. > > Thanks. -- +-------------------------------+------------------------------------+ | Vennlig hilsen, | Best regards, | | Trond Endrestøl, | Trond Endrestøl, | | IT-ansvarlig, | System administrator, | | Fagskolen Innlandet, | Gjøvik Technical College, Norway, | | tlf. mob. 952 62 567, | Cellular...: +47 952 62 567, | | sentralbord 61 14 54 00. | Switchboard: +47 61 14 54 00. | +-------------------------------+------------------------------------+ From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 22 22:48:01 2014 Return-Path: <owner-freebsd-hackers@FreeBSD.ORG> Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 81CDF899; Sat, 22 Mar 2014 22:48:01 +0000 (UTC) Received: from mail-oa0-x236.google.com (mail-oa0-x236.google.com [IPv6:2607:f8b0:4003:c02::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2B53DF10; Sat, 22 Mar 2014 22:48:01 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n16so4264812oag.27 for <multiple recipients>; Sat, 22 Mar 2014 15:48:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=wpwiPtviH485jBxxbg6mb9qFEd2CIsFnVvcW6THRZWM=; b=f92Uq78d1Wu9ySJ4u7xoIOhdtJ2tqYBZ15CqkzQ0lW+O5RlmaAUS9a6Aw7f2UkRTSz s9TlMROnmHn/j6u46UXdUUwy2MC2fZKS/uiwVI7wrtVMN5BgykK5SHwxxlDmSzmP8kQb 0XYFfDk1BIm36b+FRTIx9aa25dMDi+eZUVOGGpH9sSaJOY5XvB5n0sGz2Pz5jf0Vpwpx 08FKzsa2wnD3zUEloYN/fZfMP3YniNYcRlOSX+TnOeVxLaqi+PDsilv+HMSa8XH8sfrX aVbYvF7NiSO+TfBbs9rSbIOviIKugRQAoXOj21wFmudYX0sYQybQ2tNPiAFj0QyuuGae Jy7Q== MIME-Version: 1.0 X-Received: by 10.60.172.70 with SMTP id ba6mr49287713oec.17.1395528480555; Sat, 22 Mar 2014 15:48:00 -0700 (PDT) Received: by 10.182.130.71 with HTTP; Sat, 22 Mar 2014 15:48:00 -0700 (PDT) In-Reply-To: <532BEC4C.6080201@FreeBSD.org> References: <CA+WntOtT_SfQ4KRLyaWSQ8g-kkq3P7-YkJR7siqUZYAb-WtYTQ@mail.gmail.com> <CAJ-Vmok5RTi8SSm3PMXSYDpWonO3O+LetvH8ZaPzb66OD61S1A@mail.gmail.com> <CA+WntOufKY=7ikh2rNfXbxSKHW=WWKiW9iNXJOwZ+ar5GzXa9A@mail.gmail.com> <CA+WntOtsBu=1n134r6Te2kVQ4B=o+BzmTcKodxHdGEOwSChi9w@mail.gmail.com> <CA+WntOvLHYAxp5_s+CrWnQSy5H=G_p07WMRZn-=77Dx2kecr0g@mail.gmail.com> <53269B0E.5020904@FreeBSD.org> <CA+WntOtiKCJB9ggY48RwYyYFqtL9gAqyG+vT6tjfPVCTdJ375Q@mail.gmail.com> <532BEC4C.6080201@FreeBSD.org> Date: Sat, 22 Mar 2014 18:48:00 -0400 Message-ID: <CA+WntOsUcqws7pLXBN5eHKHmnGMyQcR8E-8RhA4f+OtMHApn+w@mail.gmail.com> Subject: Re: Definition struct and int From: Joe Nosay <superbisquit@gmail.com> To: Kevin Lo <kevlo@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org>, Adrian Chadd <adrian@freebsd.org> X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD <freebsd-hackers.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-hackers>, <mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>; List-Post: <mailto:freebsd-hackers@freebsd.org> List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>, <mailto:freebsd-hackers-request@freebsd.org?subject=subscribe> X-List-Received-Date: Sat, 22 Mar 2014 22:48:01 -0000 On Fri, Mar 21, 2014 at 3:37 AM, Kevin Lo <kevlo@freebsd.org> wrote: > On 2014/03/18 08:05, Joe Nosay wrote: > >> On Mon, Mar 17, 2014 at 2:49 AM, Kevin Lo <kevlo@freebsd.org> wrote: >> >> On 2014/03/17 14:07, Joe Nosay wrote: >>> >>> On Sun, Mar 16, 2014 at 1:07 AM, Joe Nosay <superbisquit@gmail.com> >>>> wrote: >>>> >>>> >>>> On Fri, Mar 14, 2014 at 5:43 PM, Joe Nosay <superbisquit@gmail.com> >>>>> wrote: >>>>> >>>>> >>>>> On Thu, Mar 13, 2014 at 9:26 PM, Adrian Chadd <adrian@freebsd.org> >>>>>> wrote: >>>>>> >>>>>> Is this -HEAD, or? >>>>>> >>>>>>> >>>>>>> -a >>>>>>> >>>>>>> >>>>>>> On 13 March 2014 18:13, Joe Nosay <superbisquit@gmail.com> wrote: >>>>>>> >>>>>>> Testing of a patch for using UDP Lite on FreeBSD caused no >>>>>>>> compilation >>>>>>>> errors; however, after adding the options of "VIMAGE" and "MROUTING" >>>>>>>> to >>>>>>>> conf/kern?GENERIC, make buildkernel stops with: >>>>>>>> /usr/src/sys/netinet/udp_usrreq.c:1701:18: error: too few arguments >>>>>>>> to >>>>>>>> function >>>>>>>> call, expected 2, have 1 >>>>>>>> udp_discardcb(up); >>>>>>>> ~~~~~~~~~~~~~ ^ >>>>>>>> /usr/src/sys/netinet/udp_usrreq.c:274:1: note: 'udp_discardcb' >>>>>>>> >>>>>>>> declared here >>>>>>> >>>>>>> void >>>>>>>> ^ >>>>>>>> 1 error generated. >>>>>>>> *** Error code 1 >>>>>>>> >>>>>>>> The file in question of >>>>>>>> /usr/src/sys/netinet/udp_usrreq.c >>>>>>>> >>>>>>>> has the value of >>>>>>>> udp_discardcb(struct udpcb *up, int isudp) >>>>>>>> >>>>>>>> that is causing the problem. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I believe that the compiler is looking for a value to int isudp but >>>>>>>> >>>>>>>> that >>>>>>> >>>>>>> value does not exist. >>>>>>>> _______________________________________________ >>>>>>>> freebsd-hackers@freebsd.org mailing list >>>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>>>>>> To unsubscribe, send any mail to " >>>>>>>> >>>>>>>> freebsd-hackers-unsubscribe@freebsd.org" >>>>>>> >>>>>>> >>>>>>> No, it is 10.0 RELEASE #0 r262601 >>>>>> >>>>>> Last time, I had entered too many files. Here are the relative files. >>>>>> >>>>>> >>>>>> There was no problem in compiling as listed earlier. I have not >>>>>> studied >>>>>> C enough to solve this problem; however, I can see that int isudp >>>>>> happens >>>>>> once while the next closest account is int isudplite. >>>>>> >>>>>> >>>>>> I've just upgraded source to head. I have three patches for UDP >>>>>> Lite. >>>>>> >>>>> The >>>>> question is which one(s) should I use. >>>>> >>>>> The udp-v.diff only has a reference to udp_discardcb up, while patch >>>>> udplite and udplite.diff have the struct and int references. >>>>> >>>>> >>>>> Could someone possibly point me towards some online documentation >>>>> that >>>>> >>>> would allow me to learn of a solution to this problem? This would be >>>> much >>>> better because I would be solving and learning. >>>> >>>> Included are the three patches mentioned earlier. >>>> >>>> Yes, I will be looking at them again. >>>> >>>> Apologies for any noise and if I am coming off the wrong way. >>>> >>>> Hi Joe, >>> >>> As I mentioned, my udp-lite's diff no longer applies cleanly against >>> -HEAD. I've been busy lately and haven't got time to reply to your >>> messages. >>> Give me another day or two to cook up a revised patch for you to test. >>> Thanks! >>> >>> Kevin >>> >>> >> Alright. >> > > The revised patch is available at: > http://people.freebsd.org/~kevlo/udplite.diff > > Kevin > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > I have VPS, VIMAGE, and MROUTING enabled on GENERIC. I'll tell you how it goes for me.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.00.1403211725140.56113>