From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 11 05:17:07 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5516B7A0 for ; Fri, 11 Oct 2013 05:17:07 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C932F26A8 for ; Fri, 11 Oct 2013 05:17:06 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id r9B5H2uR070478; Fri, 11 Oct 2013 08:17:02 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua r9B5H2uR070478 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id r9B5H2ZY070477; Fri, 11 Oct 2013 08:17:02 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 11 Oct 2013 08:17:02 +0300 From: Konstantin Belousov To: Dmitry Sivachenko Subject: Re: mmap() question Message-ID: <20131011051702.GE41229@kib.kiev.ua> References: <95E0B821-BF9B-4EBF-A1E5-1DDCBB1C3D1B@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="dS6nsELLbkZi5mD/" Content-Disposition: inline In-Reply-To: <95E0B821-BF9B-4EBF-A1E5-1DDCBB1C3D1B@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: "hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Oct 2013 05:17:07 -0000 --dS6nsELLbkZi5mD/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Oct 09, 2013 at 03:42:27PM +0400, Dmitry Sivachenko wrote: > Hello! >=20 > I have a program which mmap()s a lot of large files (total size more that= RAM and I have no swap), but it needs only small parts of that files at a = time. >=20 > My understanding is that when using mmap when I access some memory region= OS reads the relevant portion of that file from disk and caches the result= in memory. If there is no free memory, OS will purge previously read part= of mmap'ed file to free memory for the new chunk. >=20 > But this is not the case. I use the following simple program which gets = list of files as command line arguments, mmap()s them all and then selects = random file and random 1K parts of that file and computes a XOR of bytes fr= om that region. > After some time the program dies: > pid 63251 (a.out), uid 1232, was killed: out of swap space >=20 > It seems I incorrectly understand how mmap() works, can you please clarif= y what's going wrong? >=20 > I expect that program to run indefinitely, purging some regions out of RA= M and reading the relevant parts of files. >=20 You did not specified several very important parameters for your test: 1. total amount of RAM installed 2. count of the test files and size of the files 3. which filesystem files are located at 4. version of the system. --dS6nsELLbkZi5mD/ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJSV4nNAAoJEJDCuSvBvK1BTasQAIJrIGGMZ8NGSsbVdctIqLDk m2rfa8lzM9HjH2LucSFFMxSOnPmv6M5l7AadO3wmuItF6mgOlyFMojNxX2SpxXsQ SZ9AlIDgGGdRkBh4b+QZCNc1vy1E33wolhY8K1Kjn0dt54yr3O9SeTfHBfFJpFH9 Ty9OUtLxw54/s4vTIt8h8URCyP1s3IntOwqFXJGWgVnJ4IUO47gIUJTOt3N4t7KY zhFRpA7lqkdaL+6zWOKhU68cDL/yXnJ0S3b9iJxMLMq+W1TThHcQpzQKCqXpy5d3 i5Rsd2YjiR6tr3a/ONSfjeRD0Js/Nh7wSVnzb71y5b+l9tdqwFOLu2MlqzuPLMgR D3S2bPY4qe3RKlSEP9wmADmI6b6zw24M1+KLsPybHd3lM7oOB/bIipBbprM2HnJc 1/moz0oBtJayONFZap3waNlikyNOFEMj4HDZEB/L/udo9bp6hDetAFVjeXWHVnMS 2+tM4dE7U9LfYyTa6tK901jF+ZlRILAOaOYA78Oq9RQqbtHl/X7G9G4vx2tFJ3eP 35wNA8ocll+yK3qO67/u9s0y9p4N1rawLWiIeyEmaMhIIDvSzvpiweXvRAKCa7FM PLRSxuVkG2PBwd+JUxMGRiIRJkil2ZOhjS47Ngu+RDON0pKwG9hjqOM9BAg1KuFf CzI/Jk1cynvlGnSV10in =ApAb -----END PGP SIGNATURE----- --dS6nsELLbkZi5mD/--