From owner-freebsd-hackers@freebsd.org Sat Oct 13 23:50:37 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 76D6110D80C3 for ; Sat, 13 Oct 2018 23:50:37 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DC3568E6EE; Sat, 13 Oct 2018 23:50:36 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w9DNoLxc049887 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 14 Oct 2018 02:50:24 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w9DNoLxc049887 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w9DNoLFx049882; Sun, 14 Oct 2018 02:50:21 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 14 Oct 2018 02:50:21 +0300 From: Konstantin Belousov To: Thomas Munro Cc: freebsd-hackers@freebsd.org, alc@freebsd.org, mjg@freebsd.org Subject: Re: PostgresSQL vs super pages Message-ID: <20181013235021.GX5335@kib.kiev.ua> References: <20181011001954.GV5335@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Oct 2018 23:50:37 -0000 On Thu, Oct 11, 2018 at 02:01:20PM +1300, Thomas Munro wrote: > On Thu, 11 Oct 2018 at 13:20, Konstantin Belousov wrote: > > On Thu, Oct 11, 2018 at 12:59:41PM +1300, Thomas Munro wrote: > > > shm_open("/PostgreSQL.1721888107",O_RDWR|O_CREAT|O_EXCL,0600) = 46 (0x2e) > > > ftruncate(46,0x400000) = 0 (0x0) > > Try to write zeroes instead of truncating. > > This should activate the fast path in the fault handler, and if the > > pages allocated for backing store of the shm object were from reservation, > > you should get superpage mapping on the first fault without promotion. > > If you just write() to a newly shm_open()'d fd you get a return code > of 0 so I assume that doesn't work. If you ftruncate() to the desired > size first, then loop writing 8192 bytes of zeroes at a time, it > works. But still no super pages. I tried also with a write buffer of > 2MB of zeroes, but still no super pages. I tried abandoning > shm_open() and instead using a mapped file, and still no super pages. I did not quite scientific experiment, but you would need to try to find the differences between what I did and what you observe. Below is the naive test program that directly implements my suggestion, and the output from the procstat -v for it after all things were set up. /* $Id: shm_super.c,v 1.1 2018/10/13 23:49:37 kostik Exp kostik $ */ #include #include #include #include #include #include #include #include #define M(x) ((x) * 1024 * 1024) #define SZ M(4) int main(void) { char buf[128]; void *ptr; off_t cnt; int error, shmfd; shmfd = shm_open(SHM_ANON, O_CREAT | O_RDWR, 0600); if (shmfd == -1) err(1, "shm_open"); error = ftruncate(shmfd, SZ); if (error == -1) err(1, "truncate"); memset(buf, 0, sizeof(buf)); for (cnt = 0; cnt < SZ; cnt += sizeof(buf)) { error = write(shmfd, buf, sizeof(buf)); if (error == -1) err(1, "write"); else if (error != sizeof(buf)) errx(1, "short write %d", (int)error); } ptr = mmap(NULL, SZ, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ALIGNED_SUPER, shmfd, 0); if (ptr == MAP_FAILED) err(1, "mmap"); for (cnt = 0; cnt < SZ; cnt += PAGE_SIZE) *((char *)ptr + cnt) = 0; printf("ptr %p\n", ptr); snprintf(buf, sizeof(buf), "procstat -v %d", getpid()); system(buf); } $ ./shm_super ptr 0x800e00000 PID START END PRT RES PRES REF SHD FLAG TP PATH 98579 0x400000 0x401000 r-x 1 3 1 0 CN-- vn /usr/home/kostik/work/build/bsd/DEV/stuff/tests/shm_super 98579 0x600000 0x601000 rw- 1 1 1 0 ---- df 98579 0x800600000 0x800620000 r-x 32 34 146 72 CN-- vn /libexec/ld-elf.so.1 98579 0x800620000 0x800644000 rw- 24 24 1 0 ---- df 98579 0x80081f000 0x800820000 rw- 1 0 1 0 C--- vn /libexec/ld-elf.so.1 98579 0x800820000 0x800821000 rw- 1 1 1 0 ---- df 98579 0x800821000 0x8009b3000 r-x 402 440 146 72 CN-- vn /lib/libc.so.7 98579 0x8009b3000 0x800bb3000 --- 0 0 0 0 CN-- -- 98579 0x800bb3000 0x800bbf000 rw- 12 0 1 0 C--- vn /lib/libc.so.7 98579 0x800bbf000 0x800bd9000 rw- 5 14 2 0 ---- df 98579 0x800c00000 0x800e00000 rw- 9 14 2 0 ---- df 98579 0x800e00000 0x801200000 rw- 1024 1030 3 0 --S- df 98579 0x801200000 0x801400000 rw- 6 1030 3 0 ---- df 98579 0x7fffdffff000 0x7ffffffdf000 --- 0 0 0 0 ---- -- 98579 0x7ffffffdf000 0x7ffffffff000 rw- 4 4 1 0 ---D df 98579 0x7ffffffff000 0x800000000000 r-x 1 1 81 0 ---- ph