Date: Tue, 28 Oct 2008 11:27:55 -0700 From: Chuck Swiger <cswiger@mac.com> To: =?ISO-8859-1?Q?Francis_Dub=E9?= <freebsd@optiksecurite.com> Cc: Jeremy Chadwick <koitsu@FreeBSD.org>, freebsd-questions@freebsd.org Subject: Re: collecting pv entries -- suggest increasing PMAP_SHPGPERPROC Message-ID: <25FDD059-6BE7-4FAB-8FEB-C3D88789EFF2@mac.com> In-Reply-To: <4907428A.4030305@optiksecurite.com> References: <49060AE0.3000301@optiksecurite.com> <E0956D53-6EA3-4E70-91B2-6606EA48869A@mac.com> <49061898.60903@optiksecurite.com> <6DE465B2-C9D2-4A97-994F-D08CD7FE6211@mac.com> <20081027200253.GA29814@icarus.home.lan> <49072561.90904@optiksecurite.com> <20081028162511.GA53758@icarus.home.lan> <4907428A.4030305@optiksecurite.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Oct 28, 2008, at 9:49 AM, Francis Dub=E9 wrote: >>> Here's an example of top's output regarding our httpd process : >>> 54326 apache 1 96 0 156M 13108K select 1 0:00 =20 >>> 0.15% httpd >>> 54952 apache 1 96 0 156M 12684K select 1 0:00 =20 >>> 0.10% httpd >>> 52343 apache 1 4 0 155M 12280K select 0 0:01 =20 >>> 0.10% httpd >>> >>> Most of our page are in HTML with a LOT of images. Few PHP pages, =20= >>> very light PHP processing. >>> >>> 156M x 450 process =3D way more RAM than what we have (same for =20 >>> RES). Concretely, how must I interpret these results? First, your Apache children are huge, at least for FreeBSD. :-) =20 Also, they are mostly paged out, which suggests your system is under =20 significant VM pressure, but the vmstat output would be helpful to =20 confirm. >> It's as I expected -- you don't understand the difference between >> SIZE (SZ) and RES (RSS). The simple version: >> >> SIZE =3D=3D amount of memory that's shared across all processes on = the >> machine, e.g. shared libraries. It doesn't mean "156MB is being =20 >> taken >> up per process". SIZE =3D=3D the amount of VM address space allocated by the process. It includes things shared (copy-on-write) between many processes like =20= the shared libraries; it also includes memory-mapped files =20 (including .so's like apache modules being loaded into the process), =20 VM allocated but not yet used by malloc()/brk(), the stack, and so =20 forth. >> RES =3D=3D amount of memory that's specifically allocated to that =20 >> individual >> process. The three httpd processes above are taking up a total of >> ~38MBytes of memory (13108K + 12684K + 12280K). RES =3D=3D the amount of process VM that is resident in actual physical =20= RAM; the rest of the process is paged out to the swapfile or =20 filesystem for memory-mapped files. > As I said, even with RES the numbers dont seems to have any sense. > > Let's say 12500K x 450 =3D ~5500MBytes. Considering there's a lot of =20= > process other than Apache running on the server...there's something =20= > wrong. Is there something shared in RES too ? Yep. Quite probably a lot, but the amount of memory which is specific =20= to just that process is not easily found from FreeBSD's top, =20 regrettably. For the sake of example, and because the same explanation applies =20 pretty closly to FreeBSD, consider an httpd running on a MacOSX =20 system. Here's top output, which includes columns "RPRVT" for =20 "resident memory used by just this process", "RSHRD" which is =20 "resident, shared with other processes", "RSIZE" which is FreeBSD's =20 "RES", and "VSIZE", which is FreeBSD's "SIZE": Processes: 136 total, 4 running, 132 sleeping... 215 threads =20 11:06:40 Load Avg: 1.71, 1.66, 1.62 CPU usage: 12.5% user, 59.7% sys, =20 27.8% idle SharedLibs: num =3D 141, resident =3D 18.3M code, 2.92M data, 6.40M =20 LinkEdit MemRegions: num =3D 10360, resident =3D 101M + 5.91M private, 159M = shared PhysMem: 159M wired, 252M active, 99.0M inactive, 510M used, 1.50G =20= free VM: 7.16G + 88.8M 1378510(0) pageins, 88743(0) pageouts PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD =20 RSIZE VSIZE 2868 httpd 0.0% 43:21.28 1 12 92 1.82M 144M =20 72.9M 169M 2869 httpd 0.0% 46:29.45 1 12 92 1.95M 144M =20 73.2M 169M 2870 httpd 0.0% 46:55.84 1 12 92 1.89M 144M =20 73.0M 169M ...and the vmmap command, documented here: = http://developer.apple.com/documentation/Darwin/Reference/ManPages/man1/vm= map.1.html ...provides detailed info about a single process' VM usage: # vmmap 2870 Virtual Memory Map of process 2870 (httpd) Output report format: 2.0 =3D=3D=3D=3D Non-writable regions for process 2870 __PAGEZERO 00000000-00001000 [ 4K] ---/--- SM=3DNUL = /usr/=20 sbin/httpd __TEXT 00001000-00050000 [ 316K] r-x/rwx SM=3DCOW = /usr/=20 sbin/httpd __LINKEDIT 0005a000-00065000 [ 44K] r--/rwx SM=3DCOW = /usr/=20 sbin/httpd __TEXT 00065000-00068000 [ 12K] r-x/rwx SM=3DCOW = /usr/=20 libexec/httpd/mod_log_config.so __LINKEDIT 00069000-0006a000 [ 4K] r--/rwx SM=3DCOW = /usr/=20 libexec/httpd/mod_log_config.so __TEXT 0006a000-0006c000 [ 8K] r-x/rwx SM=3DCOW = /usr/=20 libexec/httpd/mod_mime.so __LINKEDIT 0006d000-0006e000 [ 4K] r--/rwx SM=3DCOW = /usr/=20 libexec/httpd/mod_mime.so [ ... ] __DATA a1a0e000-a1a20000 [ 72K] r--/r-- SM=3DCOW = /usr/=20 lib/libcrypto.0.9.7.dylib __DATA a1a20000-a1a23000 [ 12K] r--/r-- SM=3DCOW = /usr/=20 lib/libcrypto.0.9.7.dylib __DATA a4f2c000-a4f2f000 [ 12K] r--/r-- SM=3DCOW = /usr/=20 lib/libssl.0.9.7.dylib __DATA a7233000-a7235000 [ 8K] r--/r-- SM=3DNUL /=20 System/Library/Perl/lib/5.8/libperl.dylib system fffec000-fffef000 [ 12K] ---/rwx SM=3DNUL =20 commpage [libobjc.A.dylib] system fffef000-ffff0000 [ 4K] r-x/rwx SM=3DCOW =20 commpage [libobjc.A.dylib] system ffff8000-ffffa000 [ 8K] r--/r-- SM=3DSHM =20 commpage [libSystem.B.dylib] =3D=3D=3D=3D Writable regions for process 2870 __DATA 00050000-00059000 [ 36K] rw-/rwx SM=3DCOW = /usr/=20 sbin/httpd __DATA 00059000-0005a000 [ 4K] rw-/rwx SM=3DCOW = /usr/=20 sbin/httpd __DATA 00068000-00069000 [ 4K] rw-/rwx SM=3DCOW = /usr/=20 libexec/httpd/mod_log_config.so __DATA 0006c000-0006d000 [ 4K] rw-/rwx SM=3DCOW = /usr/=20 libexec/httpd/mod_mime.so [ ... ] __DATA a0a3a000-a0a4f000 [ 84K] rw-/rw- SM=3DCOW = /usr/=20 lib/libobjc.A.dylib __OBJC a0a4f000-a0a50000 [ 4K] rw-/rw- SM=3DCOW = /usr/=20 lib/libobjc.A.dylib __DATA a0b70000-a0b71000 [ 4K] rw-/rw- SM=3DCOW = /usr/=20 lib/libauto.dylib __DATA a1425000-a1426000 [ 4K] rw-/rw- SM=3DCOW = /usr/=20 lib/libgcc_s.1.dylib __DATA a7229000-a7233000 [ 40K] rw-/rw- SM=3DCOW /=20 System/Library/Perl/lib/5.8/libperl.dylib Stack bf800000-c0000000 [ 8192K] rw-/rwx SM=3DCOW =20 thread 0 =3D=3D=3D=3D Legend SM=3Dsharing mode: COW=3Dcopy_on_write PRV=3Dprivate NUL=3Dempty ALI=3Daliased SHM=3Dshared ZER=3Dzero_filled S/A=3Dshared_alias =3D=3D=3D=3D Summary for process 2870 ReadOnly portion of Libraries: Total=3D18840KB resident=3D10976KB(58%) =20= swapped_out_or_unallocated=3D7864KB(42%) Writable regions: Total=3D51100KB written=3D548KB(1%) =20 resident=3D27276KB(53%) swapped_out=3D0KB(0%) unallocated=3D23824KB(47%) REGION TYPE [ VIRTUAL] =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D [ =3D=3D=3D=3D=3D=3D=3D] MALLOC [ 42400K] Stack [ 8192K] VM_ALLOCATE ? [ 364K] __DATA [ 1048K] __LINKEDIT [ 4352K] __OBJC [ 4K] __PAGEZERO [ 4K] __TEXT [ 14488K] mapped file [ 120368K] shared memory [ 4K] system [ 24K] Note that you can obtain somewhat similar information under FreeBSD =20 using the sysutils/pmap port: PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU =20= COMMAND 40037 www 1 4 0 28820K 11532K accept 1:10 0.00% =20 httpd # pmap 40037 40037: /usr/local/sbin/httpd Address Kbytes RSS Shared Priv Mode Mapped File 08048000 280 208 280 - r-x /usr/local/sbin/httpd 0808E000 8 8 - 8 rw- /usr/local/sbin/httpd 08090000 16 - - - rw- [swap pager] 08094000 976 - - - rwx [swap pager] 08188000 2420 - - - rwx [swap pager] 083E5000 6968 - - - rwx [swap pager] 2808E000 144 92 144 - r-x /libexec/ld-elf.so.1 280B2000 8 4 8 - rw- /libexec/ld-elf.so.1 280B4000 20 - - - rw- [swap pager] 280B9000 32 - - - rwx [swap pager] 280C1000 64 16 64 - r-x /lib/libz.so.3 [ ... ] 2920F000 4 0 4 - r-x /usr/local/lib/php/=20 20060613/dom.so 29210000 16 0 16 - rwx /usr/local/lib/php/=20 20060613/dom.so 29214000 60 - - - rwx [swap pager] 29225000 16 - - - rwx [swap pager] BFBE0000 128 - - - rwx [swap pager] -------- ------- ------- ------- ------- Total Kb 28740 3996 17384 456 ...and the pmap manpage talks about how to understand this: " To calculate the amount of memory a group of processes is using: - for each process of the same binary, count the private =20= memory used for n-1 processes. - for each binary, count the total resident size once. For example, using the pmap data above, we can calculate that 20 =20= bash binaries would used 452Kbytes * 19 + 1416Kbytes =3D 8588 + 1416 Kbytes =3D 8588Kbytes" Regards, --=20 -Chuck
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?25FDD059-6BE7-4FAB-8FEB-C3D88789EFF2>