Date: Wed, 11 Oct 2006 14:24:29 -0400 From: Jeff Dickens <jeff@intranet.seamanpaper.com> Cc: freebsd-questions@freebsd.org Subject: Re: optimal kernel options for VMWARE guest system Message-ID: <452D36DD.1070800@intranet.seamanpaper.com> In-Reply-To: <4523C9C2.6060000@seamanpaper.com> References: <4522969F.9010504@seamanpaper.com> <200610031605.54121.lists@jnielsen.net> <4523C9C2.6060000@seamanpaper.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Jeff Dickens wrote: > John Nielsen wrote: >> On Tuesday 03 October 2006 12:58, Jeff Dickens wrote: >> >>> I have some Freebsd systems that are running as VMware guests. I'd >>> like >>> to configure their kernels so as to minimize the overhead on the VMware >>> host system. After reading and partially digesting the white paper on >>> timekeeping in VMware virtual machines >>> (http://www.vmware.com/pdf/vmware_timekeeping.pdf) it appears that I >>> might want to make some changes. >>> >>> Has anyone addressed this issue? >>> >> >> I haven't read the white paper (yet; thanks for the link), but I've >> had good results with recent -STABLE VM's running under ESX server 3. >> Some thoughts: >> >> As I do on most of my installs, I trimmed down GENERIC to include >> just the drivers I use. In this case that was mpt for the disk and le >> for the network (although I suspect forcing the VM to present e1000 >> hardware and then using the em driver would work as well if not better). >> >> The VMware tools package that comes with ESX server does a poor job >> of getting itself to run, but it can be made to work without too much >> difficulty. Don't use the port, run the included install script to >> install the files, ignore the custom network driver and compile the >> memory management module from source (included). If using X.org, use >> the built-in vmware display driver, and copy the vmmouse driver .o >> file from the VMware tools dist to the appropriate dir under >> /usr/X11. Even though the included file is for X.org 6.8, it works >> fine with 6.9/7.0 (X.org 7.1 should include the vmmouse driver.) Run >> the VMware tools config script from a non-X terminal (and you can >> ignore the warning about running it remotely if you're using SSH), so >> it won't mess with your X display (it doesn't do anything not >> accomplished above). Then run the rc.d script to start the VMware tools. >> >> I haven't noticed any timekeeping issues so far. >> >> JN >> _______________________________________________ >> > What is the advantage of using the "e1000 hardware", and is this > documented somewhere? I got the vxn network driver working without > issues; I just had to edit the .vxn file manually: I'm using the free > VMware server V1 rather than the ESX server. > > ethernet0.virtualDev="vmxnet" > > I've got timekeeping running stably on these. I turn on time sync via > vmware tools in the .vmx file: > > tools.syncTime = "TRUE" > > and in the guest file's rc.conf start ntpd with flags "-Aqgx &" so it > just syncs once at boot and exits. > > I'm not using X on these. They're supposed to be clean & lean systems > to run such things as djbdns and qmail. And they do work well. > My main goal is to reduce the background load on the VMware host > system so that it isn't spending more time than it has to simulating > interrupt controllers for the guests. I'm wondering about the > "disable ACPI" boot option. I suppose I first should figure out how > to even roughly measure the effect of any changes I might make. > Well, I've done some pseudo-scientific measurement on this. I currently have five freebsd virtual systems running, and one Centos 4 (linux 2.6), This command give some info on the background cpu usage: (The host is a Centos 3 system, linux 2.4) [root@otter root]# ps auxww | head -1 USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [root@otter root]# ps auxww | grep vmx root 18031 12.7 1.5 175440 39916 ? S< Oct09 345:50 /usr/lib/vmware/bin/vmware-vmx -C /var/lib/vmware/Virtual Machines/Goose/freebsd-6.1-i386.vmx -@ "" root 18058 12.9 1.4 174772 36916 ? S< Oct09 351:01 /usr/lib/vmware/bin/vmware-vmx -C /var/lib/vmware/Virtual Machines/Duck/freebsd-6.1-i386.vmx -@ "" root 18072 16.2 5.5 246372 141776 ? S< Oct09 440:16 /usr/lib/vmware/bin/vmware-vmx -C /var/lib/vmware/Virtual Machines/BlueJay/freebsd-6.1-i386.vmx -@ "" root 18086 12.9 1.4 174688 38464 ? S< Oct09 351:47 /usr/lib/vmware/bin/vmware-vmx -C /var/lib/vmware/Virtual Machines/Heron/freebsd-6.1-i386.vmx -@ "" root 18100 9.4 4.1 385712 107348 ? S< Oct09 256:25 /usr/lib/vmware/bin/vmware-vmx -C /var/lib/vmware/Virtual Machines/Newt/freebsd-6.1-i386.vmx -@ "" root 18139 12.2 2.5 299388 65132 ? S< Oct09 330:35 /usr/lib/vmware/bin/vmware-vmx -C /var/lib/vmware/Virtual Machines/Centos4/Centos4.vmx -@ "" root 28930 0.0 0.0 3680 672 pts/3 S 14:08 0:00 grep vmx [root@otter root]# As one can see the one called "Newt" is consistently lower in the "%CPU" column. Curiously enough, this *is* the one I built a custom kernel for. The config file I used is posted below: Besides commenting out devices I wasn't using & NFS, etc, I commented out the apic and pctimer devices. Do you think I'm on the right track for reducing interrupt frequency? Also, if I were to want to move this kernel to other FreeBSD systems, how much has to move, the whole /boot/kernel directory? Finally I did have to re-run the vmware-config-tools.pl script after rebuilding the kernel. newt# cat VMWARE1 # # GENERIC -- Generic kernel configuration file for FreeBSD/i386 # # For more information on this file, please read the handbook section on # Kernel Configuration Files: # # http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html # # The handbook is also available locally in /usr/share/doc/handbook # if you've installed the doc distribution, otherwise always see the # FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the # latest information. # # An exhaustive list of options and more detailed explanations of the # device lines is also present in the ../../conf/NOTES and NOTES files. # If you are in doubt as to the purpose or necessity of a line, check first # in NOTES. # # $FreeBSD: src/sys/i386/conf/GENERIC,v 1.429.2.7.2.2 2006/05/01 00:15:12 scottl Exp $ machine i386 cpu I686_CPU ident VMWARE1 # To statically compile in device wiring instead of /boot/device.hints #hints "GENERIC.hints" # Default places to look for devices. makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols #options SCHED_ULE # ULE scheduler options SCHED_4BSD # 4BSD scheduler options PREEMPTION # Enable kernel thread preemption options INET # InterNETworking #options INET6 # IPv6 communications protocols options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists options UFS_DIRHASH # Improve performance on big directories options MD_ROOT # MD is a potential root device #options NFSCLIENT # Network Filesystem Client #options NFSSERVER # Network Filesystem Server #options NFS_ROOT # NFS usable as /, requires NFSCLIENT #options MSDOSFS # MSDOS Filesystem options CD9660 # ISO 9660 Filesystem options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_GPT # GUID Partition Tables. options COMPAT_43 # Compatible with BSD 4.3 [KEEP THIS!] options COMPAT_FREEBSD4 # Compatible with FreeBSD4 options COMPAT_FREEBSD5 # Compatible with FreeBSD5 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support options SYSVSHM # SYSV-style shared memory options SYSVMSG # SYSV-style message queues options SYSVSEM # SYSV-style semaphores options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions options KBD_INSTALL_CDEV # install a CDEV entry in /dev options AHC_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~128k to driver. options AHD_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~215k to driver. options ADAPTIVE_GIANT # Giant mutex is adaptive. #device apic # I/O APIC # Bus support. #device eisa device pci # Floppy drives #device fdc # ATA and ATAPI devices device ata device atadisk # ATA disk drives #device ataraid # ATA RAID drives device atapicd # ATAPI CDROM drives #device atapifd # ATAPI floppy drives #device atapist # ATAPI tape drives options ATA_STATIC_ID # Static device numbering # SCSI Controllers #device ahb # EISA AHA1742 family #device ahc # AHA2940 and onboard AIC7xxx devices #device ahd # AHA39320/29320 and onboard AIC79xx devices #device amd # AMD 53C974 (Tekram DC-390(T)) #device isp # Qlogic family ##device ispfw # Firmware for QLogic HBAs- normally a module #device mpt # LSI-Logic MPT-Fusion ##device ncr # NCR/Symbios Logic #device sym # NCR/Symbios Logic (newer chipsets + those of `ncr') #device trm # Tekram DC395U/UW/F DC315U adapters #device adv # Advansys SCSI adapters #device adw # Advansys wide SCSI adapters #device aha # Adaptec 154x SCSI adapters #device aic # Adaptec 15[012]x SCSI adapters, AIC-6[23]60. #device bt # Buslogic/Mylex MultiMaster SCSI adapters #device ncv # NCR 53C500 #device nsp # Workbit Ninja SCSI-3 #device stg # TMC 18C30/18C50 # SCSI peripherals #device scbus # SCSI bus (required for SCSI) #device ch # SCSI media changers #device da # Direct Access (disks) #device sa # Sequential Access (tape etc) #device cd # CD #device pass # Passthrough device (direct SCSI access) #device ses # SCSI Environmental Services (and SAF-TE) # RAID controllers interfaced to the SCSI subsystem #device amr # AMI MegaRAID #device arcmsr # Areca SATA II RAID #device asr # DPT SmartRAID V, VI and Adaptec SCSI RAID #device ciss # Compaq Smart RAID 5* #device dpt # DPT Smartcache III, IV - See NOTES for options #device hptmv # Highpoint RocketRAID 182x #device rr232x # Highpoint RocketRAID 232x #device iir # Intel Integrated RAID #device ips # IBM (Adaptec) ServeRAID #device mly # Mylex AcceleRAID/eXtremeRAID #device twa # 3ware 9000 series PATA/SATA RAID # RAID controllers #device aac # Adaptec FSA RAID #device aacp # SCSI passthrough for aac (requires CAM) #device ida # Compaq Smart RAID #device mfi # LSI MegaRAID SAS #device mlx # Mylex DAC960 family #device pst # Promise Supertrak SX6000 #device twe # 3ware ATA RAID # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc # AT keyboard controller device atkbd # AT keyboard device psm # PS/2 mouse device kbdmux # keyboard multiplexer device vga # VGA video card driver device splash # Splash screen and screen saver support # syscons is the default console driver, resembling an SCO console device sc # Enable this for the pcvt (VT220 compatible) console driver #device vt #options XSERVER # support for X server on a vt console #options FAT_CURSOR # start with block cursor #device agp # support several AGP chipsets # Power management support (see NOTES for more options) #device apm # Add suspend/resume support for the i8254. #device pmtimer # PCCARD (PCMCIA) support # PCMCIA and cardbus bridge support #device cbb # cardbus (yenta) bridge #device pccard # PC Card (16-bit) bus #device cardbus # CardBus (32-bit) bus # Serial (COM) ports #device sio # 8250, 16[45]50 based serial ports # Parallel port #device ppc #device ppbus # Parallel port bus (required) #device lpt # Printer #device plip # TCP/IP over parallel #device ppi # Parallel port interface device ##device vpo # Requires scbus and da # If you've got a "dumb" serial or parallel PCI card that is # supported by the puc(4) glue driver, uncomment the following # line to enable it (connects to the sio and/or ppc drivers): #device puc # PCI Ethernet NICs. #device de # DEC/Intel DC21x4x (``Tulip'') #device em # Intel PRO/1000 adapter Gigabit Ethernet Card #device ixgb # Intel PRO/10GbE Ethernet Card #device txp # 3Com 3cR990 (``Typhoon'') device vx # 3Com 3c590, 3c595 (``Vortex'') # PCI Ethernet NICs that use the common MII bus controller code. # NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! device miibus # MII bus support #device bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet #device bfe # Broadcom BCM440x 10/100 Ethernet #device bge # Broadcom BCM570xx Gigabit Ethernet #device dc # DEC/Intel 21143 and various workalikes #device fxp # Intel EtherExpress PRO/100B (82557, 82558) #device lge # Level 1 LXT1001 gigabit Ethernet #device nge # NatSemi DP83820 gigabit Ethernet #device nve # nVidia nForce MCP on-board Ethernet Networking #device pcn # AMD Am79C97x PCI 10/100(precedence over 'lnc') #device re # RealTek 8139C+/8169/8169S/8110S #device rl # RealTek 8129/8139 #device sf # Adaptec AIC-6915 (``Starfire'') #device sis # Silicon Integrated Systems SiS 900/SiS 7016 #device sk # SysKonnect SK-984x & SK-982x gigabit Ethernet #device ste # Sundance ST201 (D-Link DFE-550TX) #device ti # Alteon Networks Tigon I/II gigabit Ethernet #device tl # Texas Instruments ThunderLAN #device tx # SMC EtherPower II (83c170 ``EPIC'') #device vge # VIA VT612x gigabit Ethernet #device vr # VIA Rhine, Rhine II #device wb # Winbond W89C840F #device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') # ISA Ethernet NICs. pccard NICs included. #device cs # Crystal Semiconductor CS89x0 NIC # 'device ed' requires 'device miibus' #device ed # NE[12]000, SMC Ultra, 3c503, DS8390 cards #device ex # Intel EtherExpress Pro/10 and Pro/10+ #device ep # Etherlink III based cards #device fe # Fujitsu MB8696x based cards #device ie # EtherExpress 8/16, 3C507, StarLAN 10 etc. device lnc # NE2100, NE32-VL Lance Ethernet cards #device sn # SMC's 9000 series of Ethernet chips #device xe # Xircom pccard Ethernet # Wireless NIC cards #device wlan # 802.11 support #device an # Aironet 4500/4800 802.11 wireless NICs. #device awi # BayStack 660 and others #device ral # Ralink Technology RT2500 wireless NICs. #device wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. ##device wl # Older non 802.11 Wavelan wireless NIC. # Pseudo devices. device loop # Network loopback device random # Entropy device device ether # Ethernet support #device sl # Kernel SLIP #device ppp # Kernel PPP #device tun # Packet tunnel. device pty # Pseudo-ttys (telnet etc) device md # Memory "disks" #device gif # IPv6 and IPv4 tunneling #device faith # IPv6-to-IPv4 relaying (translation) # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! # Note that 'bpf' is required for DHCP. device bpf # Berkeley packet filter # USB support #device uhci # UHCI PCI->USB interface #device ohci # OHCI PCI->USB interface #device ehci # EHCI PCI->USB interface (USB 2.0) #device usb # USB Bus (required) ##device udbp # USB Double Bulk Pipe devices #device ugen # Generic #device uhid # "Human Interface Devices" #device ukbd # Keyboard #device ulpt # Printer #device umass # Disks/Mass storage - Requires scbus and da #device ums # Mouse #device ural # Ralink Technology RT2500USB wireless NICs #device urio # Diamond Rio 500 MP3 player #device uscanner # Scanners # USB Ethernet, requires miibus #device aue # ADMtek USB Ethernet #device axe # ASIX Electronics USB Ethernet #device cdce # Generic USB over Ethernet #device cue # CATC USB Ethernet #device kue # Kawasaki LSI USB Ethernet #device rue # RealTek RTL8150 USB Ethernet # FireWire support #device firewire # FireWire bus code #device sbp # SCSI over FireWire (Requires scbus and da) #device fwe # Ethernet over FireWire (non-standard!) newt#
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?452D36DD.1070800>