From owner-freebsd-threads@FreeBSD.ORG Mon Dec 12 11:02:46 2005 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C0F4F16A425 for ; Mon, 12 Dec 2005 11:02:46 +0000 (GMT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id D068643D7C for ; Mon, 12 Dec 2005 11:02:39 +0000 (GMT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id jBCB2YmE064816 for ; Mon, 12 Dec 2005 11:02:34 GMT (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id jBCB2X7t064810 for freebsd-threads@freebsd.org; Mon, 12 Dec 2005 11:02:33 GMT (envelope-from owner-bugmaster@freebsd.org) Date: Mon, 12 Dec 2005 11:02:33 GMT Message-Id: <200512121102.jBCB2X7t064810@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Cc: Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2005 11:02:47 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2005/01/26] threads/76690threads fork hang in child for (-lc_r & -lthr) 1 problem total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/07/18] kern/20016 threads pthreads: Cannot set scheduling timer/Can o [2000/08/26] kern/20861 threads libc_r does not honor socket timeouts o [2001/01/20] threads/24472threads libc_r does not honor SO_SNDTIMEO/SO_RCVT o [2001/01/25] threads/24632threads libc_r delicate deviation from libc in ha o [2001/01/25] kern/24641 threads pthread_rwlock_rdlock can deadlock o [2001/11/26] bin/32295 threads pthread dont dequeue signals o [2002/02/01] threads/34536threads accept() blocks other threads o [2002/05/25] kern/38549 threads the procces compiled whith pthread stoppe o [2002/06/27] threads/39922threads [threads] [patch] Threaded applications e o [2002/08/04] kern/41331 threads Pthread library open sets O_NONBLOCK flag o [2003/03/02] threads/48856threads Setting SIGCHLD to SIG_IGN still leaves z o [2003/03/10] threads/49087threads Signals lost in programs linked with libc s [2004/03/15] kern/64313 threads FreeBSD (OpenBSD) pthread implicit set/un o [2004/08/26] threads/70975threads unexpected and unreliable behaviour when o [2004/10/05] threads/72353threads Assertion fails in /usr/src/lib/libpthrea o [2004/10/07] threads/72429threads threads blocked in stdio (fgets, etc) are o [2004/10/21] threads/72953threads fork() unblocks blocked signals w/o PTHRE o [2004/12/19] threads/75273threads FBSD 5.3 libpthread (KSE) bug o [2004/12/21] threads/75374threads pthread_kill() ignores SA_SIGINFO flag o [2005/01/26] threads/76694threads fork cause hang in dup()/close() function o [2005/03/10] threads/78660threads Java hangs unkillably in STOP state after o [2005/04/08] threads/79683threads svctcp_create() fails if multiple threads o [2005/04/28] threads/80435threads panic on high loads o [2005/05/19] threads/81258threads Thread specific data is sometimes assigne o [2005/08/02] threads/84483threads problems with devel/nspr and -lc_r on 4.x o [2005/08/20] threads/85160threads [libthr] [patch] libobjc + libpthread/lib o [2005/11/19] threads/89262threads [kernel] [patch] multi-threaded process h 27 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/06/13] kern/19247 threads uthread_sigaction.c does not do anything o [2000/10/21] kern/22190 threads A threaded read(2) from a socketpair(2) f o [2001/09/09] threads/30464threads pthread mutex attributes -- pshared o [2002/05/02] threads/37676threads libc_r: msgsnd(), msgrcv(), pread(), pwri s [2002/07/16] threads/40671threads pthread_cancel doesn't remove thread from o [2004/07/13] threads/69020threads pthreads library leaks _gc_mutex o [2004/09/21] threads/71966threads Mlnet Core Dumped : Fatal error '_pq_inse o [2004/11/21] threads/74180threads KSE problem. Applications those riched ma o [2005/04/13] threads/79887threads [patch] freopen() isn't thread-safe o [2005/05/13] threads/80992threads abort() sometimes not caught by gdb depen o [2005/05/26] threads/81534threads [libc_r] [patch] libc_r close() will fail 11 problems total. From owner-freebsd-threads@FreeBSD.ORG Mon Dec 12 13:40:05 2005 Return-Path: X-Original-To: freebsd-threads@hub.freebsd.org Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D765C16A41F for ; Mon, 12 Dec 2005 13:40:05 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id F105643D66 for ; Mon, 12 Dec 2005 13:40:02 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id jBCDe2ul080942 for ; Mon, 12 Dec 2005 13:40:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id jBCDe2NR080939; Mon, 12 Dec 2005 13:40:02 GMT (envelope-from gnats) Resent-Date: Mon, 12 Dec 2005 13:40:02 GMT Resent-Message-Id: <200512121340.jBCDe2NR080939@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-threads@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, leafy Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F206516A41F for ; Mon, 12 Dec 2005 13:30:58 +0000 (GMT) (envelope-from leafy@leafy.idv.tw) Received: from seed.net.tw (sn16.seed.net.tw [139.175.54.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 523B843D64 for ; Mon, 12 Dec 2005 13:30:56 +0000 (GMT) (envelope-from leafy@leafy.idv.tw) Received: from [59.104.136.38] (port=55456 helo=chihiro.leafy.idv.tw) by seed.net.tw with esmtp (Seednet 4.23:1) id 1ElnlX-000JAi-6t for FreeBSD-gnats-submit@freebsd.org; Mon, 12 Dec 2005 21:30:55 +0800 Received: from localhost (localhost [127.0.0.1]) by chihiro.leafy.idv.tw (Postfix) with ESMTP id 8CA1E58E for ; Mon, 12 Dec 2005 21:30:54 +0800 (CST) Received: from chihiro.leafy.idv.tw ([127.0.0.1]) by localhost (chihiro.leafy.idv.tw [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 56229-02-2 for ; Mon, 12 Dec 2005 21:30:41 +0800 (CST) Received: by chihiro.leafy.idv.tw (Postfix, from userid 1002) id D51CE343; Mon, 12 Dec 2005 22:30:49 +0800 (CST) Received: by chihiro.leafy.idv.tw (Postfix, from userid 1002) id 8B4E0351; Mon, 12 Dec 2005 15:54:05 +0800 (CST) Received: by chihiro.leafy.idv.tw (Postfix, from userid 1000) id 54B9168; Mon, 12 Dec 2005 15:37:25 +0800 (CST) Message-Id: <20051212073725.54B9168@chihiro.leafy.idv.tw> Date: Mon, 12 Dec 2005 15:37:25 +0800 (CST) From: leafy To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Cc: Subject: threads/90278: libthr, ULE and -current produces >100% WCPU with apache20/22 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: leafy List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2005 13:40:06 -0000 >Number: 90278 >Category: threads >Synopsis: libthr, ULE and -current produces >100% WCPU with apache20/22 >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-threads >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Mon Dec 12 13:40:02 GMT 2005 >Closed-Date: >Last-Modified: >Originator: leafy >Release: FreeBSD 7.0-CURRENT i386 >Organization: >Environment: System: FreeBSD chihiro.leafy.idv.tw 7.0-CURRENT FreeBSD 7.0-CURRENT #12: Fri Nov 18 19:02:38 CST 2005 leafy@chihiro.leafy.idv.tw:/usr/obj/usr/src/sys/CHIHIRO i386 >Description: Using SCHED_ULE + libthr via libmap.conf and apache20/22 from ports generats >100% WCPU usage output in top when apache is starting. last pid: 86960; load averages: 0.17, 0.04, 0.01 up 23+17:15:47 15:29:23 132 processes: 1 running, 131 sleeping CPU states: 0.3% user, 0.0% nice, 1.5% system, 0.0% interrupt, 98.2% idle Mem: 88M Active, 162M Inact, 164M Wired, 26M Cache, 60M Buf, 53M Free Swap: 512M Total, 60K Used, 512M Free PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 86958 www 27 4 0 16464K 9076K accept 0:00 446.92% httpd 86960 www 27 4 0 16464K 9076K accept 0:00 446.92% httpd 86959 www 27 4 0 16464K 9076K accept 0:00 446.92% httpd 86955 root 1 85 0 13000K 8712K select 0:03 16.55% httpd Attaching kernel config and libmap.conf # # GENERIC -- Generic kernel configuration file for FreeBSD/i386 # # For more information on this file, please read the handbook section on # Kernel Configuration Files: # # http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html # # The handbook is also available locally in /usr/share/doc/handbook # if you've installed the doc distribution, otherwise always see the # FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the # latest information. # # An exhaustive list of options and more detailed explanations of the # device lines is also present in the ../../conf/NOTES and NOTES files. # If you are in doubt as to the purpose or necessity of a line, check first # in NOTES. # # $FreeBSD: src/sys/i386/conf/GENERIC,v 1.418 2004/09/19 00:52:22 mjacob Exp $ machine i386 cpu I686_CPU ident CHIHIRO # To statically compile in device wiring instead of /boot/device.hints #hints "GENERIC.hints" # Default places to look for devices. makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols options SCHED_ULE # ULE scheduler #options SCHED_4BSD # 4BSD scheduler options PREEMPTION # Enable kernel thread preemption options INET # InterNETworking options INET6 # IPv6 communications protocols options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists options UFS_DIRHASH # Improve performance on big directories #options MD_ROOT # MD is a potential root device #options NFSCLIENT # Network Filesystem Client #options NFSSERVER # Network Filesystem Server #options NFS_ROOT # NFS usable as /, requires NFSCLIENT #options MSDOSFS # MSDOS Filesystem #options CD9660 # ISO 9660 Filesystem options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework #options GEOM_GPT # GUID Partition Tables. options COMPAT_43 # Compatible with BSD 4.3 [KEEP THIS!] #options COMPAT_FREEBSD4 # Compatible with FreeBSD4 options SCSI_DELAY=2000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support #options SYSVSHM # SYSV-style shared memory #options SYSVMSG # SYSV-style message queues #options SYSVSEM # SYSV-style semaphores options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions options KBD_INSTALL_CDEV # install a CDEV entry in /dev #options AHC_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~128k to driver. #options AHD_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~215k to driver. options ADAPTIVE_GIANT # Giant mutex is adaptive. options AHC_ALLOW_MEMIO # To make an SMP kernel, the next two lines are needed #options SMP # Symmetric MultiProcessor Kernel device apic # I/O APIC # ALTQ support options ALTQ options ALTQ_CBQ options ALTQ_RED options ALTQ_RIO options ALTQ_HFSC options ALTQ_CDNR options ALTQ_PRIQ # Polling settings options DEVICE_POLLING options HZ=1268 # Options for the VM subsystem # L2 cache size (in KB) can be specified in PQ_CACHESIZE options PQ_CACHESIZE=256 # color for 256k cache # Bus support. Do not remove isa, even if you have no isa slots device isa device pci # Floppy drives #device fdc # ATA and ATAPI devices #device ata #device atadisk # ATA disk drives #device ataraid # ATA RAID drives #device atapicd # ATAPI CDROM drives #device atapifd # ATAPI floppy drives #device atapist # ATAPI tape drives #options ATA_STATIC_ID # Static device numbering # SCSI Controllers #device ahb # EISA AHA1742 family #device ahc # AHA2940 and onboard AIC7xxx devices #device ahd # AHA39320/29320 and onboard AIC79xx devices #device amd # AMD 53C974 (Tekram DC-390(T)) #device isp # Qlogic family #device ispfw # Firmware for QLogic HBAs- normally a module #device mpt # LSI-Logic MPT-Fusion #device ncr # NCR/Symbios Logic #device sym # NCR/Symbios Logic (newer chipsets + those of `ncr') #device trm # Tekram DC395U/UW/F DC315U adapters #device adv # Advansys SCSI adapters #device adw # Advansys wide SCSI adapters #device aha # Adaptec 154x SCSI adapters #device aic # Adaptec 15[012]x SCSI adapters, AIC-6[23]60. #device bt # Buslogic/Mylex MultiMaster SCSI adapters #device ncv # NCR 53C500 #device nsp # Workbit Ninja SCSI-3 #device stg # TMC 18C30/18C50 # SCSI peripherals #device scbus # SCSI bus (required for SCSI) #device ch # SCSI media changers #device da # Direct Access (disks) #device sa # Sequential Access (tape etc) #device cd # CD #device pass # Passthrough device (direct SCSI access) #device ses # SCSI Environmental Services (and SAF-TE) # RAID controllers interfaced to the SCSI subsystem #device amr # AMI MegaRAID #device asr # DPT SmartRAID V, VI and Adaptec SCSI RAID #device ciss # Compaq Smart RAID 5* #device dpt # DPT Smartcache III, IV - See NOTES for options #device iir # Intel Integrated RAID #device ips # IBM (Adaptec) ServeRAID #device mly # Mylex AcceleRAID/eXtremeRAID #device twa # 3ware 9000 series PATA/SATA RAID # RAID controllers #device aac # Adaptec FSA RAID #device aacp # SCSI passthrough for aac (requires CAM) #device ida # Compaq Smart RAID #device mlx # Mylex DAC960 family #device pst # Promise Supertrak SX6000 #device twe # 3ware ATA RAID # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc # AT keyboard controller device atkbd # AT keyboard device psm # PS/2 mouse device vga # VGA video card driver #device splash # Splash screen and screen saver support # syscons is the default console driver, resembling an SCO console device sc # Enable this for the pcvt (VT220 compatible) console driver #device vt #options XSERVER # support for X server on a vt console #options FAT_CURSOR # start with block cursor device agp # support several AGP chipsets # Floating point support - do not disable. device npx # Power management support (see NOTES for more options) #device apm # Add suspend/resume support for the i8254. device pmtimer # PCCARD (PCMCIA) support # PCMCIA and cardbus bridge support #device cbb # cardbus (yenta) bridge #device pccard # PC Card (16-bit) bus #device cardbus # CardBus (32-bit) bus # Serial (COM) ports device sio # 8250, 16[45]50 based serial ports # Parallel port #device ppc #device ppbus # Parallel port bus (required) #device lpt # Printer #device plip # TCP/IP over parallel #device ppi # Parallel port interface device #device vpo # Requires scbus and da # If you've got a "dumb" serial or parallel PCI card that is # supported by the puc(4) glue driver, uncomment the following # line to enable it (connects to the sio and/or ppc drivers): #device puc # PCI Ethernet NICs. #device de # DEC/Intel DC21x4x (``Tulip'') #device em # Intel PRO/1000 adapter Gigabit Ethernet Card #device ixgb # Intel PRO/10GbE Ethernet Card #device txp # 3Com 3cR990 (``Typhoon'') #device vx # 3Com 3c590, 3c595 (``Vortex'') # PCI Ethernet NICs that use the common MII bus controller code. # NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! #device miibus # MII bus support #device bfe # Broadcom BCM440x 10/100 Ethernet #device bge # Broadcom BCM570xx Gigabit Ethernet #device dc # DEC/Intel 21143 and various workalikes #device fxp # Intel EtherExpress PRO/100B (82557, 82558) #device lge # Level 1 LXT1001 gigabit ethernet #device nge # NatSemi DP83820 gigabit ethernet #device pcn # AMD Am79C97x PCI 10/100 (precedence over 'lnc') #device re # RealTek 8139C+/8169/8169S/8110S #device rl # RealTek 8129/8139 #device sf # Adaptec AIC-6915 (``Starfire'') #device sis # Silicon Integrated Systems SiS 900/SiS 7016 #device sk # SysKonnect SK-984x & SK-982x gigabit Ethernet #device ste # Sundance ST201 (D-Link DFE-550TX) #device ti # Alteon Networks Tigon I/II gigabit Ethernet #device tl # Texas Instruments ThunderLAN #device tx # SMC EtherPower II (83c170 ``EPIC'') #device vge # VIA VT612x gigabit ethernet #device vr # VIA Rhine, Rhine II #device wb # Winbond W89C840F #device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') # ISA Ethernet NICs. pccard NICs included. #device cs # Crystal Semiconductor CS89x0 NIC # 'device ed' requires 'device miibus' #device ed # NE[12]000, SMC Ultra, 3c503, DS8390 cards #device ex # Intel EtherExpress Pro/10 and Pro/10+ #device ep # Etherlink III based cards #device fe # Fujitsu MB8696x based cards #device ie # EtherExpress 8/16, 3C507, StarLAN 10 etc. #device lnc # NE2100, NE32-VL Lance Ethernet cards #device sn # SMC's 9000 series of Ethernet chips #device xe # Xircom pccard Ethernet # ISA devices that use the old ISA shims #device le # Wireless NIC cards #device wlan # 802.11 support #device an # Aironet 4500/4800 802.11 wireless NICs. #device awi # BayStack 660 and others #device wi # WaveLAN/Intersil/Symbol 802.11 wireless NICs. #device wl # Older non 802.11 Wavelan wireless NIC. # Pseudo devices. device loop # Network loopback #device mem # Memory and kernel memory devices #device io # I/O device #device random # Entropy device device ether # Ethernet support #device sl # Kernel SLIP #device ppp # Kernel PPP #device tun # Packet tunnel. device pty # Pseudo-ttys (telnet etc) #device md # Memory "disks" #device gif # IPv6 and IPv4 tunneling #device faith # IPv6-to-IPv4 relaying (translation) # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! device bpf # Berkeley packet filter # USB support #device uhci # UHCI PCI->USB interface #device ohci # OHCI PCI->USB interface #device usb # USB Bus (required) #device udbp # USB Double Bulk Pipe devices #device ugen # Generic #device uhid # "Human Interface Devices" #device ukbd # Keyboard #device ulpt # Printer #device umass # Disks/Mass storage - Requires scbus and da #device ums # Mouse #device urio # Diamond Rio 500 MP3 player #device uscanner # Scanners # USB Ethernet, requires mii #device aue # ADMtek USB Ethernet #device axe # ASIX Electronics USB Ethernet #device cue # CATC USB Ethernet #device kue # Kawasaki LSI USB Ethernet #device rue # RealTek RTL8150 USB Ethernet # FireWire support #device firewire # FireWire bus code #device sbp # SCSI over FireWire (Requires scbus and da) #device fwe # Ethernet over FireWire (non-standard!) # /etc/libmap.conf # # candidate mapping # #libc_r.so.5 libpthread.so.2 # Everything uses 'libpthread' #libc_r.so libpthread.so libpthread.so.2 libthr.so.2 libpthread.so libthr.so libc_r.so.5 libthr.so.2 # Everything uses 'libthr' libc_r.so libthr.so [/usr/local/lib/linux-flashplugin6/libflashplayer.so] libpthread.so.0 pluginwrapper/flash6.so libdl.so.2 pluginwrapper/flash6.so libz.so.1 libz.so.2 libstdc++-libc6.2-2.so.3 libstdc++.so.4 libm.so.6 libm.so.2 libc.so.6 pluginwrapper/flash6.so www/apache20|22 is built with following flags www/apache22|WITH_CUSTOM_PROXY="proxy proxy_http" WITH_MPM=worker WITH_KQUEUE_SUPPORT=yes WITH_THREADS=yes WITH_BERKELEYDB=db42 WITH_KQUEUE_SUPPORT=yes WITH_SUEXEC=yes| >How-To-Repeat: >Fix: >Release-Note: >Audit-Trail: >Unformatted: From owner-freebsd-threads@FreeBSD.ORG Wed Dec 14 16:30:05 2005 Return-Path: X-Original-To: freebsd-threads@hub.freebsd.org Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3E34016A41F for ; Wed, 14 Dec 2005 16:30:05 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 935A543D67 for ; Wed, 14 Dec 2005 16:30:04 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id jBEGU4LL074204 for ; Wed, 14 Dec 2005 16:30:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id jBEGU4kT074202; Wed, 14 Dec 2005 16:30:04 GMT (envelope-from gnats) Resent-Date: Wed, 14 Dec 2005 16:30:04 GMT Resent-Message-Id: <200512141630.jBEGU4kT074202@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-threads@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Daniel Hartmeier Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A785316A41F; Wed, 14 Dec 2005 16:27:46 +0000 (GMT) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (insomnia.benzedrine.cx [62.65.145.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id 65EB543D4C; Wed, 14 Dec 2005 16:27:45 +0000 (GMT) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (dhartmei@localhost [127.0.0.1]) by insomnia.benzedrine.cx (8.13.4/8.12.11) with ESMTP id jBEGRjac014476 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Wed, 14 Dec 2005 17:27:45 +0100 (MET) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.13.4/8.12.10/Submit) id jBEGRjp0024926; Wed, 14 Dec 2005 17:27:45 +0100 (MET) Message-Id: <20051214162744.GI17140@insomnia.benzedrine.cx> Date: Wed, 14 Dec 2005 17:27:44 +0100 From: Daniel Hartmeier To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Cc: Daniel Eischen Subject: threads/90392: libc stdio memory leak with pthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Hartmeier List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2005 16:30:05 -0000 >Number: 90392 >Category: threads >Synopsis: libc stdio memory leak with pthread >Confidential: no >Severity: serious >Priority: low >Responsible: freebsd-threads >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Wed Dec 14 16:30:03 GMT 2005 >Closed-Date: >Last-Modified: >Originator: Daniel Hartmeier >Release: FreeBSD 6.0-STABLE i386 >Organization: >Environment: >Description: Calling sscanf(3) with a '%c' conversion from a multi-threaded (-lpthread) program leaks memory. >How-To-Repeat: Compile and run the following minimal example and observe the leak, e.g. with top(1) $ gcc -o test test.c -lpthread #include #include #include static void *leak(void *p) { const char *s = "foo"; char t[4]; while (1) { (void)sscanf(s, "%3c", t); usleep(1000); } return (NULL); } int main() { pthread_t t; int r; if ((r = pthread_create(&t, NULL, &leak, NULL))) return (1); while (1) usleep(1000); return (0); } >Fix: See /usr/src/lib/libc/stdio/ sscanf.c sscanf() sets up a private FILE object, initializes the thread locking extra with INITEXTRA(), and passes it to __svfscanf(). vfscanf.c __svfscanf() enters case CT_CHAR: with neither LONG nor SUPPRESS flags set, and calls fread(). fread.c fread() uses FLOCKFILE() and FUNLOCKFILE(), which in this case are _flockfile() and _funlockfile(). _flock_stub.c _flockfile() calls _pthread_mutex_lock(2) on the mutex in the extra passed from sscanf(). The mutex was initialized using PTHREAD_MUTEX_INITIALIZER through INITEXTRA(). While it appears to be valid to use the mutex after such initialization (without a pthread_create(2) call), that usage causes delayed initialization which requires a call to pthread_mutex_destroy(2) afterwards. The following patch fixes the leak for me: --- sscanf.c.orig Wed Dec 14 17:16:18 2005 +++ sscanf.c Wed Dec 14 17:16:36 2005 @@ -78,5 +78,6 @@ va_start(ap, fmt); ret = __svfscanf(&f, fmt, ap); va_end(ap); + _pthread_mutex_destroy(&f._extra->fl_mutex); return (ret); } If you grep for other INITEXTRA() calls, those might require a similar fix, possibly justifying an additional macro like FREEEXTRA(). I'm not familiar enough with libc's FILE extra structure nor with pthread_mutex semantics to be perfectly sure. >Release-Note: >Audit-Trail: >Unformatted: From owner-freebsd-threads@FreeBSD.ORG Wed Dec 14 20:49:28 2005 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 64E7E16A420 for ; Wed, 14 Dec 2005 20:49:28 +0000 (GMT) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (insomnia.benzedrine.cx [62.65.145.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1503F43D9D for ; Wed, 14 Dec 2005 20:49:07 +0000 (GMT) (envelope-from dhartmei@insomnia.benzedrine.cx) Received: from insomnia.benzedrine.cx (dhartmei@localhost [127.0.0.1]) by insomnia.benzedrine.cx (8.13.4/8.12.11) with ESMTP id jBEKn10X022972 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO) for ; Wed, 14 Dec 2005 21:49:01 +0100 (MET) Received: (from dhartmei@localhost) by insomnia.benzedrine.cx (8.13.4/8.12.10/Submit) id jBEKn0Ex005096 for freebsd-threads@freebsd.org; Wed, 14 Dec 2005 21:49:00 +0100 (MET) Date: Wed, 14 Dec 2005 21:48:59 +0100 From: Daniel Hartmeier To: freebsd-threads@freebsd.org Message-ID: <20051214204859.GJ17140@insomnia.benzedrine.cx> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20051214162744.GI17140@insomnia.benzedrine.cx> User-Agent: Mutt/1.5.10i Subject: Re: threads/90392: libc stdio memory leak with pthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2005 20:49:28 -0000 It's actually a waste to lock the mutex and destroy it again, since it's really private to sscanf() and only provided as part of the API and not because it serves a real locking purpose in this case. The patch below splits fread() into a MT-safe and Non-MT-safe version, so __svfscanf() can call the latter. This is in the spirit of how many other stdio functions were split, for instance see the diff and commit message of rev. 1.8 of ungetc.c http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc/stdio/ungetc.c With this, sscanf() never triggers a mutex allocation or lock, and there's no memory leak due to lacking pthread_mutex_destroy(). I checked all the other uses of INITEXTRA(), and none of them share this problem, since in those cases only the non-locking functions (with prefix __) are called. There might some risk of someone later introducing calls to locking functions, but that fread() call in __svfscanf() was really the only offender I can spot. You could add the pthread_mutex_destroy() as suggested in the first patch additionally, to cover future mistakes. Daniel --- local.h.orig Wed Dec 14 20:29:56 2005 +++ local.h Wed Dec 14 20:31:44 2005 @@ -79,6 +79,8 @@ extern int __vfwprintf(FILE *, const wchar_t *, __va_list); extern int __vfwscanf(FILE * __restrict, const wchar_t * __restrict, __va_list); +extern size_t __fread(void * __restrict buf, size_t size, size_t count, + FILE * __restrict fp); extern int __sdidinit; --- fread.c.orig Wed Dec 14 20:22:31 2005 +++ fread.c Wed Dec 14 20:29:11 2005 @@ -47,11 +47,25 @@ #include "local.h" #include "libc_private.h" +/* + * MT-safe version + */ size_t -fread(buf, size, count, fp) - void * __restrict buf; - size_t size, count; - FILE * __restrict fp; +fread(void * __restrict buf, size_t size, size_t count, FILE * __restrict fp) +{ + int ret; + + FLOCKFILE(fp); + ret = __fread(buf, size, count, fp); + FUNLOCKFILE(fp); + return (ret); +} + +/* + * Non-MT-safe version + */ +size_t +__fread(void * __restrict buf, size_t size, size_t count, FILE * __restrict fp) { size_t resid; char *p; @@ -65,7 +79,6 @@ */ if ((resid = count * size) == 0) return (0); - FLOCKFILE(fp); ORIENT(fp, -1); if (fp->_r < 0) fp->_r = 0; @@ -79,13 +92,11 @@ resid -= r; if (__srefill(fp)) { /* no more input: return partial result */ - FUNLOCKFILE(fp); return ((total - resid) / size); } } (void)memcpy((void *)p, (void *)fp->_p, resid); fp->_r -= resid; fp->_p += resid; - FUNLOCKFILE(fp); return (count); } --- vfscanf.c.orig Wed Dec 14 20:22:53 2005 +++ vfscanf.c Wed Dec 14 20:32:08 2005 @@ -412,7 +412,7 @@ } nread += sum; } else { - size_t r = fread((void *)va_arg(ap, char *), 1, + size_t r = __fread((void *)va_arg(ap, char *), 1, width, fp); if (r == 0) From owner-freebsd-threads@FreeBSD.ORG Thu Dec 15 09:50:09 2005 Return-Path: X-Original-To: freebsd-threads@hub.freebsd.org Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2060A16A41F for ; Thu, 15 Dec 2005 09:50:09 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2E4D743D66 for ; Thu, 15 Dec 2005 09:50:04 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id jBF9o4LV039281 for ; Thu, 15 Dec 2005 09:50:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id jBF9o46W039280; Thu, 15 Dec 2005 09:50:04 GMT (envelope-from gnats) Date: Thu, 15 Dec 2005 09:50:04 GMT Message-Id: <200512150950.jBF9o46W039280@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: David Xu Cc: Subject: Re: threads/90392: libc stdio memory leak with pthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: David Xu List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2005 09:50:09 -0000 The following reply was made to PR threads/90392; it has been noted by GNATS. From: David Xu To: bug-followup@freebsd.org, dhartmei@freebsd.org Cc: Subject: Re: threads/90392: libc stdio memory leak with pthread Date: Thu, 15 Dec 2005 17:45:11 +0800 This definitely is a bug in libc, in fact, not only memory leak, but also deadlock. Functions like fscanf, vfscanf will deadlock itself in threaded program, because __svfscanf() calls fread() which will recursively lock the FILE itself! Try following in threaded program: fscanf(stdin, s, "%3c", t); It should deadlock itself. I am fixing it now. David Xu From owner-freebsd-threads@FreeBSD.ORG Thu Dec 15 10:10:10 2005 Return-Path: X-Original-To: freebsd-threads@hub.freebsd.org Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A0D4116A41F for ; Thu, 15 Dec 2005 10:10:10 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id A55C943D6A for ; Thu, 15 Dec 2005 10:10:06 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id jBFAA6ln039963 for ; Thu, 15 Dec 2005 10:10:06 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id jBFAA6MZ039962; Thu, 15 Dec 2005 10:10:06 GMT (envelope-from gnats) Date: Thu, 15 Dec 2005 10:10:06 GMT Message-Id: <200512151010.jBFAA6MZ039962@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: David Xu Cc: Subject: Re: threads/90392: libc stdio memory leak with pthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: David Xu List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2005 10:10:10 -0000 The following reply was made to PR threads/90392; it has been noted by GNATS. From: David Xu To: bug-followup@freebsd.org, dhartmei@freebsd.org Cc: Subject: Re: threads/90392: libc stdio memory leak with pthread Date: Thu, 15 Dec 2005 18:04:02 +0800 It does not deadlock, as I found the FILELOCK is a recursive lock. ;-) David Xu From owner-freebsd-threads@FreeBSD.ORG Thu Dec 15 17:22:22 2005 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 81DEE16A420; Thu, 15 Dec 2005 17:22:22 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3472D43DA7; Thu, 15 Dec 2005 17:22:05 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.13.5/8.13.5/NETPLEX) with ESMTP id jBFHM0bO005638; Thu, 15 Dec 2005 12:22:00 -0500 (EST) Date: Thu, 15 Dec 2005 12:22:00 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: David Xu In-Reply-To: <200512151010.jBFAA6MZ039962@freefall.freebsd.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) Cc: freebsd-threads@freebsd.org Subject: Re: threads/90392: libc stdio memory leak with pthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2005 17:22:22 -0000 On Thu, 15 Dec 2005, David Xu wrote: > The following reply was made to PR threads/90392; it has been noted by GNATS. > > From: David Xu > To: bug-followup@freebsd.org, dhartmei@freebsd.org > Cc: > Subject: Re: threads/90392: libc stdio memory leak with pthread > Date: Thu, 15 Dec 2005 18:04:02 +0800 > > It does not deadlock, as I found the FILELOCK is a recursive lock. ;-) Still, we shouldn't use recursive locks in our implementation of libc. There really is no need to. If you can fix it, please do so :-) -- DE