From owner-freebsd-current@freebsd.org Sun Sep 8 22:03:13 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E5F63E4C9D for ; Sun, 8 Sep 2019 22:03:13 +0000 (UTC) (envelope-from clhamilto@gmail.com) Received: from mail-qt1-x82c.google.com (mail-qt1-x82c.google.com [IPv6:2607:f8b0:4864:20::82c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46RQKD0wP9z4BS0 for ; Sun, 8 Sep 2019 22:03:11 +0000 (UTC) (envelope-from clhamilto@gmail.com) Received: by mail-qt1-x82c.google.com with SMTP id l22so13872902qtp.10 for ; Sun, 08 Sep 2019 15:03:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:references:to:from:message-id:date:user-agent:mime-version :in-reply-to:content-language; bh=m4Sra4RNGt8sikLxxC6a17KbXN6WPyDa/itdJkosKSc=; b=OGWSFg6avt3L0rZi12OJRpXR9oAdGy9cdPURIFzeW0O6AtXDHUKjgz7FD4We2VsGnr ZS40coktkWjqzVZnxuCtIROB8dycpuEJ7NCHe5RwP5l024SnH/PHGa51tar+tsJbsL6P AayvRNk2ibVN336kShsQM2W7pvOz5/7HnpEjLaj5eMD4itkY51w3cCxPFJg2SR+iNOiI +JBnp01F/cSmMk+WVL3CMk/U8ZqIZ26Jp4WrRTgOrUvvqReBkFtE1giPmgWe4RFNANSs WvfC2wxftyvTZtjj31/ML4QxMG6VnPRmbm3NamZhJHsat/n/Izo9ObZ1uW1j0A6qKMT8 52Ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:references:to:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=m4Sra4RNGt8sikLxxC6a17KbXN6WPyDa/itdJkosKSc=; b=m6ZM/F85oGz9xvJ3VDaLQK1FL8eaAhevRpd8HC2I6lXtCEzQr3vyUTBziN1txmqvh1 H9wVXaYdVHACK1LEIa/cMTbTikj2y4jbZS45KoDgW9WbNOZ6URiegiiOVR/nG746F2uh DOPhn8Oj2TK48b1XYwf5E/NljHz1zk8fUMUxxK2J1uILuNwSd9YIzaXHDbgZc+GJlN2H ePqRbeKcu4reUNg+pVoHjf0AJkZVXGHAuQvm0F5/iFNZZet7UgN0ITEGo/it/FpHkRj1 noRUCcDmuRdTWmVAzGqX2+CkgQPkBFvIN9EKJgbAjSuccbtuxhnuMty+yj0nNAXn/4al Q7+Q== X-Gm-Message-State: APjAAAUQxEFr6LsIunnOUUIzIfcuKPuk/OFtwaZP58WDzogJe9FEJFYQ p1VvRUJW7/k2teos5WhE8alZGzka X-Google-Smtp-Source: APXvYqxMLImzgazZBT9h5E56ffu794avfGNwCjf3m8DDfamQCr9PiZbdIaCmFLMwv04bKr+mt9Gv9g== X-Received: by 2002:ac8:5357:: with SMTP id d23mr12246069qto.266.1567980191041; Sun, 08 Sep 2019 15:03:11 -0700 (PDT) Received: from lenoil8.lenoil.net (c-76-106-45-221.hsd1.va.comcast.net. [76.106.45.221]) by smtp.gmail.com with ESMTPSA id 60sm6044382qta.77.2019.09.08.15.03.10 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 08 Sep 2019 15:03:10 -0700 (PDT) Subject: Panic with Current 351901 ISO image References: To: freebsd-current@freebsd.org From: Curtis Hamilton X-Forwarded-Message-Id: Message-ID: <01378a14-471d-31b4-c60a-98a7a059748b@gmail.com> Date: Sun, 8 Sep 2019 18:03:09 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD powerpc; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Rspamd-Queue-Id: 46RQKD0wP9z4BS0 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=OGWSFg6a; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of clhamilto@gmail.com designates 2607:f8b0:4864:20::82c as permitted sender) smtp.mailfrom=clhamilto@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RECEIVED_SPAMHAUS_PBL(0.00)[221.45.106.76.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(0.00)[ip: (-9.28), ipnet: 2607:f8b0::/32(-2.75), asn: 15169(-2.27), country: US(-0.05)]; RCVD_IN_DNSWL_NONE(0.00)[c.2.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Sep 2019 22:03:14 -0000 I'm encountering (randomly) the below error when trying to install 351901 from CD/DVD. timeout stopping cpus panic: acquiring blockable sleep lock with spinlock or critical section held (sleep mutex) vm map (system) @ /usr/src/sys/vm/vm_map.c:4066 cpuid = 0 time = 1 KDB: stack backtrace: 0xe00000008879d4f0: at .kdb_backtrace+0x5c 0xe00000008879d620: at .vpanic+0x1b4 0xe00000008879d6e0: at .panic+0x38 0xe00000008879d770: at .witness_checkorder+0xf4 0xe00000008879d860: at .__mtx_lock_flags+0xfc 0xe00000008879d910: at ._vm_map_lock_read+0x34 0xe00000008879d990: at .vm_map_lookup+0x94 0xe00000008879dad0: at .vm_fault_hold+0x158 0xe00000008879dcf0: at .vm_fault+0x94 0xe00000008879dda0: at .cpu_fetch_syscall_args+0x404 0xe00000008879de50: at .trap+0xf28 0xe00000008879e010: at .powerpc_interrupt+0x290 0xe00000008879e0b0: kernel DSI read trap @ 0x392 by .sched_relinquish+0x290: srr1=0x9000000000001032 r1=0xe00000008879e360 cr=0x20009032 xer=0 ctr=0xc000000002ea198c r2=0xc00000000393f7e0 sr=0x40000000 0xe00000008879e360: at .sched_tdname+0xbc 0xe00000008879e3f0: at .sched_switch+0x414 0xe00000008879e530: at .mi_switch+0x2d4 0xe00000008879e5c0: at .sched_bind+0xd8 0xe00000008879e650: at .taskqgroup_config_gtask_init+0x108 0xe00000008879e700: at .gtaskqueue_cancel+0x2ac 0xe00000008879e7c0: at .taskqgroup_adjust+0xb0c 0xe00000008879e850: at .fork_exit+0xd0 0xe00000008879e8f0: at .fork_trampoline+0x10 0xe00000008879e920: at -0x4 KDB: enter: panic [ thread pid 0 tid 100144 ] Stopped at .kdb_enter+0x60:        ld      r2, r1, 0x28 db> From owner-freebsd-current@freebsd.org Sun Sep 8 22:59:57 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 28B8DE6117 for ; Sun, 8 Sep 2019 22:59:57 +0000 (UTC) (envelope-from clay.daniels.jr@gmail.com) Received: from mail-vk1-xa2d.google.com (mail-vk1-xa2d.google.com [IPv6:2607:f8b0:4864:20::a2d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46RRZh0zvRz4DDM for ; Sun, 8 Sep 2019 22:59:55 +0000 (UTC) (envelope-from clay.daniels.jr@gmail.com) Received: by mail-vk1-xa2d.google.com with SMTP id t136so2335231vkt.9 for ; Sun, 08 Sep 2019 15:59:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=1gjYTLptPfjmAxzBg3/wHMYmTZrxSbmmoXk1938i4Q0=; b=aYRf9+NyW6/0IZ8AUYSBiWUszGDwYuuHPAFjEDgxHEqj7FJRxURGFwVMk+y+zj/BpJ YZKVGxQzHpSjBtioMhRRfqOIVfdq9xGTbDMP3GiJ6AT5+fhlzBf4eX89DjN7a6HqYU9c 7HZtZB/IdYNIjUzSsbs7HO5YjFnWG4kCbWL65EMAA5RXo8jbKVoViYNOAbpFaa+FW+Ai f0z3SlvYRRZX3v6kk3S0HXvq3rqBuJdrh/o3r/PNcrGz0le0bPQTemP1QIP0CQc3z5PW hpxI6XYocM3PjhETqGxSqsRVwb73dijQnInvUWyinqusy8vBvq3w4pYOl9IWr7Evwnu9 fdKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=1gjYTLptPfjmAxzBg3/wHMYmTZrxSbmmoXk1938i4Q0=; b=fqOp0/irDy6tK4BlNMJYNfRY4vdO4IOfbESkenYBXZTg+Ufd+Wd3y0O9f6jGgNVVMa +yyW3XBksBSUbEC/iwTXEW6DJBO9K361/DxHvwNKUUZCgePFUtXRKdQkuPTRajMGafe9 g1TJX8cH4sy1uuuo1oHclW3cn5RQEXE5mS9VRnkS2dQTZcNVzd8LjoeSc+f1UPQi46jt 9rjYedMbsuVMIpMHR1MkDLcG2Yhjdrzw4vD+vk63qgtcTiH0mid7dPC/7Zar3YKG7yXO 42QUwNgxX3RLLEGiV2SJ2CnnoJE+p2xZY2F6n/DnMaEzLMNMtgU3omhrQDS0WvW0GWXC NSIA== X-Gm-Message-State: APjAAAWDG9NTHHYpalsoMSrU5V26vTq+NUp3f/6EL+8RCwlkC8wBT5g7 NzEnzGjitCXAWvPoX7hcdsHO2xBIFSCfXZSw0K8A5Ow= X-Google-Smtp-Source: APXvYqwIeSLSlGPha7T9K9Zxpg2T2XlP6r8SYPYxwb9IHLhIcV8K1h7onUhY8fP05cIlpLMB5tCZ2CCWwE3Gs1iK3Qs= X-Received: by 2002:a1f:78c5:: with SMTP id t188mr9490715vkc.87.1567983593979; Sun, 08 Sep 2019 15:59:53 -0700 (PDT) MIME-Version: 1.0 From: "Clay Daniels Jr." Date: Sun, 8 Sep 2019 17:59:41 -0500 Message-ID: Subject: fusefs & ntfs-3g To: "freebsd-current@freebsd.org" X-Rspamd-Queue-Id: 46RRZh0zvRz4DDM X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=aYRf9+Ny; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of claydanielsjr@gmail.com designates 2607:f8b0:4864:20::a2d as permitted sender) smtp.mailfrom=claydanielsjr@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; URI_COUNT_ODD(1.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (-9.25), ipnet: 2607:f8b0::/32(-2.75), asn: 15169(-2.27), country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[d.2.a.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; TO_DN_EQ_ADDR_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Sep 2019 22:59:57 -0000 I want to view my Windows C: drive on ata0p4. This is what I have: FreeBSD bsd13 13.0-CURRENT FreeBSD 13.0-CURRENT r351901 GENERIC amd64 clay@bsd13:~ % pkg info -r fusefs-libs fusefs-libs-2.9.9: clay@bsd13:~ % pkg info -r fusefs-libs3 fusefs-libs3-3.6.2: clay@bsd13:~ % cat /boot/loader.conf hw.syscons.disable=1 fusefs_load="YES" clay@bsd13:~ % kldstat Id Refs Address Size Name 1 72 0xffffffff80200000 2336f80 kernel 2 1 0xffffffff82538000 27990 fusefs.ko 3 1 0xffffffff82720000 2519c4 amdgpu.ko 4 2 0xffffffff82972000 77bd0 drm.ko 5 5 0xffffffff829ea000 12470 linuxkpi.ko 6 3 0xffffffff829fd000 12f30 linuxkpi_gplv2.ko 7 2 0xffffffff82a10000 8e0 lindebugfs.ko 8 1 0xffffffff82a11000 f281 ttm.ko 9 1 0xffffffff82a21000 23f7 radeon_kabini_pfp_bin.ko 10 1 0xffffffff82a24000 23f5 radeon_kabini_me_bin.ko 11 1 0xffffffff82a27000 23f5 radeon_kabini_ce_bin.ko 12 1 0xffffffff82a2a000 43f7 radeon_kabini_mec_bin.ko 13 1 0xffffffff82a2f000 2a77 radeon_kabini_rlc_bin.ko 14 1 0xffffffff82a32000 12e9 radeon_kabini_sdma_bin.ko 15 1 0xffffffff82a34000 12eb radeon_kabini_sdma1_bin.ko 16 1 0xffffffff82a36000 38ea7 radeon_kabini_uvd_bin.ko 17 1 0xffffffff82a6f000 18c47 radeon_kabini_vce_bin.ko 18 1 0xffffffff82a88000 2538 intpm.ko 19 1 0xffffffff82a8b000 a50 smbus.ko 20 1 0xffffffff82a8c000 1820 uhid.ko 21 1 0xffffffff82a8e000 2928 ums.ko 22 1 0xffffffff82a91000 1b00 wmt.ko 23 1 0xffffffff82a93000 acf mac_ntpd.ko 24 1 0xffffffff82a94000 a9f8 tmpfs.ko clay@bsd13:~ % pkg info fusefs-ntfs-2017.3.23 fusefs-ntfs-2017.3.23 Name : fusefs-ntfs Version : 2017.3.23 Installed on : Sun Sep 8 17:24:36 2019 CDT Origin : sysutils/fusefs-ntfs Architecture : FreeBSD:13:amd64 Prefix : /usr/local Categories : sysutils Licenses : GPLv2+ Maintainer : freebsd@dussan.org WWW : https://www.tuxera.com/community/open-source-ntfs-3g/ Comment : Mount NTFS partitions (read/write) and disk images Options : DOCS : on LOCK : on UBLIO : on Shared Libs required: libfuse.so.2 libuuid.so.1 libublio.so.1 Shared Libs provided: libntfs-3g.so.88 Annotations : FreeBSD_version: 1300044 repo_type : binary repository : FreeBSD Flat size : 1.93MiB Description : The ntfs-3g driver is an open source, freely available read/write NTFS driver, which provides safe and fast handling of the Windows XP, Windows Server 2003, Windows 2000, Windows Vista, Windows Server 2008, Windows 7 and Windows 8 NTFS file systems. Almost the full POSIX filesystem functionality is supported, the major exceptions are changing the file ownerships and the access rights. root@bsd13:/home/clay # ntfs-3g -o default_permissons /dev/ada0p4 /mnt * Windows is hibernated, refused to mount. The disk contains an unclean file system (0, 0). Metadata kept in Windows cache, refused to mount. Falling back to read-only mount because the NTFS partition is in an unsafe state. Please resume and shutdown Windows fully (no hibernation or fast restarting. I'm having a roadblock here. Maybe someone knows the answer. From owner-freebsd-current@freebsd.org Sun Sep 8 23:10:01 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5DCFFE6587 for ; Sun, 8 Sep 2019 23:10:01 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46RRpJ2dzyz4DhG for ; Sun, 8 Sep 2019 23:10:00 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf1-f42.google.com with SMTP id w6so9025830lfl.2 for ; Sun, 08 Sep 2019 16:10:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=iU5nryxKj0AfQ2MVWdRu/mZ43Zn9UoK0XQIJAazpRnU=; b=WFirmc4Vim8kKyj93wDyfNH4U5/bRpkLNxhaK8+6oWMtqwzOSitWe7paks8bV67r1t 8aeJYIGpXVvD6qQL8InVyrX97Fkt+3tWGz55Nx691B+RxFW5Ie6A8SeuBGOG07aPYc9w pfabyJDu3FjCp9VyQXK5EBG2uVLdx+T9UK2wJMBK/nKInqmzVUqgZ/PXGEw8Aq28It6c ZNYwIbHsZSdI3Rw4YIom1yxA/VVlD52z4NnoQB4bdieYAIbLF0yzeOS36IQGkn9UsIyh xH8epYooFIlshDzAEeM1n03QgpEgq4TR8x4P7iN/URJzhIrQSNHSHdUZ0WwN+6HM2inb ef1Q== X-Gm-Message-State: APjAAAVVvM46lhnrVjGVuHhMSw5eycQV662Tg48RxpfGNQGVjkP6GiIM ins8MBdfiMVWm5B1Ba3q/D/dBJpmrtTHK/Cr+TmJow== X-Google-Smtp-Source: APXvYqxNYkOzBKZxvHslnIQGiUdzoU2UDfEhvGV/x69tG7xOW5XnaLIIpY5eoQSjQU3zuS6S0ytOSXv4uM1YDLAsvDk= X-Received: by 2002:ac2:4352:: with SMTP id o18mr14487042lfl.164.1567984198554; Sun, 08 Sep 2019 16:09:58 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Sun, 8 Sep 2019 17:09:46 -0600 Message-ID: Subject: Re: fusefs & ntfs-3g To: "Clay Daniels Jr." Cc: "freebsd-current@freebsd.org" , freebsd@dussan.org X-Rspamd-Queue-Id: 46RRpJ2dzyz4DhG X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.42 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-2.24 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; URI_COUNT_ODD(1.00)[3]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[42.167.85.209.list.dnswl.org : 127.0.5.0]; TO_DN_SOME(0.00)[]; IP_SCORE(-1.24)[ip: (-0.55), ipnet: 209.85.128.0/17(-3.32), asn: 15169(-2.27), country: US(-0.05)]; FREEMAIL_TO(0.00)[gmail.com]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Sep 2019 23:10:01 -0000 On Sun, Sep 8, 2019 at 5:00 PM Clay Daniels Jr. wrote: > I want to view my Windows C: drive on ata0p4. > This is what I have: > FreeBSD bsd13 13.0-CURRENT FreeBSD 13.0-CURRENT r351901 GENERIC amd64 > clay@bsd13:~ % pkg info -r fusefs-libs > fusefs-libs-2.9.9: > clay@bsd13:~ % pkg info -r fusefs-libs3 > fusefs-libs3-3.6.2: > clay@bsd13:~ % cat /boot/loader.conf > hw.syscons.disable=1 > fusefs_load="YES" > clay@bsd13:~ % kldstat > Id Refs Address Size Name > 1 72 0xffffffff80200000 2336f80 kernel > 2 1 0xffffffff82538000 27990 fusefs.ko > 3 1 0xffffffff82720000 2519c4 amdgpu.ko > 4 2 0xffffffff82972000 77bd0 drm.ko > 5 5 0xffffffff829ea000 12470 linuxkpi.ko > 6 3 0xffffffff829fd000 12f30 linuxkpi_gplv2.ko > 7 2 0xffffffff82a10000 8e0 lindebugfs.ko > 8 1 0xffffffff82a11000 f281 ttm.ko > 9 1 0xffffffff82a21000 23f7 radeon_kabini_pfp_bin.ko > 10 1 0xffffffff82a24000 23f5 radeon_kabini_me_bin.ko > 11 1 0xffffffff82a27000 23f5 radeon_kabini_ce_bin.ko > 12 1 0xffffffff82a2a000 43f7 radeon_kabini_mec_bin.ko > 13 1 0xffffffff82a2f000 2a77 radeon_kabini_rlc_bin.ko > 14 1 0xffffffff82a32000 12e9 radeon_kabini_sdma_bin.ko > 15 1 0xffffffff82a34000 12eb radeon_kabini_sdma1_bin.ko > 16 1 0xffffffff82a36000 38ea7 radeon_kabini_uvd_bin.ko > 17 1 0xffffffff82a6f000 18c47 radeon_kabini_vce_bin.ko > 18 1 0xffffffff82a88000 2538 intpm.ko > 19 1 0xffffffff82a8b000 a50 smbus.ko > 20 1 0xffffffff82a8c000 1820 uhid.ko > 21 1 0xffffffff82a8e000 2928 ums.ko > 22 1 0xffffffff82a91000 1b00 wmt.ko > 23 1 0xffffffff82a93000 acf mac_ntpd.ko > 24 1 0xffffffff82a94000 a9f8 tmpfs.ko > > clay@bsd13:~ % pkg info fusefs-ntfs-2017.3.23 > fusefs-ntfs-2017.3.23 > Name : fusefs-ntfs > Version : 2017.3.23 > Installed on : Sun Sep 8 17:24:36 2019 CDT > Origin : sysutils/fusefs-ntfs > Architecture : FreeBSD:13:amd64 > Prefix : /usr/local > Categories : sysutils > Licenses : GPLv2+ > Maintainer : freebsd@dussan.org > WWW : https://www.tuxera.com/community/open-source-ntfs-3g/ > Comment : Mount NTFS partitions (read/write) and disk images > Options : > DOCS : on > LOCK : on > UBLIO : on > Shared Libs required: > libfuse.so.2 > libuuid.so.1 > libublio.so.1 > Shared Libs provided: > libntfs-3g.so.88 > Annotations : > FreeBSD_version: 1300044 > repo_type : binary > repository : FreeBSD > Flat size : 1.93MiB > Description : > The ntfs-3g driver is an open source, freely available read/write NTFS > driver, which provides safe and fast handling of the Windows XP, Windows > Server 2003, Windows 2000, Windows Vista, Windows Server 2008, Windows 7 > and Windows 8 NTFS file systems. Almost the full POSIX filesystem > functionality is supported, the major exceptions are changing the file > ownerships and the access rights. > > root@bsd13:/home/clay # ntfs-3g -o default_permissons /dev/ada0p4 /mnt > * Windows is hibernated, refused to mount. > The disk contains an unclean file system (0, 0). > Metadata kept in Windows cache, refused to mount. > Falling back to read-only mount because the NTFS partition is in an unsafe > state. Please resume and shutdown Windows fully (no hibernation > or fast restarting. > > I'm having a roadblock here. Maybe someone knows the answer. > I'm assuming that you already ascertained that the Windows file system was in fact clean? If so, then this sounds like a problem with fusefs-ntfs, not with fuse in general. I've CC'd its maintainer. He may be able to help you. -Alan From owner-freebsd-current@freebsd.org Sun Sep 8 23:46:29 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D40E8E7031 for ; Sun, 8 Sep 2019 23:46:29 +0000 (UTC) (envelope-from imb@protected-networks.net) Received: from sarah.protected-networks.net (sarah.protected-networks.net [IPv6:2001:470:1f07:4e1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (1024 bits) client-digest SHA256) (Client CN "sarah.protected-networks.net", Issuer "Protected Networks CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46RScP52gwz4Fs3; Sun, 8 Sep 2019 23:46:29 +0000 (UTC) (envelope-from imb@protected-networks.net) Received: from mail.protected-networks.net (mail.protected-networks.net [202.12.127.228]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (1024 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.protected-networks.net", Issuer "Protected Networks CA" (verified OK)) by sarah.protected-networks.net (Postfix) with ESMTPS id E98E2D79A1; Sun, 8 Sep 2019 19:36:20 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= protected-networks.net; h=content-transfer-encoding :content-language:content-type:content-type:in-reply-to :mime-version:user-agent:date:date:message-id:from:from :references:subject:subject; s=201508; t=1567985765; bh=fN4bXXCL Sftnq6D/1d1gOxU4oCjWFZCxM1cYLnxpjYM=; b=gC0A7Qvk42ejZpC3sQqE0mk/ XVCgXPQGk7gzMo1vT+fjzntA09zodT5WxERMel532A1ZHbQ86yxm6rdMR7MctK0t 0anFk6Pj5b59rJtwBkU8HCAiyEChKRDIR0gsXwoQtLtmBChjt4eKMZh2H3CGjmBu fFfes3Ow0fuiThnB2Ok= Received: from toshi.auburn.protected-networks.net (toshi.auburn.protected-networks.net [192.168.1.10]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: imb@mail.protected-networks.net) by mail.protected-networks.net (Postfix) with ESMTPSA id B8C3D2DFA3; Sun, 8 Sep 2019 19:36:05 -0400 (EDT) Subject: Re: fusefs & ntfs-3g To: Alan Somers , "Clay Daniels Jr." Cc: "freebsd-current@freebsd.org" , freebsd@dussan.org References: From: Michael Butler Message-ID: Date: Sun, 8 Sep 2019 19:36:05 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.1.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Language: en-NZ Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46RScP52gwz4Fs3 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; TAGGED_RCPT(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Sep 2019 23:46:29 -0000 On 9/8/19 7:09 PM, Alan Somers wrote: > On Sun, Sep 8, 2019 at 5:00 PM Clay Daniels Jr. > wrote: > >> I want to view my Windows C: drive on ata0p4. >> This is what I have: >> FreeBSD bsd13 13.0-CURRENT FreeBSD 13.0-CURRENT r351901 GENERIC amd64 >> clay@bsd13:~ % pkg info -r fusefs-libs >> fusefs-libs-2.9.9: >> clay@bsd13:~ % pkg info -r fusefs-libs3 >> fusefs-libs3-3.6.2: >> clay@bsd13:~ % cat /boot/loader.conf >> hw.syscons.disable=1 >> fusefs_load="YES" >> clay@bsd13:~ % kldstat >> Id Refs Address Size Name >> 1 72 0xffffffff80200000 2336f80 kernel >> 2 1 0xffffffff82538000 27990 fusefs.ko >> 3 1 0xffffffff82720000 2519c4 amdgpu.ko >> 4 2 0xffffffff82972000 77bd0 drm.ko >> 5 5 0xffffffff829ea000 12470 linuxkpi.ko >> 6 3 0xffffffff829fd000 12f30 linuxkpi_gplv2.ko >> 7 2 0xffffffff82a10000 8e0 lindebugfs.ko >> 8 1 0xffffffff82a11000 f281 ttm.ko >> 9 1 0xffffffff82a21000 23f7 radeon_kabini_pfp_bin.ko >> 10 1 0xffffffff82a24000 23f5 radeon_kabini_me_bin.ko >> 11 1 0xffffffff82a27000 23f5 radeon_kabini_ce_bin.ko >> 12 1 0xffffffff82a2a000 43f7 radeon_kabini_mec_bin.ko >> 13 1 0xffffffff82a2f000 2a77 radeon_kabini_rlc_bin.ko >> 14 1 0xffffffff82a32000 12e9 radeon_kabini_sdma_bin.ko >> 15 1 0xffffffff82a34000 12eb radeon_kabini_sdma1_bin.ko >> 16 1 0xffffffff82a36000 38ea7 radeon_kabini_uvd_bin.ko >> 17 1 0xffffffff82a6f000 18c47 radeon_kabini_vce_bin.ko >> 18 1 0xffffffff82a88000 2538 intpm.ko >> 19 1 0xffffffff82a8b000 a50 smbus.ko >> 20 1 0xffffffff82a8c000 1820 uhid.ko >> 21 1 0xffffffff82a8e000 2928 ums.ko >> 22 1 0xffffffff82a91000 1b00 wmt.ko >> 23 1 0xffffffff82a93000 acf mac_ntpd.ko >> 24 1 0xffffffff82a94000 a9f8 tmpfs.ko >> >> clay@bsd13:~ % pkg info fusefs-ntfs-2017.3.23 >> fusefs-ntfs-2017.3.23 >> Name : fusefs-ntfs >> Version : 2017.3.23 >> Installed on : Sun Sep 8 17:24:36 2019 CDT >> Origin : sysutils/fusefs-ntfs >> Architecture : FreeBSD:13:amd64 >> Prefix : /usr/local >> Categories : sysutils >> Licenses : GPLv2+ >> Maintainer : freebsd@dussan.org >> WWW : https://www.tuxera.com/community/open-source-ntfs-3g/ >> Comment : Mount NTFS partitions (read/write) and disk images >> Options : >> DOCS : on >> LOCK : on >> UBLIO : on >> Shared Libs required: >> libfuse.so.2 >> libuuid.so.1 >> libublio.so.1 >> Shared Libs provided: >> libntfs-3g.so.88 >> Annotations : >> FreeBSD_version: 1300044 >> repo_type : binary >> repository : FreeBSD >> Flat size : 1.93MiB >> Description : >> The ntfs-3g driver is an open source, freely available read/write NTFS >> driver, which provides safe and fast handling of the Windows XP, Windows >> Server 2003, Windows 2000, Windows Vista, Windows Server 2008, Windows 7 >> and Windows 8 NTFS file systems. Almost the full POSIX filesystem >> functionality is supported, the major exceptions are changing the file >> ownerships and the access rights. >> >> root@bsd13:/home/clay # ntfs-3g -o default_permissons /dev/ada0p4 /mnt >> * Windows is hibernated, refused to mount. >> The disk contains an unclean file system (0, 0). >> Metadata kept in Windows cache, refused to mount. >> Falling back to read-only mount because the NTFS partition is in an unsafe >> state. Please resume and shutdown Windows fully (no hibernation >> or fast restarting. >> >> I'm having a roadblock here. Maybe someone knows the answer. >> > > I'm assuming that you already ascertained that the Windows file system was > in fact clean? If so, then this sounds like a problem with fusefs-ntfs, > not with fuse in general. I've CC'd its maintainer. He may be able to > help you. > -Alan Windows-10 has a changed default; it does the equivalent of hibernation to facilitate "fast start". ntfs-3g won't touch a partition for writing in that mode :-( If I recall correctly, there is a setting you must change from in Windows under control panel -> system -> power and sleep. From there you should be able to disable the "fast start" option and, after shutting down, ntfs-3g will allow a read/write mount, imb From owner-freebsd-current@freebsd.org Mon Sep 9 00:08:32 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A758CE7BC8 for ; Mon, 9 Sep 2019 00:08:32 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from atl4mhob01.registeredsite.com (atl4mhob01.registeredsite.com [209.17.115.39]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.registeredsite.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46RT5q3hN9z4Ghr for ; Mon, 9 Sep 2019 00:08:31 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mailpod.hostingplatform.com (atl4qobmail01pod2.registeredsite.com [10.30.77.35]) by atl4mhob01.registeredsite.com (8.14.4/8.14.4) with ESMTP id x8908SGa019894 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Sun, 8 Sep 2019 20:08:29 -0400 Received: (qmail 49025 invoked by uid 0); 9 Sep 2019 00:08:28 -0000 X-TCPREMOTEIP: 99.253.177.25 X-Authenticated-UID: dclarke@blastwave.org Received: from unknown (HELO ?172.16.35.3?) (dclarke@blastwave.org@99.253.177.25) by 0 with ESMTPA; 9 Sep 2019 00:08:28 -0000 Subject: Re: Panic with Current 351901 ISO image To: freebsd-current@freebsd.org References: <01378a14-471d-31b4-c60a-98a7a059748b@gmail.com> From: Dennis Clarke Message-ID: Date: Sun, 8 Sep 2019 20:08:18 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Thunderbird/69.0 MIME-Version: 1.0 In-Reply-To: <01378a14-471d-31b4-c60a-98a7a059748b@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46RT5q3hN9z4Ghr X-Spamd-Bar: +++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of dclarke@blastwave.org has no SPF policy when checking 209.17.115.39) smtp.mailfrom=dclarke@blastwave.org X-Spamd-Result: default: False [5.64 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; RBL_VIRUSFREE_BOTNET(2.00)[39.115.17.209.bip.virusfree.cz : 127.0.0.2]; NEURAL_SPAM_MEDIUM(1.00)[0.998,0]; NEURAL_SPAM_LONG(1.00)[1.000,0]; RCVD_IN_DNSWL_NONE(0.00)[39.115.17.209.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; DMARC_NA(0.00)[blastwave.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:19871, ipnet:209.17.112.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.74)[ipnet: 209.17.112.0/21(2.08), asn: 19871(1.66), country: US(-0.05)] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 00:08:32 -0000 On 9/8/19 6:03 PM, Curtis Hamilton wrote: > I'm encountering (randomly) the below error when trying to install > 351901 from CD/DVD. > On what type of system ? -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken GreyBeard and suspenders optional From owner-freebsd-current@freebsd.org Mon Sep 9 00:19:50 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3067AE8371 for ; Mon, 9 Sep 2019 00:19:50 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-ot1-x32d.google.com (mail-ot1-x32d.google.com [IPv6:2607:f8b0:4864:20::32d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46RTLs237Fz4HJG; Mon, 9 Sep 2019 00:19:48 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-ot1-x32d.google.com with SMTP id 100so10847693otn.2; Sun, 08 Sep 2019 17:19:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=MWBP8BB3zcz/pq/7X8Sy5EluOTyL6O43XxnZzP9DrLs=; b=rhTwVhaRRk0TuKaKdyvjHBgR/t1xV9R29xpX5AiP7VXN/Ioe3zz1ycXLqItSlnpaxR 7T8wP8BZ5zzs13bzKoBKAF+1j7crhDLRS9Z8LrWKEepfiQVhLQj/MLCzfTySpSIMmNhQ CT1u3H2VV0IxFYDtdGq51IH4KwVTuDj0J/ypY0I69BIk9A2n57AxSK+9VbhQXSArgdkt LxbAgAHBtbCac4VNmvlykeDm8noKFqTOFlLC12v22hZ1m57y/Z5sj628HdZU9CoFWHSI cNf1vgAf4YI1QOItapvnlEBjTFkR5T7EEbDTGTVaqDhMKISKT5dmeMEC69rkqYL7JVc1 TT4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=MWBP8BB3zcz/pq/7X8Sy5EluOTyL6O43XxnZzP9DrLs=; b=YO2gJOmMtNx7PmqD+Cxx/SphGR4GrAtoJm2oGNcVb/EkcNhLqVt0A7vQKiVbvl2v04 zxF1SC8PET5sE0AUIPKdvtLKxAbux8LV2J5TxBlaD1CR5tEhTKEBL56/JOQ+6+Heaalr +iTfNjbTOwOjuFVIOm8n6nNMiV3g7pkHZjRTgcylxlPP/im1XNOHqHZgQb0GqqrJwtwu Q11S1BP9H5YeqqcU5XHPHjLZurin3qgDV1yNhcXS+hbljBjT3djOrb0mmP+SvFrnHV7I xonTOoI3lsVbboLlqaGoPfxflfRt2OaRS2H5lTNCDcHSroeIAqERGNQPczcRwIODpZwV wm2A== X-Gm-Message-State: APjAAAWUT0lK6sNM8TtdqutA6/Z7BLcWgwhZQQ4xQb3JCoPwkG5Tourq WDhxqqL5e2uM5lJPUgcPdYcy5VWqP4P+k/ZBB/c= X-Google-Smtp-Source: APXvYqyqo8tX9LVKgeDMeVlKifchCsUfr0BnjArZXtJXFqiNhTvwasXY4UpK581ckS2+FJx76DRwBWXaNIlV8zTYzIs= X-Received: by 2002:a05:6830:138b:: with SMTP id d11mr15144259otq.199.1567988387516; Sun, 08 Sep 2019 17:19:47 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Kevin Oberman Date: Sun, 8 Sep 2019 17:19:30 -0700 Message-ID: Subject: Re: fusefs & ntfs-3g To: Michael Butler Cc: Alan Somers , "Clay Daniels Jr." , "freebsd-current@freebsd.org" , freebsd@dussan.org X-Rspamd-Queue-Id: 46RTLs237Fz4HJG X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=rhTwVhaR; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of kob6558@gmail.com designates 2607:f8b0:4864:20::32d as permitted sender) smtp.mailfrom=kob6558@gmail.com X-Spamd-Result: default: False [-1.20 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; RCPT_COUNT_FIVE(0.00)[5]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FORGED_SENDER(0.30)[rkoberman@gmail.com,kob6558@gmail.com]; IP_SCORE(0.00)[ip: (-6.44), ipnet: 2607:f8b0::/32(-2.75), asn: 15169(-2.27), country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[rkoberman@gmail.com,kob6558@gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[d.2.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 00:19:50 -0000 On Sun, Sep 8, 2019 at 4:46 PM Michael Butler wrote: > On 9/8/19 7:09 PM, Alan Somers wrote: > > On Sun, Sep 8, 2019 at 5:00 PM Clay Daniels Jr. < > clay.daniels.jr@gmail.com> > > wrote: > > > >> I want to view my Windows C: drive on ata0p4. > >> This is what I have: > [...] > >> root@bsd13:/home/clay # ntfs-3g -o default_permissons /dev/ada0p4 /mnt > >> * Windows is hibernated, refused to mount. > >> The disk contains an unclean file system (0, 0). > >> Metadata kept in Windows cache, refused to mount. > >> Falling back to read-only mount because the NTFS partition is in an > unsafe > >> state. Please resume and shutdown Windows fully (no hibernation > >> or fast restarting. > >> > >> I'm having a roadblock here. Maybe someone knows the answer. > >> > > > > I'm assuming that you already ascertained that the Windows file system > was > > in fact clean? If so, then this sounds like a problem with fusefs-ntfs, > > not with fuse in general. I've CC'd its maintainer. He may be able to > > help you. > > -Alan > > Windows-10 has a changed default; it does the equivalent of hibernation > to facilitate "fast start". ntfs-3g won't touch a partition for writing > in that mode :-( > > If I recall correctly, there is a setting you must change from in > Windows under control panel -> system -> power and sleep. From there you > should be able to disable the "fast start" option and, after shutting > down, ntfs-3g will allow a read/write mount, > I can confirm this. It is documented, but not obvious. You ave two choices: 1. Change the system setting, more or less as Michael suggests. I'm away from my only W10 system, so I can't check the exact details. 2. Force a full shutdown by starting a command window and entering "shutdown /s /f /t 0". This is a one-time full shutdown. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 From owner-freebsd-current@freebsd.org Mon Sep 9 00:41:49 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 62B45E8CA5 for ; Mon, 9 Sep 2019 00:41:49 +0000 (UTC) (envelope-from clay.daniels.jr@gmail.com) Received: from mail-vs1-xe42.google.com (mail-vs1-xe42.google.com [IPv6:2607:f8b0:4864:20::e42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46RTrD0JBWz4JLP; Mon, 9 Sep 2019 00:41:47 +0000 (UTC) (envelope-from clay.daniels.jr@gmail.com) Received: by mail-vs1-xe42.google.com with SMTP id s18so7676055vsa.0; Sun, 08 Sep 2019 17:41:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Cuz3UuG/o/Qscibhkn3FVXucBCZmcvUTW8pnbrB8M94=; b=CIWFMnsfutwPAOl1jOb/2XNS/FnV4NniyS3hpCxYbRA6o+NdEJjB+xLBTbQgA+EenI FpE3ZGcOE3MkzavUIYlBgSXMw/uRW0kRMQ29plh64wz+aveD1/BQ0YyWg0QP3RvkXEh6 mVZFRxXpEkszexUyqcsNBQgkDBHRhtOVRDABkhvgRlwONhyc2OEAOFBAiEURR5BBUwvl xMEPuF6Xm/cpYv3P8rDq9IaySICsYlnHF8acLhMouKS19czhZoiSzRKAkTQbI9skEuOl TcQUVTmEe/X9MOP4pB0DFjTwBngNhcjgoYG3+ydYNHI1vg2S3cLHg/5gWIiMGdiloZkY D7KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Cuz3UuG/o/Qscibhkn3FVXucBCZmcvUTW8pnbrB8M94=; b=falakkH4hZ7do8GnqHCIRFLcFJUemoPh7huyCVWVIUR65YHSYgDgZYlXwaNVM7ZpOF HZk/WMF5smXC11gtR6i413L0cw10bNMmh9RWEaVREsfd26myUeh057KWtK4di9wnnblD /nATozzvu5Yy6HYnhAk0mzzX3MDRA2WMR8EyPlJodURRXOpNK3VixOHWijyMyhJ2Zind 5QDzsEk8Z35I+S4snYjAnn7ZvARfhtLiBYBvqkZR57dfJeWc9244FKWuk9pKresCvpjz 6gLngEiPjrzR4a2WzfiNCtmqABg36CGapVy1Xr4PQVzZmI7m7n4wj8mNllXwkx+sB2Tl TRjA== X-Gm-Message-State: APjAAAVCvjyPG8DfGJEQy1V9EKf5XSFhMG8DT1GFxc2xsdTCB0zC++zW 7SotH4u1qOwZ1CcDYLuLD7VnekeZZ1HdZ4ZLqA== X-Google-Smtp-Source: APXvYqynZ2g2IrKGdIK/pKn4J6BUa5BnaREqpzcq++RnfPCc8BsiS0bG7EB57SFHoXhO5KMnS7I6EbOD6POu2KyO+6A= X-Received: by 2002:a67:ce88:: with SMTP id c8mr7640334vse.221.1567989706398; Sun, 08 Sep 2019 17:41:46 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: "Clay Daniels Jr." Date: Sun, 8 Sep 2019 19:41:34 -0500 Message-ID: Subject: Re: fusefs & ntfs-3g To: Kevin Oberman Cc: Michael Butler , Alan Somers , "freebsd-current@freebsd.org" , freebsd@dussan.org X-Rspamd-Queue-Id: 46RTrD0JBWz4JLP X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=CIWFMnsf; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of claydanielsjr@gmail.com designates 2607:f8b0:4864:20::e42 as permitted sender) smtp.mailfrom=claydanielsjr@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; RCPT_COUNT_FIVE(0.00)[5]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (2.16), ipnet: 2607:f8b0::/32(-2.75), asn: 15169(-2.27), country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2.4.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 00:41:49 -0000 Alan, Michael, & Kevin, THANKS very much. This is kind of neat: root@bsd13:/home/clay # ntfs-3g -o default_permissons /dev/ada0p4 /mnt root@bsd13:/home/clay # cd /mnt root@bsd13:/mnt # ls -al total 2368276 drwxrwxrwx 1 root wheel 0 Jul 17 16:40 $Recycle.Bin drwxrwxrwx 1 root wheel 4096 Aug 6 02:08 . drwxr-xr-x 19 root wheel 1024 Sep 8 19:25 .. drwxrwxrwx 1 root wheel 0 Aug 13 15:05 Config.Msi lrwxrwxrwx 2 root wheel 10 Jul 17 17:07 Documents and Settings -> /mnt/Users drwxrwxrwx 1 root wheel 0 Mar 18 23:52 PerfLogs drwxrwxrwx 1 root wheel 4096 Aug 4 16:11 Planetdance drwxrwxrwx 1 root wheel 8192 Sep 3 18:55 Program Files drwxrwxrwx 1 root wheel 8192 Aug 11 02:20 Program Files (x86) drwxrwxrwx 1 root wheel 4096 Aug 6 02:01 ProgramData drwxrwxrwx 1 root wheel 0 Jul 17 15:42 Recovery drwxrwxrwx 1 root wheel 4096 Sep 8 13:59 System Volume Information drwxrwxrwx 1 root wheel 4096 Jul 17 16:01 Users drwxrwxrwx 1 root wheel 16384 Aug 31 20:39 Windows -rwxrwxrwx 1 root wheel 1485533184 Sep 8 19:03 hiberfil.sys -rwxrwxrwx 1 root wheel 671088640 Sep 8 00:24 pagefile.sys -rwxrwxrwx 1 root wheel 268435456 Sep 8 00:24 swapfile.sys root@bsd13:/mnt # Clay On Sun, Sep 8, 2019 at 7:19 PM Kevin Oberman wrote: > On Sun, Sep 8, 2019 at 4:46 PM Michael Butler > wrote: > >> On 9/8/19 7:09 PM, Alan Somers wrote: >> > On Sun, Sep 8, 2019 at 5:00 PM Clay Daniels Jr. < >> clay.daniels.jr@gmail.com> >> > wrote: >> > >> >> I want to view my Windows C: drive on ata0p4. >> >> This is what I have: >> [...] >> >> root@bsd13:/home/clay # ntfs-3g -o default_permissons /dev/ada0p4 /mnt >> >> * Windows is hibernated, refused to mount. >> >> The disk contains an unclean file system (0, 0). >> >> Metadata kept in Windows cache, refused to mount. >> >> Falling back to read-only mount because the NTFS partition is in an >> unsafe >> >> state. Please resume and shutdown Windows fully (no hibernation >> >> or fast restarting. >> >> >> >> I'm having a roadblock here. Maybe someone knows the answer. >> >> >> > >> > I'm assuming that you already ascertained that the Windows file system >> was >> > in fact clean? If so, then this sounds like a problem with fusefs-ntfs, >> > not with fuse in general. I've CC'd its maintainer. He may be able to >> > help you. >> > -Alan >> >> Windows-10 has a changed default; it does the equivalent of hibernation >> to facilitate "fast start". ntfs-3g won't touch a partition for writing >> in that mode :-( >> >> If I recall correctly, there is a setting you must change from in >> Windows under control panel -> system -> power and sleep. From there you >> should be able to disable the "fast start" option and, after shutting >> down, ntfs-3g will allow a read/write mount, >> > > I can confirm this. It is documented, but not obvious. You ave two choices: > 1. Change the system setting, more or less as Michael suggests. I'm away > from my only W10 system, so I can't check the exact details. > 2. Force a full shutdown by starting a command window and entering > "shutdown /s /f /t 0". This is a one-time full shutdown. > -- > Kevin Oberman, Part time kid herder and retired Network Engineer > E-mail: rkoberman@gmail.com > PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 > > From owner-freebsd-current@freebsd.org Mon Sep 9 16:23:47 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 89560D7296 for ; Mon, 9 Sep 2019 16:23:47 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound3d.ore.mailhop.org (outbound3d.ore.mailhop.org [54.186.57.195]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46Rtl716H5z4BWv for ; Mon, 9 Sep 2019 16:23:46 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1568046225; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=DMHRkqd0qMhzvhET7okL/oQvH8SjkRcvPx88TQRMVHB7aX64XlkD6FCrra7/rMpBBc799XwLUqQxT /OI7Rbcasg95iwquePimpd+KnC8jGfC5S4rB4PdWBoGh5PMHD8DRpg4Mhsn0FM0FC4M9lq/3HJ5yfa CHCsj08P10zev4Gwf9JhjaFTAkYywSowcmRtqg2qWoBXi37ytmTOj2zIqiEOzMin1Gpbvk0zSd7hAQ DKWPscSiTJdKYMZZiJIVcfFMoZWnBH+rZP2Z83Xfyw7158pgwsmo5h7Opd7oL+dRXKEWK0crxf9hu6 2f0lFT16zDLjZRAZFJfYk8kzzlB4ooA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=ud7Q7xdQPRE4fkkD0wRKHXhnzBfclKNdJdmcyv1JiSk=; b=jNH1WQhvhkicmjFXaklmSha6Ha/KgCcAkmBy9FKNGZHcpDJDoVWPz1Cq+X0OT9g1QrSW/X7BN4dSN QdQuflvOOJrO6JnFCepPlcNsHyRcvBWOf/GlbAzkILTM0JBvvSgjROl4Ui9MMPddSGSRxzEFb3pabY rkMxvAUBQC0Gz99nZeIbeWDYPmrqY2j1oXe3SJwx0oVx64qer/3S2yvnjdK8zsyvw92lMtUuKr6X7P wotu78zdjBPAVrv0/GQmJuNzWl3Oo98NeTP/56UPKHPmZio3jizFpobKo+y7HgNerF3SJet9Z/XLYM kdlxyw34CUJmLO1fDA0ijHOhFCU0RrQ== ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=ud7Q7xdQPRE4fkkD0wRKHXhnzBfclKNdJdmcyv1JiSk=; b=rkpLS3dwfbw0nskVY1CZJPx/JMpX9qtSDolcCFAU30LzVzMQy/LgJt7Bvs6WwsCBFzJpqqinz9z6X JjLec9EDDYORsxcooJHJ3SJyyizWILoj4+0DytMkzaCWdf8EDDysi++Z2GINmanQaKWsfDS/wBtRiH MQ7c2xTgjzw3tBL+oCaP1javiQvck8ON+Kl5Uj73KxLXdct3XdOSxKG3tLMJ+VC+hah7EoZg69zdVE cLprhVbXEem2gjq6FpdkOmbCnst984NzOjCjbq4+hAl+XH3JrqlqGNHKVmgs8DxcCdftWgkCiLpDqa 8MkC6gJdNDeL80SNCT7PrIMecIIDz4A== X-MHO-RoutePath: aGlwcGll X-MHO-User: 2f71b0be-d31e-11e9-b67d-cdd75d6ce7a8 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound3.ore.mailhop.org (Halon) with ESMTPSA id 2f71b0be-d31e-11e9-b67d-cdd75d6ce7a8; Mon, 09 Sep 2019 16:23:43 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x89GNeFc062698; Mon, 9 Sep 2019 10:23:40 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <66f80012757134b6317b673f9eeb24db66c996a2.camel@freebsd.org> Subject: Re: ntpd segfaults on start From: Ian Lepore To: Cy Schubert , Konstantin Belousov Cc: Harlan Stenn , Vladimir Zakharov , freebsd-current@freebsd.org Date: Mon, 09 Sep 2019 10:23:40 -0600 In-Reply-To: <201909071628.x87GSFPV005827@slippy.cwsent.com> References: <201909051307.x85D7MGs034053@slippy.cwsent.com> <20190905142817.GB2559@kib.kiev.ua> <201909060355.x863tRhP089169@slippy.cwsent.com> <201909060639.x866dJ7f090176@slippy.cwsent.com> <201909062356.x86NuKdk003780@slippy.cwsent.com> <156d1e7c-0dbb-8707-90b3-13ae97c87449@nwtime.org> <20190907075619.GG2559@kib.kiev.ua> <201909071309.x87D9GxZ089964@slippy.cwsent.com> <20190907153226.GI2559@kib.kiev.ua> <201909071545.x87FjLS6004448@slippy.cwsent.com> <20190907161749.GJ2559@kib.kiev.ua> <201909071628.x87GSFPV005827@slippy.cwsent.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46Rtl716H5z4BWv X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-2.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; ASN(0.00)[asn:16509, ipnet:54.186.0.0/15, country:US] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 16:23:47 -0000 On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote: > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin Belousov writes: > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote: > > > I've been able to set the memlock rlimit as low as 20 MB. The issue is > > > letting it default to 0 which allows ntp to mlockall() anything it wants. > > > ntpd on my sandbox is currently using 18 MB. > > > > Default stack size on amd64 is 512M, and default stack gap percentage is > > 3%. This means that the gap can be as large as ~17MB. If 3MB is enough > > for the stack of the main thread of ntpd, then fine. > > The default stack is 200K, which is also tuneable in ntp.conf. > > [...] I haven't seen anyone ask what I consider to be the crucial question yet: why are we locking ntpd into memory by default at all? I have seen two rationales for ntpd using mlockall() and setrlimit(): - There are claims that it improves timing performance. - Because ntpd is a daemon that can run for months at a time, setting limits on memory and stack growth can help detect and mitigate against memory leak problems in the daemon. I am *highly* skeptical of claims that locking ntpd in memory will improve timing performance on freebsd (to the point where I'm inclined to dismiss them out of hand, but I'd be willing to look at any actual evidence). The second point has at least some validity, but would be better implemented (IMO) by decoupling the address space limit from the act of locking down memory, and allowing configuration of RLIMIT_AS separately from RLIMIT_MEMLOCK. If there's any interest in this, I could try to put together a patchset and submit it upstream for that. I would propose that for freebsd, the autoconf-generated value for DFLT_RLIMIT_MEMLOCK should be -1 to avoid calling setrlimit() and mlockall() by default. Then in the ntp.conf we distribute we have a section like: # To lock ntpd into memory, uncomment the following rlimit line. # This should not be necessary on most systems, but may improve # performance on heavily-loaded servers experiencing enough # memory pressure to cause ntpd to be paged out. # rlimit memlock stacksize Then we would need to come up with reasonable values for . -- Ian From owner-freebsd-current@freebsd.org Mon Sep 9 16:30:57 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 13408D75A0 for ; Mon, 9 Sep 2019 16:30:57 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46RtvN5YC0z4BwC; Mon, 9 Sep 2019 16:30:56 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x89GUjvG044289; Mon, 9 Sep 2019 09:30:45 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x89GUjGX044288; Mon, 9 Sep 2019 09:30:45 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net> Subject: Re: ntpd segfaults on start In-Reply-To: <66f80012757134b6317b673f9eeb24db66c996a2.camel@freebsd.org> To: Ian Lepore Date: Mon, 9 Sep 2019 09:30:45 -0700 (PDT) CC: Cy Schubert , Konstantin Belousov , Harlan Stenn , Vladimir Zakharov , freebsd-current@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 46RtvN5YC0z4BwC X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; TAGGED_RCPT(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 16:30:57 -0000 > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote: > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin Belousov writes: > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote: > > > > I've been able to set the memlock rlimit as low as 20 MB. The issue is > > > > letting it default to 0 which allows ntp to mlockall() anything it wants. > > > > ntpd on my sandbox is currently using 18 MB. > > > > > > Default stack size on amd64 is 512M, and default stack gap percentage is > > > 3%. This means that the gap can be as large as ~17MB. If 3MB is enough > > > for the stack of the main thread of ntpd, then fine. > > > > The default stack is 200K, which is also tuneable in ntp.conf. > > > > [...] > > I haven't seen anyone ask what I consider to be the crucial question > yet: why are we locking ntpd into memory by default at all? > > I have seen two rationales for ntpd using mlockall() and setrlimit(): > > - There are claims that it improves timing performance. > > - Because ntpd is a daemon that can run for months at a time, > setting limits on memory and stack growth can help detect and > mitigate against memory leak problems in the daemon. Doesn't locking this memory down also protect ntpd from OOM kills? If so that is a MUST preserve functionality, as IMHO killing ntpd on a box that has it configured is a total no win situation. Regards, Rod > I am *highly* skeptical of claims that locking ntpd in memory will > improve timing performance on freebsd (to the point where I'm inclined > to dismiss them out of hand, but I'd be willing to look at any actual > evidence). > > The second point has at least some validity, but would be better > implemented (IMO) by decoupling the address space limit from the act of > locking down memory, and allowing configuration of RLIMIT_AS separately > from RLIMIT_MEMLOCK. If there's any interest in this, I could try to > put together a patchset and submit it upstream for that. > > I would propose that for freebsd, the autoconf-generated value for > DFLT_RLIMIT_MEMLOCK should be -1 to avoid calling setrlimit() and > mlockall() by default. Then in the ntp.conf we distribute we have a > section like: > > # To lock ntpd into memory, uncomment the following rlimit line. > # This should not be necessary on most systems, but may improve > # performance on heavily-loaded servers experiencing enough > # memory pressure to cause ntpd to be paged out. > # rlimit memlock stacksize > > Then we would need to come up with reasonable values for . > > -- Ian > > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-current@freebsd.org Mon Sep 9 18:13:34 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0815ADAA4E for ; Mon, 9 Sep 2019 18:13:34 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound2m.ore.mailhop.org (outbound2m.ore.mailhop.org [54.149.155.156]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46Rx9n489Tz4LkQ for ; Mon, 9 Sep 2019 18:13:33 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1568052811; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=dDMr0Boc3Th5HN+DoCayLzudhO8wvUQy5Pp/AJ0CbtX0Ai8WthQGLPGxwu4EWAZM3w8gkYzEqbzYT oYi81heJhcoXW7XrzXkS0AgAzfYjqRT2FrznG/mCD9uM1D2WA7SoVoqO/Pv+jVaJW5vedx2t9zzynv /sQSeHanBUzEiyd3aeEeYALLuz+WKvw7XBXNA/gYwgp6NnsSGGPn3E3tUCjx/nZKdU22yHP9K6tHfh n5I9ArPAx5WjFeAHSHtJI0AUD5EBnVji30fJFEqnGiw/UhvLPITV1Cn/0zArvsg+buyVj/Yfx3Rx7x AC3HhK+QmaGGMrSlxJ/ZbwCFtrXIgMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=M6kE9IAaGxc8PIJ7D5sV1E8cnc9Ni9NeV7sCnyDluTw=; b=pdrTkgJp46fFHgVkw0Jxt9nYTocGQfYZLp5MQC9WfvUmrR14p+E4Pwtltz4aweclI6iBqvCq+qFMc cfZ/9wBgUHWsTpGXLaE3S/DzoKDt7DMbtNxe8X9I7QdZVNauEmwbHePte1B6k8rkb2HcWyKg+v/eKh lMQgo1V+OrAGu5LnyAl5qwgvyl12qDmhAHYa/OVpJHG79oCQGebxEzZvkiJevGNHIWDEGJXF6+QRO4 2N4q+2IS/MbPXUhvcd4wPilni9AyJMFmXOON+91mSiLTLAMzZmVhgdPBxEmNmV7uQRS+VudqHwiEGS aoLHaI/Ju4Dm+PvMGrOi6ZCbwBsa9EQ== ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=M6kE9IAaGxc8PIJ7D5sV1E8cnc9Ni9NeV7sCnyDluTw=; b=hbTbpjWqKPr2EHmVLGMW7XvABCQlKGkST9hj6rot0EQChgLnPMtRbC4MlEcjgIQ2jDcNWL1qz9aF+ jvjj5HWygkBAYA0ezbRWq3ewIY5G8UpqQ4VyPIRUJEYhbO7txfuXMGXYn7Jn1+JJTx9RbRoMiAOp5d Lrpr/ffCqelB/ou0sDGhPwnL7Mkyj0EaYQrSnPa+ePOTTZxWM3jwfihNp8c0NGTTe6fruljSd3opcP r7aKUz+rvi6fSU8OYF/XyF6i0IjVTkCI+ga+uPEXsr+2wY+VcwCwRcGeCcFwJ6XexZ18L+fWUAdV47 0ssc5zpZI4Wz4/K55kCcbjJvFNguOng== X-MHO-RoutePath: aGlwcGll X-MHO-User: 8375d43e-d32d-11e9-85ed-13b9aae3a1d2 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound4.ore.mailhop.org (Halon) with ESMTPSA id 8375d43e-d32d-11e9-85ed-13b9aae3a1d2; Mon, 09 Sep 2019 18:13:30 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x89IDOXq063123; Mon, 9 Sep 2019 12:13:24 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: Subject: Re: ntpd segfaults on start From: Ian Lepore To: "Rodney W. Grimes" Cc: Cy Schubert , Konstantin Belousov , Harlan Stenn , Vladimir Zakharov , freebsd-current@freebsd.org Date: Mon, 09 Sep 2019 12:13:24 -0600 In-Reply-To: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net> References: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46Rx9n489Tz4LkQ X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-2.00 / 15.00]; TAGGED_RCPT(0.00)[]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; ASN(0.00)[asn:16509, ipnet:54.148.0.0/15, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 18:13:34 -0000 On Mon, 2019-09-09 at 09:30 -0700, Rodney W. Grimes wrote: > > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote: > > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin > > > Belousov writes: > > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote: > > > > > I've been able to set the memlock rlimit as low as 20 MB. The > > > > > issue is > > > > > letting it default to 0 which allows ntp to mlockall() > > > > > anything it wants. > > > > > ntpd on my sandbox is currently using 18 MB. > > > > > > > > Default stack size on amd64 is 512M, and default stack gap > > > > percentage is > > > > 3%. This means that the gap can be as large as ~17MB. If 3MB is > > > > enough > > > > for the stack of the main thread of ntpd, then fine. > > > > > > The default stack is 200K, which is also tuneable in ntp.conf. > > > > > > [...] > > > > I haven't seen anyone ask what I consider to be the crucial > > question > > yet: why are we locking ntpd into memory by default at all? > > > > I have seen two rationales for ntpd using mlockall() and > > setrlimit(): > > > > - There are claims that it improves timing performance. > > > > - Because ntpd is a daemon that can run for months at a time, > > setting limits on memory and stack growth can help detect and > > mitigate against memory leak problems in the daemon. > > Doesn't locking this memory down also protect ntpd from OOM kills? > If so that is a MUST preserve functionality, as IMHO killing ntpd > on a box that has it configured is a total no win situation. > Does it have that effect? I don't know. But I would argue that that's a separate issue, and we should make that happen by adding ntpd_oomprotect=YES to /etc/defaults/rc.conf Right now only syslogd has oomprotect set to YES by default. Maybe that's a good choice -- once we start declaring one daemon to be more important than others, you'll discover there's a whole back lot full of bikesheds that need painting. So maybe we should just document ntpd_oomprotect=YES in some more- prominent way. If we were to add a comment block to ntp.conf describing rlimit, that might be a good place to mention setting ntpd_oomprotect in rc.conf. -- Ian From owner-freebsd-current@freebsd.org Mon Sep 9 18:22:52 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A38D2DB174 for ; Mon, 9 Sep 2019 18:22:52 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46RxNW1lFtz4MYK for ; Mon, 9 Sep 2019 18:22:50 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x89IMkAt044809; Mon, 9 Sep 2019 11:22:46 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x89IMkdK044808; Mon, 9 Sep 2019 11:22:46 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201909091822.x89IMkdK044808@gndrsh.dnsmgr.net> Subject: Re: ntpd segfaults on start In-Reply-To: To: Ian Lepore Date: Mon, 9 Sep 2019 11:22:46 -0700 (PDT) CC: "Rodney W. Grimes" , Cy Schubert , Konstantin Belousov , Harlan Stenn , Vladimir Zakharov , freebsd-current@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 46RxNW1lFtz4MYK X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd-rwg@gndrsh.dnsmgr.net has no SPF policy when checking 69.59.192.140) smtp.mailfrom=freebsd-rwg@gndrsh.dnsmgr.net X-Spamd-Result: default: False [1.93 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.64)[-0.636,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[dnsmgr.net]; AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(0.04)[ip: (0.15), ipnet: 69.59.192.0/19(0.07), asn: 13868(0.04), country: US(-0.05)]; NEURAL_SPAM_LONG(0.13)[0.125,0]; RCPT_COUNT_SEVEN(0.00)[7]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SUSPICIOUS_RECIPS(1.50)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 18:22:52 -0000 > On Mon, 2019-09-09 at 09:30 -0700, Rodney W. Grimes wrote: > > > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote: > > > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin > > > > Belousov writes: > > > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote: > > > > > > I've been able to set the memlock rlimit as low as 20 MB. The > > > > > > issue is > > > > > > letting it default to 0 which allows ntp to mlockall() > > > > > > anything it wants. > > > > > > ntpd on my sandbox is currently using 18 MB. > > > > > > > > > > Default stack size on amd64 is 512M, and default stack gap > > > > > percentage is > > > > > 3%. This means that the gap can be as large as ~17MB. If 3MB is > > > > > enough > > > > > for the stack of the main thread of ntpd, then fine. > > > > > > > > The default stack is 200K, which is also tuneable in ntp.conf. > > > > > > > > [...] > > > > > > I haven't seen anyone ask what I consider to be the crucial > > > question > > > yet: why are we locking ntpd into memory by default at all? > > > > > > I have seen two rationales for ntpd using mlockall() and > > > setrlimit(): > > > > > > - There are claims that it improves timing performance. > > > > > > - Because ntpd is a daemon that can run for months at a time, > > > setting limits on memory and stack growth can help detect and > > > mitigate against memory leak problems in the daemon. > > > > Doesn't locking this memory down also protect ntpd from OOM kills? > > If so that is a MUST preserve functionality, as IMHO killing ntpd > > on a box that has it configured is a total no win situation. > > > > Does it have that effect? I don't know. But I would argue that that's > a separate issue, and we should make that happen by adding > ntpd_oomprotect=YES to /etc/defaults/rc.conf > > Right now only syslogd has oomprotect set to YES by default. Maybe > that's a good choice -- once we start declaring one daemon to be more > important than others, you'll discover there's a whole back lot full of > bikesheds that need painting. > > So maybe we should just document ntpd_oomprotect=YES in some more- > prominent way. If we were to add a comment block to ntp.conf > describing rlimit, that might be a good place to mention setting > ntpd_oomprotect in rc.conf. > > -- Ian All very reasonable, thanks Ian. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-current@freebsd.org Mon Sep 9 18:45:00 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B7D5CDBCA2 for ; Mon, 9 Sep 2019 18:45:00 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46Rxt43bTMz4PGt; Mon, 9 Sep 2019 18:45:00 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x89Iikci082709 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 9 Sep 2019 21:44:49 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x89Iikci082709 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x89IikiX082708; Mon, 9 Sep 2019 21:44:46 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 9 Sep 2019 21:44:46 +0300 From: Konstantin Belousov To: Ian Lepore Cc: "Rodney W. Grimes" , Cy Schubert , Harlan Stenn , Vladimir Zakharov , freebsd-current@freebsd.org Subject: Re: ntpd segfaults on start Message-ID: <20190909184446.GU2559@kib.kiev.ua> References: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-Rspamd-Queue-Id: 46Rxt43bTMz4PGt X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; REPLY(-4.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 18:45:00 -0000 On Mon, Sep 09, 2019 at 12:13:24PM -0600, Ian Lepore wrote: > On Mon, 2019-09-09 at 09:30 -0700, Rodney W. Grimes wrote: > > > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote: > > > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin > > > > Belousov writes: > > > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote: > > > > > > I've been able to set the memlock rlimit as low as 20 MB. The > > > > > > issue is > > > > > > letting it default to 0 which allows ntp to mlockall() > > > > > > anything it wants. > > > > > > ntpd on my sandbox is currently using 18 MB. > > > > > > > > > > Default stack size on amd64 is 512M, and default stack gap > > > > > percentage is > > > > > 3%. This means that the gap can be as large as ~17MB. If 3MB is > > > > > enough > > > > > for the stack of the main thread of ntpd, then fine. > > > > > > > > The default stack is 200K, which is also tuneable in ntp.conf. > > > > > > > > [...] > > > > > > I haven't seen anyone ask what I consider to be the crucial > > > question > > > yet: why are we locking ntpd into memory by default at all? > > > > > > I have seen two rationales for ntpd using mlockall() and > > > setrlimit(): > > > > > > - There are claims that it improves timing performance. > > > > > > - Because ntpd is a daemon that can run for months at a time, > > > setting limits on memory and stack growth can help detect and > > > mitigate against memory leak problems in the daemon. > > > > Doesn't locking this memory down also protect ntpd from OOM kills? > > If so that is a MUST preserve functionality, as IMHO killing ntpd > > on a box that has it configured is a total no win situation. > > > > Does it have that effect? I don't know. But I would argue that that's > a separate issue, and we should make that happen by adding > ntpd_oomprotect=YES to /etc/defaults/rc.conf Wiring process memory has no effect on OOM selection. More, because all potentially allocated pages are allocated for real after mlockall(), the size of the vmspace, as accounted by OOM, is the largest possible size from the whole lifetime. On the other hand, the code execution times are not predictable if the process's pages can be paged out. Under severe load next instruction might take several seconds or even minutes to start. It is quite unlike the scheduler delays. That introduces a jitter in the local time measurements and their usage as done in userspace. Wouldn't this affect the accuracy ? > > Right now only syslogd has oomprotect set to YES by default. Maybe > that's a good choice -- once we start declaring one daemon to be more > important than others, you'll discover there's a whole back lot full of > bikesheds that need painting. > > So maybe we should just document ntpd_oomprotect=YES in some more- > prominent way. If we were to add a comment block to ntp.conf > describing rlimit, that might be a good place to mention setting > ntpd_oomprotect in rc.conf. > > -- Ian > From owner-freebsd-current@freebsd.org Mon Sep 9 19:21:48 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9258ADCB6B for ; Mon, 9 Sep 2019 19:21:48 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46RyhW3C97z4RbJ; Mon, 9 Sep 2019 19:21:47 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA id 7PEKitZ6asAGk7PEMiQHYZ; Mon, 09 Sep 2019 13:21:44 -0600 X-Authority-Analysis: v=2.3 cv=WeVylHpX c=1 sm=1 tr=0 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17 a=kj9zAlcOel0A:10 a=J70Eh1EUuV4A:10 a=6I5d2MoRAAAA:8 a=YxBL1-UpAAAA:8 a=DdwSh0sFBUtyfsAMNqAA:9 a=0_iv0QpZzRoiHx9C:21 a=HJwVhIzJ0gusrvkh:21 a=CjuIK1q_8ugA:10 a=IjZwj45LgO3ly-622nXo:22 a=Ia-lj3WSrqcvXOmTRaiG:22 Received: from slippy.cwsent.com (slippy8 [10.2.2.6]) by spqr.komquats.com (Postfix) with ESMTPS id 97C18A99; Mon, 9 Sep 2019 12:21:39 -0700 (PDT) Received: from slippy.cwsent.com (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id x89JLdLp051527; Mon, 9 Sep 2019 12:21:39 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Received: from slippy (cy@localhost) by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id x89JLcBv051522; Mon, 9 Sep 2019 12:21:38 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <201909091921.x89JLcBv051522@slippy.cwsent.com> X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Ian Lepore cc: Cy Schubert , Konstantin Belousov , Harlan Stenn , Vladimir Zakharov , freebsd-current@freebsd.org Subject: Re: ntpd segfaults on start In-reply-to: <66f80012757134b6317b673f9eeb24db66c996a2.camel@freebsd.org> References: <201909051307.x85D7MGs034053@slippy.cwsent.com> <20190905142817.GB2559@kib.kiev.ua> <201909060355.x863tRhP089169@slippy.cwsent.com> <201909060639.x866dJ7f090176@slippy.cwsent.com> <201909062356.x86NuKdk003780@slippy.cwsent.com> <156d1e7c-0dbb-8707-90b3-13ae97c87449@nwtime.org> <20190907075619.GG2559@kib.kiev.ua> <201909071309.x87D9GxZ089964@slippy.cwsent.com> <20190907153226.GI2559@kib.kiev.ua> <201909071545.x87FjLS6004448@slippy.cwsent.com> <20190907161749.GJ2559@kib.kiev.ua> <201909071628.x87GSFPV005827@slippy.cwsent.com> <66f80012757134b6317b673f9eeb24db66c996a2.camel@freebsd.org> Comments: In-reply-to Ian Lepore message dated "Mon, 09 Sep 2019 10:23:40 -0600." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 09 Sep 2019 12:21:38 -0700 X-CMAE-Envelope: MS4wfJtQIxh5ZXCHWA13c9h8NEFmOVKf05hF8UW4wVpcTjWQ+osI1B3orEhQr1RZqzteT6LpKEaU6LE/N72UrXmemDrdL1FVw7225J4NN0bMTgPpjpIo6p7z QD4IqtxawWrfG6z8ts+52npNW8x+49bVZg8V4E6wSZxzlEthzofuU1kDEwSkb3epeWptH8FNa8GWmy2XWOGInajVgDrf/idHj3skObBh1FziaMPkSdtr2MRD x+3zU4hiX8uuWMzK2uLk7g== X-Rspamd-Queue-Id: 46RyhW3C97z4RbJ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of cy.schubert@cschubert.com has no SPF policy when checking 64.59.134.13) smtp.mailfrom=cy.schubert@cschubert.com X-Spamd-Result: default: False [-4.04 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; HAS_XAW(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_FIVE(0.00)[6]; REPLYTO_EQ_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[13.134.59.64.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-2.44)[ip: (-6.65), ipnet: 64.59.128.0/20(-3.08), asn: 6327(-2.39), country: CA(-0.09)]; RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 19:21:48 -0000 In message <66f80012757134b6317b673f9eeb24db66c996a2.camel@freebsd.org>, Ian Le pore writes: > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote: > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin Belousov writes: > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote: > > > > I've been able to set the memlock rlimit as low as 20 MB. The issue is > > > > letting it default to 0 which allows ntp to mlockall() anything it want > s. > > > > ntpd on my sandbox is currently using 18 MB. > > > > > > Default stack size on amd64 is 512M, and default stack gap percentage is > > > 3%. This means that the gap can be as large as ~17MB. If 3MB is enough > > > for the stack of the main thread of ntpd, then fine. > > > > The default stack is 200K, which is also tuneable in ntp.conf. > > > > [...] > > I haven't seen anyone ask what I consider to be the crucial question > yet: why are we locking ntpd into memory by default at all? > > I have seen two rationales for ntpd using mlockall() and setrlimit(): > > - There are claims that it improves timing performance. > > - Because ntpd is a daemon that can run for months at a time, > setting limits on memory and stack growth can help detect and > mitigate against memory leak problems in the daemon. > > I am *highly* skeptical of claims that locking ntpd in memory will > improve timing performance on freebsd (to the point where I'm inclined > to dismiss them out of hand, but I'd be willing to look at any actual > evidence). > > The second point has at least some validity, but would be better > implemented (IMO) by decoupling the address space limit from the act of > locking down memory, and allowing configuration of RLIMIT_AS separately > from RLIMIT_MEMLOCK. If there's any interest in this, I could try to > put together a patchset and submit it upstream for that. Our upstream is already cc'd on this thread. diff --git a/usr.sbin/ntp/config.h b/usr.sbin/ntp/config.h index 56dbfedba6e..b26dd417885 100644 --- a/usr.sbin/ntp/config.h +++ b/usr.sbin/ntp/config.h @@ -287,7 +287,7 @@ #define DEFAULT_HZ 100 /* Default number of megabytes for RLIMIT_MEMLOCK */ -#define DFLT_RLIMIT_MEMLOCK 32 +#define DFLT_RLIMIT_MEMLOCK -1 /* Default number of 4k pages for RLIMIT_STACK */ #define DFLT_RLIMIT_STACK 50 But I'd wait for Harlan to weigh in on this. I agree with kib@ that this may introduce unwanted jitter. I'd also want to understand why this defaults to -1 for Linux only. > > I would propose that for freebsd, the autoconf-generated value for For the port but in base we control this directly. > DFLT_RLIMIT_MEMLOCK should be -1 to avoid calling setrlimit() and > mlockall() by default. Then in the ntp.conf we distribute we have a > section like: > > # To lock ntpd into memory, uncomment the following rlimit line. > # This should not be necessary on most systems, but may improve > # performance on heavily-loaded servers experiencing enough > # memory pressure to cause ntpd to be paged out. > # rlimit memlock stacksize > > Then we would need to come up with reasonable values for . I haven't had a chance to look at this in depth yet but I suspect that isn't in fact 32 as specified in config.h. It behaves as if this is set to 0 to mlockall() all it asks for. > > -- Ian -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. From owner-freebsd-current@freebsd.org Mon Sep 9 19:28:43 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 08429DCF66 for ; Mon, 9 Sep 2019 19:28:43 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-so.shaw.ca (smtp-out-so.shaw.ca [64.59.136.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46RyrV0BJqz4S8k; Mon, 9 Sep 2019 19:28:41 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA id 7PL2i1PpQIhW97PL3i2KAX; Mon, 09 Sep 2019 13:28:39 -0600 X-Authority-Analysis: v=2.3 cv=FcFJO626 c=1 sm=1 tr=0 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17 a=kj9zAlcOel0A:10 a=J70Eh1EUuV4A:10 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=dGt7y4mFooTMp67Xen0A:9 a=7Zwj6sZBwVKJAoWSPKxL6X1jA+E=:19 a=CjuIK1q_8ugA:10 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 Received: from slippy.cwsent.com (slippy8 [10.2.2.6]) by spqr.komquats.com (Postfix) with ESMTPS id E8A6AAC0; Mon, 9 Sep 2019 12:28:35 -0700 (PDT) Received: from slippy.cwsent.com (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id x89JSZ23062485; Mon, 9 Sep 2019 12:28:35 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Received: from slippy (cy@localhost) by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id x89JSZMm062482; Mon, 9 Sep 2019 12:28:35 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <201909091928.x89JSZMm062482@slippy.cwsent.com> X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Konstantin Belousov cc: Ian Lepore , "Rodney W. Grimes" , Cy Schubert , Harlan Stenn , Vladimir Zakharov , freebsd-current@freebsd.org Subject: Re: ntpd segfaults on start In-reply-to: <20190909184446.GU2559@kib.kiev.ua> References: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net> <20190909184446.GU2559@kib.kiev.ua> Comments: In-reply-to Konstantin Belousov message dated "Mon, 09 Sep 2019 21:44:46 +0300." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 09 Sep 2019 12:28:35 -0700 X-CMAE-Envelope: MS4wfCNdLMMFu3T+xzAFxBXSvUKj0SzhkIIgldO5/QnxhzgxjV1wEHU17Yd6R+/KmL5Va0LZ3UcIP5BjQKQrofbyr2fZJiLiYiwEJaBkTpxZhzIUqBQS+Wqp LDmk5oNVllnFT3wYus0ZKaGHCramev9Qvdzu0dbLzYiBn0pyokfo5zBVnE20zs83lO4H3rTduhOMCkf9ZQSgcqKOTjZ9Tfn1iM64ZN1ymO/cuz6D9H2I2tr/ 2BAyflQHGM3LlnBS/VofA9mvzkv6GsLses6ss/RmcaquqNYRe5JGiE60mxVH+HCZ X-Rspamd-Queue-Id: 46RyrV0BJqz4S8k X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of cy.schubert@cschubert.com has no SPF policy when checking 64.59.136.138) smtp.mailfrom=cy.schubert@cschubert.com X-Spamd-Result: default: False [-2.42 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; HAS_XAW(0.00)[]; RCPT_COUNT_SEVEN(0.00)[7]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-2.32)[ip: (-6.06), ipnet: 64.59.128.0/20(-3.08), asn: 6327(-2.39), country: CA(-0.09)]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA]; MIME_TRACE(0.00)[0:+]; RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_COUNT_FIVE(0.00)[5]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[138.136.59.64.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 19:28:43 -0000 In message <20190909184446.GU2559@kib.kiev.ua>, Konstantin Belousov writes: > On Mon, Sep 09, 2019 at 12:13:24PM -0600, Ian Lepore wrote: > > On Mon, 2019-09-09 at 09:30 -0700, Rodney W. Grimes wrote: > > > > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote: > > > > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin > > > > > Belousov writes: > > > > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote: > > > > > > > I've been able to set the memlock rlimit as low as 20 MB. The > > > > > > > issue is > > > > > > > letting it default to 0 which allows ntp to mlockall() > > > > > > > anything it wants. > > > > > > > ntpd on my sandbox is currently using 18 MB. > > > > > > > > > > > > Default stack size on amd64 is 512M, and default stack gap > > > > > > percentage is > > > > > > 3%. This means that the gap can be as large as ~17MB. If 3MB is > > > > > > enough > > > > > > for the stack of the main thread of ntpd, then fine. > > > > > > > > > > The default stack is 200K, which is also tuneable in ntp.conf. > > > > > > > > > > [...] > > > > > > > > I haven't seen anyone ask what I consider to be the crucial > > > > question > > > > yet: why are we locking ntpd into memory by default at all? > > > > > > > > I have seen two rationales for ntpd using mlockall() and > > > > setrlimit(): > > > > > > > > - There are claims that it improves timing performance. > > > > > > > > - Because ntpd is a daemon that can run for months at a time, > > > > setting limits on memory and stack growth can help detect and > > > > mitigate against memory leak problems in the daemon. > > > > > > Doesn't locking this memory down also protect ntpd from OOM kills? > > > If so that is a MUST preserve functionality, as IMHO killing ntpd > > > on a box that has it configured is a total no win situation. > > > > > > > Does it have that effect? I don't know. But I would argue that that's > > a separate issue, and we should make that happen by adding > > ntpd_oomprotect=YES to /etc/defaults/rc.conf > Wiring process memory has no effect on OOM selection. More, because > all potentially allocated pages are allocated for real after mlockall(), > the size of the vmspace, as accounted by OOM, is the largest possible > size from the whole lifetime. > > On the other hand, the code execution times are not predictable if the > process's pages can be paged out. Under severe load next instruction > might take several seconds or even minutes to start. It is quite unlike > the scheduler delays. That introduces a jitter in the local time > measurements and their usage as done in userspace. Wouldn't this affect > the accuracy ? IMO it would and would be unacceptable when used as a stratum N server or with some OLTP or database applications. Locking a few megabytes is a small cost. > > > > > Right now only syslogd has oomprotect set to YES by default. Maybe > > that's a good choice -- once we start declaring one daemon to be more > > important than others, you'll discover there's a whole back lot full of > > bikesheds that need painting. > > > > So maybe we should just document ntpd_oomprotect=YES in some more- > > prominent way. If we were to add a comment block to ntp.conf > > describing rlimit, that might be a good place to mention setting > > ntpd_oomprotect in rc.conf. > > > > -- Ian > > -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. From owner-freebsd-current@freebsd.org Mon Sep 9 20:14:18 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 330FBDF17F for ; Mon, 9 Sep 2019 20:14:18 +0000 (UTC) (envelope-from neel@neelc.org) Received: from nyc1.neelc.org (nyc1.neelc.org [IPv6:2a0d:5600:33:11::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46Rzs51zZQz4ZM9 for ; Mon, 9 Sep 2019 20:14:16 +0000 (UTC) (envelope-from neel@neelc.org) Received: from mail.neelc.org (nyc1.neelc.org [IPv6:2a0d:5600:33:11::2]) by nyc1.neelc.org (Postfix) with ESMTPSA id 32D7914D3E5 for ; Mon, 9 Sep 2019 16:07:39 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Mon, 09 Sep 2019 16:07:39 -0400 From: Neel Chauhan To: freebsd-current@freebsd.org Subject: TSC-low clock slow on WhiskeyLake i7-8565U (HP Spectre x360 13-ap0043dx) Message-ID: <72978644df330013de27c75e6285ab4d@neelc.org> X-Sender: neel@neelc.org User-Agent: Roundcube Webmail/1.3.9 X-Rspamd-Queue-Id: 46Rzs51zZQz4ZM9 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of neel@neelc.org designates 2a0d:5600:33:11::2 as permitted sender) smtp.mailfrom=neel@neelc.org X-Spamd-Result: default: False [-2.30 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-0.998,0]; DMARC_NA(0.00)[neelc.org]; IP_SCORE(-0.11)[asn: 63473(-0.50), country: US(-0.05)]; RCVD_COUNT_ONE(0.00)[1]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:63473, ipnet:2a0d:5600:33::/48, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; ONCE_RECEIVED(0.10)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 20:14:18 -0000 Hi, I recently got a HP Spectre x360 13-ap0043dx as a gift and by default the clock runs slower on FreeBSD than Windows or Linux. Yes, I know the ThinkPad is the best laptop for FreeBSD, but getting an X1 Carbon would increase the price of the gift even more which couldn't be done. The kern.timecounter.hardware by default is set to TSC-low and the clock is slow on the Spectre x360. Setting it to ACPI-slow resolves this issue. The CPU is a Intel WhiskeyLake Core i7-8565U. Is the slow TSC-low an issue with WhiskeyLake in general or specifically HP? Is it something else? I am considering writing a patch but before I write one, do other people with WhiskeyLake laptops (or newer) have slow clocks (where one second on the system is actually more in real life), or not. -Neel === https://www.neelc.org/ From owner-freebsd-current@freebsd.org Mon Sep 9 21:12:24 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D6B01E175F for ; Mon, 9 Sep 2019 21:12:24 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound2m.ore.mailhop.org (outbound2m.ore.mailhop.org [54.149.155.156]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46S1883K7mz4fd9 for ; Mon, 9 Sep 2019 21:12:24 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1568063543; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=oqbi0IAv/cKwJMQFruTgPztIVmNBTbTr9ZidypcZf2YRLTA8Lm4B+7F3Zq7x0fh7yp/VXnihPeFll 40iyxUuymg+TPEZaIWXJzlHaDcWdIuwW8EKnekgvf7VdDEHlAUC/OfYbrf3QTNAQTmbwSk05443JGs Iw5J7XO9hxMjUuximoK8HWEDm+CYoNC9+8h1VZqz1xRyKZVFdGTiMqnYpl70PrO5U50LmJol1wGeAY 4EubSgMZTdNpkYHxPwljT6kbOSm82bDppJCBQZY1nyph/lkxJey5COYWG/p9g0mUKHY0lbyyPSF4Cd xGxD7yfrV7ghdWNaIvQgUMAcyCHL07g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=Bv2ydlIUZ+kCz/pyFE3b2tNML0DRTgLRmiKYZUrQKvk=; b=Ggl4wP5AJLe9h6yE3WNlbv0DwvxlJ1NlfcVttSosn4LatykkN1GhcoZbtU1xv9anmXO5S4ljCHtO3 qu013Qt+1HKJqr1ue53Gxx8xmj5ukr/amhpgdM4gZw5srH+R0WvFWpcuZmSv1FRPi/P5+/mIQvkpRJ tfXvKNfwQDWiCFjU3xsaiBLJubVqL+BUJS0hruPFXpxX/tFdsKT3gkjIuHsvI6ar/lo740uaM1cvwL lMj5pKUtVYAMlleVmPQaRvcY8QeYFNJ+npxnrghH3kl1ZpimBZPkUdIK+U8vUYj4i6/WZg1YCAkDkF 4c9gs8pPTXznOXYOI6pb7//yNK81TOA== ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=Bv2ydlIUZ+kCz/pyFE3b2tNML0DRTgLRmiKYZUrQKvk=; b=H17egwVEMpkmRlxWj+DYCZsPcoVOZXK636hAGZWnJkYTGEmuJamQCamzOLTeAhMeJdooAq3ZeDLoJ 1RjbRbVQPktOqplLbecgIDcbM0L5kyaHjFg5NdN52llusnQq3/9w6MoTVUlcX6BXgc5yyuDKdku7ma 4b5Y2HcmJgRbU5Gs6jedJaJ683qQjpaz6Ys1XNPzBSZ+xkpFk2quvcaZhfB5pE9kf3zGuwrDFI4rfJ hiKSpoVU9nLUuInIJ9VOD5G954rsWAidE+av9Bo3p7XdG4XhUMh1WARf+O+JbH4AwjSFdlg12QDnQV TKkala8PPUvJN/YzKYt9yypPWd8Y8LQ== X-MHO-RoutePath: aGlwcGll X-MHO-User: 82052336-d346-11e9-85ed-13b9aae3a1d2 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound4.ore.mailhop.org (Halon) with ESMTPSA id 82052336-d346-11e9-85ed-13b9aae3a1d2; Mon, 09 Sep 2019 21:12:21 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x89LCJk0063759; Mon, 9 Sep 2019 15:12:19 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: Subject: Re: ntpd segfaults on start From: Ian Lepore To: Konstantin Belousov Cc: "Rodney W. Grimes" , Cy Schubert , Harlan Stenn , Vladimir Zakharov , freebsd-current@freebsd.org Date: Mon, 09 Sep 2019 15:12:18 -0600 In-Reply-To: <20190909184446.GU2559@kib.kiev.ua> References: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net> <20190909184446.GU2559@kib.kiev.ua> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46S1883K7mz4fd9 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-2.00 / 15.00]; TAGGED_RCPT(0.00)[]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; ASN(0.00)[asn:16509, ipnet:54.148.0.0/15, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 21:12:24 -0000 On Mon, 2019-09-09 at 21:44 +0300, Konstantin Belousov wrote: > On Mon, Sep 09, 2019 at 12:13:24PM -0600, Ian Lepore wrote: > > On Mon, 2019-09-09 at 09:30 -0700, Rodney W. Grimes wrote: > > > > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote: > > > > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin > > > > > Belousov writes: > > > > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert > > > > > > wrote: > > > > > > > [...] > > > > > > Doesn't locking this memory down also protect ntpd from OOM kills? > > > If so that is a MUST preserve functionality, as IMHO killing ntpd > > > on a box that has it configured is a total no win situation. > > > > > > > Does it have that effect? I don't know. But I would argue that that's > > a separate issue, and we should make that happen by adding > > ntpd_oomprotect=YES to /etc/defaults/rc.conf > > Wiring process memory has no effect on OOM selection. More, because > all potentially allocated pages are allocated for real after mlockall(), > the size of the vmspace, as accounted by OOM, is the largest possible > size from the whole lifetime. > > On the other hand, the code execution times are not predictable if the > process's pages can be paged out. Under severe load next instruction > might take several seconds or even minutes to start. It is quite unlike > the scheduler delays. That introduces a jitter in the local time > measurements and their usage as done in userspace. Wouldn't this affect > the accuracy ? > IMO, there is a large gap between "in theory, paging could cause indeterminate delays in code execution" and "time will be inaccurate on your system". If there were a delay in a part of the code where it matters that amounted to "seconds or even minutes", what you'd end up with is a measurement that would be discarded by the median filter as an outlier. There would be some danger that if that kind of delay happened for too many polling cycles in a row, you'd end up with no usable measurements after a while and clock accuracy would suffer. Sub-second delays would be more worriesome because they might not be rejected as outliers. There are only a couple code paths in freebsd ntpd processing where a paging (or scheduling) delay could cause measurement inaccuracy: - When stepping the clock, the code that runs between calling clock_gettime() and calling clock_settime() to apply the step adjustment to the clock. - When beginning an exchange with or replying to a peer, the code that runs between obtaining system time for the outgoing Transmit Timestamp and actually transmitting that packet. Stepping the clock typically only happens once at startup. The ntpd code itself recognizes that this is a time-critical path (it has comments to that effect) but unfortunately the code that runs is scattered among several different .c files so it's hard to say what the likelyhood is that code in the critical section will all be in the same page (or be already-resident because other startup-time code faulted in those pages). IMO, the right fix for this would be a kernel interface that let you apply a step-delta to the clock with a single syscall (perhaps as an extension to the existing ntp_adjtime() using a new mode flag). On freebsd, the Receive timestamps are captured in the kernel and delivered along with the packet to userland, and are retrieved by the ntpd code from the SCM_BINTIME control message in the packet, so there is no latency problem in the receive path. There isn't a corresponding kernel mechanism for setting the outgoing timestamps, so whether it's originating a request to a peer or replying to a request from a peer, the transmit timestamp could be wrong due to: - paging delays - scheduler delays - network stack, outgoing queues, and driver delays So the primary vulnerability is on the transmit path between obtaining system time and the packet leaving the system. A quick glance at that code makes me think that most of the data being touched has already been referenced pretty recently during the process of assembling the outgoing packet, so it's unlikely that storing the timestamp into the outgoing packet or the other bit of work that happens after that triggers a pagein unless the system is pathologically overloaded. Naturally, obtaining the timestamp and putting it into the packet is one of the last things it does before sending, so the code path is relatively short, but it's not clear to me whether it's likely or not that the code involved all lives in the same page. Still, it's one of the heavily exercised paths within ntpd, which should increase the odds of the pages being resident because of recent use. So, I'm not disputing the point that a sufficiently overloaded system can lead to an indeterminate delay between *any* two instructions executed in userland. What I've said above is more along the lines of considering the usual situation, not the most pathlogical one. In the most pathological cases, either the delays introduced are fairly minor and you get some minor jitter in system time (ameliorated by the median filtering built in to ntpd), or the delays are major (a full second or more) and get rejected as outliers, not affecting system time at all unless the situation persists and prevents getting any good measurements for many hours. -- Ian From owner-freebsd-current@freebsd.org Mon Sep 9 21:17:00 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CFD29E19A5 for ; Mon, 9 Sep 2019 21:17:00 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound3d.ore.mailhop.org (outbound3d.ore.mailhop.org [54.186.57.195]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46S1FS33sJz4fxD for ; Mon, 9 Sep 2019 21:17:00 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1568063818; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=AvOJBCLAJm/BOLWBjnzW9nIxnLi6q5W1nY5M3K4flQmsDdoESPci2bzqKcXrOZ8Tm/WkRMoQQf5KZ rETrMwP4x3kj5g1oO8RPoQGSDYQFvUpT06tWqrkX02BwdHcmLJCv6/YHHZxaWNJVzQMENNGKspFilp GqZjhoaNB4KmaTac1wTfYpp+xn6hhJfu7XgqVqixq4VmHr+1sczc/7Qu2i+ELWJ39V1oQPiNIC4AxI ml0kPhZU/vdIAy/HiCq0fM7TVW21XsA11FQeQKZwDk8uSAbgcjvjspyZ6qAFGZKknVtiqq8WceyIRL g/gYGCcCa9jy6AimlbNlIMzkQd528PA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=K8JlJv5D8SqwpQDVvyqkrrgFFVQ7wa3mT+zv7Vk5bJA=; b=qvp/zU3IOmsNoHmKWO2o8njhO0tyYRGMo+Sn7dSMjuwNQ/4x2giW++gGC7g2jjD47RYxVUifI5Gej K0jMvQk1lfyi5VNItepTA5pUWQMCSlbQVF2ianGeQ2voLgzvSSpc1rF/NzKQus4vpsRTkrM5p8mUq6 FGOulReOhupzWekwXN1/XJm9ZW7ivfRaOkywHC4IK9VpV8kq155GocvRp8XA/rY95dBa1wrb/Qcr2z tQqzkrOPhaa9YMQqizLsr9hWd2qC3DlnxLAC2py3NCVDKPO8qMz/wcKIQFc7J0KDQwaz4pWV+CSgAJ +bxaDFqdwPCwdKEXPEOD783ArsuWCog== ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=K8JlJv5D8SqwpQDVvyqkrrgFFVQ7wa3mT+zv7Vk5bJA=; b=V5Xxjj01rwxohA69FqToybtkzl5syv4IfWI7y5ph9eg53Tc0c5g7cQFUSaZkCWsj+04MZo5kQHNHQ pbv+IC9+0x6demgpZW+7C1Tuoa45bDmDpKjoXcKAXoCQTTiwiaX/iZUaRvJExAzUyHQCovHqaLYGyy X8rByfL0+CbaQeD2H4PPQ36Qzw5lH6qPYDhrYvLRMQPeV9VwYtnmZt7YphaD/LFvOC4IKHZYbaSPN/ khCevnF9R4TYO9Ja23NX0bC5+yK3lIkmiPCXsCv0h9uT+Nmhuw4X52pJT9bFbIPlLM+JMvInLH/CpS 8h1FMcF27Y4lzOBXuCDEUX3QeGG/duw== X-MHO-RoutePath: aGlwcGll X-MHO-User: 24b2fc5f-d347-11e9-b67d-cdd75d6ce7a8 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound3.ore.mailhop.org (Halon) with ESMTPSA id 24b2fc5f-d347-11e9-b67d-cdd75d6ce7a8; Mon, 09 Sep 2019 21:16:57 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x89LGqjV063776; Mon, 9 Sep 2019 15:16:52 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: Subject: Re: ntpd segfaults on start From: Ian Lepore To: Cy Schubert , Konstantin Belousov Cc: "Rodney W. Grimes" , Harlan Stenn , Vladimir Zakharov , freebsd-current@freebsd.org Date: Mon, 09 Sep 2019 15:16:52 -0600 In-Reply-To: <201909091928.x89JSZMm062482@slippy.cwsent.com> References: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net> <20190909184446.GU2559@kib.kiev.ua> <201909091928.x89JSZMm062482@slippy.cwsent.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46S1FS33sJz4fxD X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-2.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; ASN(0.00)[asn:16509, ipnet:54.186.0.0/15, country:US] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 21:17:00 -0000 On Mon, 2019-09-09 at 12:28 -0700, Cy Schubert wrote: > > > > On the other hand, the code execution times are not predictable if the > > process's pages can be paged out. Under severe load next instruction > > might take several seconds or even minutes to start. It is quite unlike > > the scheduler delays. That introduces a jitter in the local time > > measurements and their usage as done in userspace. Wouldn't this affect > > the accuracy ? > > IMO it would and would be unacceptable when used as a stratum N server or > with some OLTP or database applications. Locking a few megabytes is a small > cost. What I proposed was changing the default to not lock all of ntpd into memory, because I think that would work well for most users. Admins running stratum 1 or 2 servers are probably a bit more savvy about ntp configuration than the average user, and would use the rlimit command in ntp.conf. -- Ian From owner-freebsd-current@freebsd.org Mon Sep 9 21:42:56 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C585FE266F for ; Mon, 9 Sep 2019 21:42:56 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46S1qM60nkz4hWD for ; Mon, 9 Sep 2019 21:42:55 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x842.google.com with SMTP id c9so18128648qth.9 for ; Mon, 09 Sep 2019 14:42:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=1dFVajpxXasov17Y9iIIJ8EAzEXfFSpxDyhlcCvDCQI=; b=wOoxKvJUJ9wvl+EEqu4H7k3ecdBVa4sM+ivLY01og/kDPHo0dia/414bOLScIk+XG8 QVFlwz2MqEPOvuCpEQ34nLiHtH6AIdqnZBN8DzqnmfSydfwn6HZ/guXxLoIGOAEu15z/ DFTpwtGn/1osPxtORk9veJeZgf1khKuZao0TlxC417RUi3JlW9If9KRJqNFRAPlbZxzL 9rCoMKmPlKRgjUtAOZ+iW0efyfzkMDmZf+bFPLt8ZBjdRGzrzH/He9zdSF3EA8P1SS8C KXSmYQcAfvvco0N5iEUiZEu7MslY9QMBUIognOyonsfvp9gdF7A1I5fDNUOzFT43q+V/ 9Glw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=1dFVajpxXasov17Y9iIIJ8EAzEXfFSpxDyhlcCvDCQI=; b=nZBV1Z/CEdOWL3M1zurGQVABIyktYgJ/nGKdLonxGBvLppHgPT9OFd6xb3Qkc2Vi6A MdbcCZPvOvyYLDHwg55rwAqHMNH+RVPNWVLZvBLRl5+FUJC9dX+WAswbw/duwWyC7Lfo vw9cvIn2Q0UBF/yhqePqLAvSyGJjaQfY8v4GMsCf56mju03OKRdLYUXAD4w+WJN2h8tX s8LWWzQoOgkv7+Pl4R0s51tL6KetsC+BKWtEoFYvMH8EElw+IPSxhBUdGk3X4IZBNi+T ullyY6MaAfvSRH7ex9OwvVPnv/qVOUFNFu6S3QhfTtKaiJKJdluMVzApTglA9o98ewxE L0vw== X-Gm-Message-State: APjAAAXqZwPZnaLsZqmeJekHBDjzhJs7F6J8GUwe8q4dAjDdMVsKpq1d Yh8VusF2pEAvq5eFXP7g7+Ka0k3ePmCmf9cVcmvLSw== X-Google-Smtp-Source: APXvYqzYeW2qJ5ZcC2svv/0Q1jJX857T4V6a6ii43l206NAo1GVv1UP/2e+rr3FAV0rh+/Ib2ruvQbK46ste68kyzps= X-Received: by 2002:ac8:760e:: with SMTP id t14mr25774010qtq.175.1568065374445; Mon, 09 Sep 2019 14:42:54 -0700 (PDT) MIME-Version: 1.0 References: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net> <20190909184446.GU2559@kib.kiev.ua> In-Reply-To: From: Warner Losh Date: Mon, 9 Sep 2019 15:42:43 -0600 Message-ID: Subject: Re: ntpd segfaults on start To: Ian Lepore Cc: Konstantin Belousov , "Rodney W. Grimes" , Cy Schubert , Harlan Stenn , Vladimir Zakharov , FreeBSD Current X-Rspamd-Queue-Id: 46S1qM60nkz4hWD X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=wOoxKvJU; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::842) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-1.08 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[2.4.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCPT_COUNT_SEVEN(0.00)[7]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-0.58)[ip: (2.17), ipnet: 2607:f8b0::/32(-2.74), asn: 15169(-2.26), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; SUSPICIOUS_RECIPS(1.50)[]; FREEMAIL_CC(0.00)[gmail.com] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2019 21:42:56 -0000 On Mon, Sep 9, 2019 at 3:12 PM Ian Lepore wrote: > On Mon, 2019-09-09 at 21:44 +0300, Konstantin Belousov wrote: > > On Mon, Sep 09, 2019 at 12:13:24PM -0600, Ian Lepore wrote: > > > On Mon, 2019-09-09 at 09:30 -0700, Rodney W. Grimes wrote: > > > > > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote: > > > > > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin > > > > > > Belousov writes: > > > > > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert > > > > > > > wrote: > > > > > > > > [...] > > > > > > > > Doesn't locking this memory down also protect ntpd from OOM kills? > > > > If so that is a MUST preserve functionality, as IMHO killing ntpd > > > > on a box that has it configured is a total no win situation. > > > > > > > > > > Does it have that effect? I don't know. But I would argue that that's > > > a separate issue, and we should make that happen by adding > > > ntpd_oomprotect=YES to /etc/defaults/rc.conf > > > > Wiring process memory has no effect on OOM selection. More, because > > all potentially allocated pages are allocated for real after mlockall(), > > the size of the vmspace, as accounted by OOM, is the largest possible > > size from the whole lifetime. > > > > On the other hand, the code execution times are not predictable if the > > process's pages can be paged out. Under severe load next instruction > > might take several seconds or even minutes to start. It is quite unlike > > the scheduler delays. That introduces a jitter in the local time > > measurements and their usage as done in userspace. Wouldn't this affect > > the accuracy ? > > > > IMO, there is a large gap between "in theory, paging could cause > indeterminate delays in code execution" and "time will be inaccurate on > your system". If there were a delay in a part of the code where it > matters that amounted to "seconds or even minutes", what you'd end up > with is a measurement that would be discarded by the median filter as > an outlier. There would be some danger that if that kind of delay > happened for too many polling cycles in a row, you'd end up with no > usable measurements after a while and clock accuracy would suffer. > Sub-second delays would be more worriesome because they might not be > rejected as outliers. > > There are only a couple code paths in freebsd ntpd processing where a > paging (or scheduling) delay could cause measurement inaccuracy: > > - When stepping the clock, the code that runs between calling > clock_gettime() and calling clock_settime() to apply the step > adjustment to the clock. > > - When beginning an exchange with or replying to a peer, the code that > runs between obtaining system time for the outgoing Transmit Timestamp > and actually transmitting that packet. > > Stepping the clock typically only happens once at startup. The ntpd > code itself recognizes that this is a time-critical path (it has > comments to that effect) but unfortunately the code that runs is > scattered among several different .c files so it's hard to say what the > likelyhood is that code in the critical section will all be in the same > page (or be already-resident because other startup-time code faulted in > those pages). IMO, the right fix for this would be a kernel interface > that let you apply a step-delta to the clock with a single syscall > (perhaps as an extension to the existing ntp_adjtime() using a new mode > flag). > > On freebsd, the Receive timestamps are captured in the kernel and > delivered along with the packet to userland, and are retrieved by the > ntpd code from the SCM_BINTIME control message in the packet, so there > is no latency problem in the receive path. There isn't a corresponding > kernel mechanism for setting the outgoing timestamps, so whether it's > originating a request to a peer or replying to a request from a peer, > the transmit timestamp could be wrong due to: > > - paging delays > - scheduler delays > - network stack, outgoing queues, and driver delays > > So the primary vulnerability is on the transmit path between obtaining > system time and the packet leaving the system. A quick glance at that > code makes me think that most of the data being touched has already > been referenced pretty recently during the process of assembling the > outgoing packet, so it's unlikely that storing the timestamp into the > outgoing packet or the other bit of work that happens after that > triggers a pagein unless the system is pathologically overloaded. > Naturally, obtaining the timestamp and putting it into the packet is > one of the last things it does before sending, so the code path is > relatively short, but it's not clear to me whether it's likely or not > that the code involved all lives in the same page. Still, it's one of > the heavily exercised paths within ntpd, which should increase the odds > of the pages being resident because of recent use. > > So, I'm not disputing the point that a sufficiently overloaded system > can lead to an indeterminate delay between *any* two instructions > executed in userland. What I've said above is more along the lines of > considering the usual situation, not the most pathlogical one. In the > most pathological cases, either the delays introduced are fairly minor > and you get some minor jitter in system time (ameliorated by the median > filtering built in to ntpd), or the delays are major (a full second or > more) and get rejected as outliers, not affecting system time at all > unless the situation persists and prevents getting any good > measurements for many hours. > I've read through all this and agree with it as well. Paging delays can happen, but if they do who cares: the measurements will be rejected as outliers for long delays, but might introduce some noise if the delay is on the order of tens of milliseconds. Shorter may won't matter long term: they will average out. Longer will definitely be rejected. And it will likely be just for the first packet in the exchange since the code path will be paged/swapped into the working set for that. The loop is a combination of PLL/FLL so that if we think there's a big phase step, we'll also think there's a frequency error. Both are steered out the same way: by setting the offset. But ntpd is wise enough to know that there will be noisy measurements, so even if the measurements make it through the filters, we only remove a portion of the error each polling interval anyway. Any over or undershoot will be correct the next measurement interval. And if you have so much memory pressure that ntpd is paged out every single measurement interval, your system likely needs careful tuning anyway. In days of yore (like the mid 90s), these defaults were setup when it took tens or even hundreds of milliseconds to page in it mattered. Today those numbers are submillisecond to single digit milliseconds, so in the typical case the disruption is much less severe than it was when things were initially locked into memory. I'm guessing that's why Linux has moved to -1 for default. In addition, ntpd's algorithms have improved somewhat since then as well to cope with noise. These days, good ntpd performance is submillisecond over the internet, so the noise has to be approximately on that order to affect the filters in ntpd. On the LAN you're good to 10's of microseconds, typically, so any noise > several 10's of microseconds would be eliminated as an outlier anyway. I strongly suspect careful measurements will declare no difference in performance except, maybe, on the most overloaded of servers (and then it would need to be extremely overloaded to have a delay in scheduling often enough to matter). For people that want to be sure, by all means lock it into memory. But as a default, I'm extremely skeptical one could measure a difference, at all, let alone measure a difference that matters. Warner From owner-freebsd-current@freebsd.org Tue Sep 10 03:25:23 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B35ABEDC1A for ; Tue, 10 Sep 2019 03:25:23 +0000 (UTC) (envelope-from rebecca@bsdio.com) Received: from muon.bsdio.com (muon.bluestop.org [65.103.231.193]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46S9QV4Yp1z3K9J for ; Tue, 10 Sep 2019 03:25:22 +0000 (UTC) (envelope-from rebecca@bsdio.com) Received: from muon.bsdio.com (localhost [127.0.0.1]) by muon.bsdio.com (Postfix) with ESMTP id 5EDCF108956 for ; Mon, 9 Sep 2019 21:25:27 -0600 (MDT) Received: from muon.bsdio.com ([127.0.0.1]) by muon.bsdio.com (muon.bsdio.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 5nxINbmbwiN7 for ; Mon, 9 Sep 2019 21:25:26 -0600 (MDT) Received: from photon.int.bluestop.org (unknown [10.0.10.120]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by muon.bsdio.com (Postfix) with ESMTPSA for ; Mon, 9 Sep 2019 21:25:26 -0600 (MDT) To: FreeBSD Current From: Rebecca Cran Subject: Building world with gcc9 not working Autocrypt: addr=rebecca@bsdio.com; keydata= mQINBFrUMZ4BEADI1yUEGeZeXeTCPay1ZpTBdDEpGPAw1dq2VCSTc1VhsnrEBa1iZxAfaeSv Uu5Ti7jlhQ/3sQMl0bJMKGB/RtmIW7k8h2w476oZmG8gChk8su5ZEx/pV1gdqInyFmmJKTYc gabJz8pL+m82w07qPv+oalepZ4dbj+HF++RAK/iEju+q9UHlsjj8e3mMNsvtrOz1K6bnpveO jZ+ms/2H3Hs5a4k8y6buwe2RvwhJQaXa13cR3LhzL+nwj4B9PHZZEa2WpEyYpw/bI0V9YSQN QgC1CYRzDyakZge6BCM6wHOgZSUzRPufGilrNKUwIVbRoIBR9/85+0wR+PlFUOUOfOc6ox7T dWcIx6PuPhek48rh4uwmmwsPtPiH4Z3T5p+GmWQ9NLFZKA1YnEdaSkWtYZsDxwVZZeYG2plt MfhXP0Hj4rf9Y3eoUenCaGioxAbUOBCtXdTGNAhNjz1g5NGDBVyhjKkzwJQvt9UrYTseERit 5dX2CMTy8hYLvSXd/Ivy+HylUS5IslfZxW5z9LgWX7Z97kILgkH3N0ewtLkygkG+Y+x7uaAV dFqp9ASOyzaiwKbJdeOI+WxRSh+AqeCR0S+bpkcLudLmbjrPmaFwjKycy1H85Z5R2J3YHyXY oT6OYjD8vLbUU2GWp6Onkcy1Pu8EMbRuzKil6HnpYg3BexbPFwARAQABtCNSZWJlY2NhIENy YW4gPHJlYmVjY2FAYmx1ZXN0b3Aub3JnPokCVwQTAQgAQQIbIwUJCWYBgAULCQgHAgYVCgkI CwIEFgIDAQIeAQIXgBYhBB+5fZtkTdO940Yr4g0CK1MRvhAgBQJa2B9zAhkBAAoJEA0CK1MR vhAgzJEQAJUqVmTRO9OqCSS2CVKjrqnEWJMvyo0K8B+WiXo0nSQg9+uyoVU7h2s/kkWVGy4u IWbGy2Qe8LiXzBJjHC3TadGvOvakfdMeKKXcgxgX6KlhA9hA2LW6tg22aHUk7Flr/8diHpgf qIwrXhqJXZmK72GR1QfhgoHsOsTJ9GWPswo1kUMc0cJowq0qP1RDdua6BwvDHHPJwu9OmC/i oQlMNm9gkBDq8H2B+m125ANwCnqBizXaiTTLQdewTMbCSuxbsni2icDqwBfFXzEgcJGaYYfB cQeFsfCmtXQK3JUd4Myx128Dxk9P3X64I93SB7QzB0nmWlyvmCFBNoCp0PCLA4qbwbw2sMRX Wx4BqYa8nI/jg+Nqo+Ut2BfltNZIlsHxK+XhxejfLqAjRCZeLnu1otvFnFuGLaAVYx9x1Y1q J8VizZxq6ujio62Qpultp6KNhlkJ+OKoGwA0k4NHh26SxvlsNxlfg/2v9b1LqWRzNujnwbcF 8g4902XjyBLxV+9YpXZEa8H6zzEHxpeDPWT3QfvrT8JuoHa1IyYnUKvG674UKW5zEGEwkQc9 cuQwR1RHd1ZrKtH1duXzaLr/caMp8ZDfGDDxFpenJTRxNRlg4+K7HSdhpac7sBVMUA8uVdE+ iuTThOmdf0c4DorL3BIh6Yv3FV4/NSqT1Wn3CG2fgG1guQINBFrUMZ4BEADkc4mvMcMcDF1t dNxNQuIBE1F243oZamG3LACCKfc1Yur3CPzHwIk5LXCUmbq23iE5bowxMWw3mlVT0p5xM0Wn UidIBwCKu4kRyy/fY4NyWWBuwy9srpTdmUcKRBRNB8zEZE8xIlidD1ijjgqLBfeM7n9ylawA xHLxwU96sdpdHFzb7Z0yKY2e/bzDaHiG0fUvcCmkgLf+uwKKZid1j8zR5PzKpgPqfy/PF01e KyGV3MNu8Y90xMoiEMWfCI2IB1m+hTuzZoboFvGV54SiMuvfWK/VMQjhsL6K2ddOqwVuy2nI MI4G3xDQW/v8KVyn43OSIAyW1eaklhzu0Ir2sO60PXRkvbTUrouvmSvpJfIQS49rU0M/X6FS DgXQLKrZ3my94+g8ptz9KoVml6s4OAwYVz+sb49nuSxipFKkU5FwhKOLmzbsBxCtytcUJoLm juJPJPDQue6YJiIXyc86GVY2pH3DjemKdbB4dSgqAJIp+lCzKSJzz7bgueh2Ox8vzx1tSxKj 7V8Nal+UTKKbkxPmMh+e20YZ4esAVifO3bS6IJP/aDnfagghB71vA7+aWGXPbjPlc2UHpCBi RSsl+IgoQXvdvZBsKRyfBx8neODa2C6JIE5vcaCjilSeKF8SzsFXvimnndhQNhAPU/DwQwSX dCl4gTsFVi5d8Oxq1sce+wARAQABiQI8BBgBCAAmFiEEH7l9m2RN073jRiviDQIrUxG+ECAF AlrUMZ4CGwwFCQlmAYAACgkQDQIrUxG+ECAWnRAAsmZX+KgNxW3v7R/76Tz4Wjmh4AGeE+Ji 3p5QsdTYny1B6vYBL9vCzPJ/AK8pgKMDRaweUP5eZQpfrdWC8Q7SNGgi4Q+97KEs+i2xZLQ+ WJb8a+WEEIc716u0y4ITiHfOgM5jWcFO4MXQATbJgv0drLLesa+LQCvZgPBqupt307EsCubQ s+Sxt+RVjf6rOUolp1GJXEQYwGsKklVd6yqLC8M1BSG53/WE5tSv5GzBZ8fp6EtmjT7leuid FtEvKYHQz4DqG9ELpHUF0X0UUCBK/MgXe3kCVLKE060UrJ4M6uPSx57rmVFA2MvwQR8M7GsW C5UsSM4PYwPWBhwxE7vcx0691YKAHT/5q8LxRVBdUyzPSprMhSQFttsBt+ygm6wRi3Pi3TuC EARNubPkQefyeC34yr40SAUCkOl3eWxSXPF4NfXFQb4AAzZSE5hv3qbDuwo3lrL0LqpIpEQP Az+JZ1QZ6mMFQ5/JD9Gukj54kZc0X8w3sQt0a8vyE/qrJg8vKgv2rCHrPc5MeDkEUEFiiJiC EDdkJtMyoRlU3S4NrnbyLOLEcHE8fGe3hStPX8hY62id2ecdQ5WZ7vLZW5SFeLarbUciuHIk VL6MHnUjbV7XlY50N7ebeFCIdlCWhdum2FJs/Ni+SSxbZC564vrokwlBBGSo6WTPQTa8IWx1 DtU= Message-ID: <4c4d4ccf-4662-572a-d6ff-7ce9a0c36f38@bsdio.com> Date: Mon, 9 Sep 2019 21:25:20 -0600 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 46S9QV4Yp1z3K9J X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of rebecca@bsdio.com has no SPF policy when checking 65.103.231.193) smtp.mailfrom=rebecca@bsdio.com X-Spamd-Result: default: False [-2.95 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[bsdio.com]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:209, ipnet:65.103.224.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(-1.85)[ip: (-9.14), asn: 209(-0.05), country: US(-0.05)] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 03:25:23 -0000 Is building world with gcc still supported? I'm getting an error running the following: make XCC=/usr/local/bin/gcc9 XCXX=/usr/local/bin/g++9 buildworld /usr/local/bin/gcc9 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin  -O2 -pipe   -DNO__SCCSID -DNO__RCSID -I/usr/src/lib/libc/include -I/usr/src/include -I/usr/src/lib/libc/amd64 -DNLS  -D__DBINTERFACE_PRIVATE -I/usr/src/contrib/gdtoa -I/usr/src/contrib/libc-vis -DINET6 -I/usr/obj/usr/src/amd64.amd64/lib/libc -I/usr/src/lib/libc/resolv -D_ACL_PRIVATE -DPOSIX_MISTAKE -I/usr/src/lib/libmd -I/usr/src/contrib/jemalloc/include -DMALLOC_PRODUCTION -I/usr/src/contrib/tzcode/stdtime -I/usr/src/lib/libc/stdtime -I/usr/src/lib/libc/locale -DBROKEN_DES -DPORTMAP -DDES_BUILTIN -I/usr/src/lib/libc/rpc -DWANT_HYPERV -DYP -DNS_CACHING -DSYMBOL_VERSIONING -g -MD  -MF.depend.jemalloc_malloc_io.o -MTjemalloc_malloc_io.o -std=gnu99 -Wno-format-zero-length -fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k -Wno-uninitialized -Wno-pointer-sign -Wno-error=address -Wno-error=array-bounds -Wno-error=attributes -Wno-error=bool-compare -Wno-error=cast-align -Wno-error=clobbered -Wno-error=enum-compare -Wno-error=extra -Wno-error=inline -Wno-error=logical-not-parentheses -Wno-error=strict-aliasing -Wno-error=uninitialized -Wno-error=unused-but-set-variable -Wno-error=unused-function -Wno-error=unused-value -Wno-error=misleading-indentation -Wno-error=nonnull-compare -Wno-error=shift-negative-value -Wno-error=tautological-compare -Wno-error=unused-const-variable -Wno-error=bool-operation -Wno-error=deprecated -Wno-error=expansion-to-defined -Wno-error=format-overflow -Wno-error=format-truncation -Wno-error=implicit-fallthrough -Wno-error=int-in-bool-context -Wno-error=memset-elt-size -Wno-error=noexcept-type -Wno-error=nonnull -Wno-error=pointer-compare -Wno-error=stringop-overflow -Wno-error=aggressive-loop-optimizations -Wno-error=cast-function-type -Wno-error=catch-value -Wno-error=multistatement-macros -Wno-error=restrict -Wno-error=sizeof-pointer-memaccess -Wno-error=stringop-truncation       -I/usr/src/lib/libutil -I/usr/src/lib/msun/amd64 -I/usr/src/lib/msun/x86 -I/usr/src/lib/msun/src -c jemalloc_malloc_io.c -o jemalloc_malloc_io.o jemalloc_malloc_io.c: In function '__je_malloc_vsnprintf': jemalloc_malloc_io.c:383:2: error: case label value exceeds maximum value for type [-Werror]   383 |  case '?' | 0x80:      \       |  ^~~~ jemalloc_malloc_io.c:595:5: note: in expansion of macro 'GET_ARG_NUMERIC'   595 |     GET_ARG_NUMERIC(val, 'p');       |     ^~~~~~~~~~~~~~~ jemalloc_malloc_io.c:401:2: error: case label value exceeds maximum value for type [-Werror]   401 |  case 'j' | 0x80:      \       |  ^~~~ jemalloc_malloc_io.c:595:5: note: in expansion of macro 'GET_ARG_NUMERIC'   595 |     GET_ARG_NUMERIC(val, 'p');       |     ^~~~~~~~~~~~~~~ jemalloc_malloc_io.c:389:2: error: case label value exceeds maximum value for type [-Werror]   389 |  case 'l' | 0x80:      \       |  ^~~~ jemalloc_malloc_io.c:595:5: note: in expansion of macro 'GET_ARG_NUMERIC'   595 |     GET_ARG_NUMERIC(val, 'p');       |     ^~~~~~~~~~~~~~~ jemalloc_malloc_io.c:395:2: error: case label value exceeds maximum value for type [-Werror]   395 |  case 'q' | 0x80:      \       |  ^~~~ jemalloc_malloc_io.c:595:5: note: in expansion of macro 'GET_ARG_NUMERIC'   595 |     GET_ARG_NUMERIC(val, 'p');       |     ^~~~~~~~~~~~~~~ jemalloc_malloc_io.c:410:2: error: case label value exceeds maximum value for type [-Werror]   410 |  case 'z' | 0x80:      \       |  ^~~~ jemalloc_malloc_io.c:595:5: note: in expansion of macro 'GET_ARG_NUMERIC'   595 |     GET_ARG_NUMERIC(val, 'p');       |     ^~~~~~~~~~~~~~~ cc1: all warnings being treated as errors *** Error code 1 -- Rebecca Cran From owner-freebsd-current@freebsd.org Tue Sep 10 05:05:03 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AE037F0EA3 for ; Tue, 10 Sep 2019 05:05:03 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-io1-f45.google.com (mail-io1-f45.google.com [209.85.166.45]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46SCdV3rRwz3Pgl for ; Tue, 10 Sep 2019 05:05:02 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-io1-f45.google.com with SMTP id r4so34583326iop.4 for ; Mon, 09 Sep 2019 22:05:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc:content-transfer-encoding; bh=Lx2Wsm+tG+mRRAukRGgExonOq0DXQJ8qRJiwBuSAQoQ=; b=BF7rS1mAfRaqffs+4Q4lgIJaLGnzI/+y0/AaSXask6cqWojg07rBM9w7XalvDDyijn 14YG1qs11G3ziDP97d7Tmh3SvedvZ7jFxnUiFyDytK0gLIItuuOlpztkHYpCsZWMmH0b RPTjE+DtfRMbATPNzp/2OIBsT+/+68vjW2m/ZRJDRTmZVxjKo+mwau/rFFXdxYVSqqHg Z0zLI/IOmc9uu2b8qfpK8VVsZFiUvhuH9GvaFhiiF69hpVVE15BMsMi1e59STGOuEbv9 pZNwCB7FWkOIXPk8LgLXRsL0qUS8tzxZ16njFHYZM3McRTisOS6sd5VexsWO6SHYEDa+ 1ZAw== X-Gm-Message-State: APjAAAU92bBgITJ6zA8WwF3DZYQqS53Ku9YFc9OjKUwaXVwwol8EiBfT dbTVxJFpwgMf5ZLnRrhLnxfp05I/ X-Google-Smtp-Source: APXvYqztkh/yWf4sK1YW3KvofK2VqOJaZupq7xLF82lDIKeXR3lecMTZKmM3rbr7y07RXMfIJuPYVA== X-Received: by 2002:a6b:dc0e:: with SMTP id s14mr4650766ioc.184.1568091900778; Mon, 09 Sep 2019 22:05:00 -0700 (PDT) Received: from mail-io1-f41.google.com (mail-io1-f41.google.com. [209.85.166.41]) by smtp.gmail.com with ESMTPSA id g66sm6300219ioa.53.2019.09.09.22.05.00 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 09 Sep 2019 22:05:00 -0700 (PDT) Received: by mail-io1-f41.google.com with SMTP id k13so19353216ioj.1 for ; Mon, 09 Sep 2019 22:05:00 -0700 (PDT) X-Received: by 2002:a5d:9856:: with SMTP id p22mr7342621ios.231.1568091899742; Mon, 09 Sep 2019 22:04:59 -0700 (PDT) MIME-Version: 1.0 References: <4c4d4ccf-4662-572a-d6ff-7ce9a0c36f38@bsdio.com> In-Reply-To: <4c4d4ccf-4662-572a-d6ff-7ce9a0c36f38@bsdio.com> Reply-To: cem@freebsd.org From: Conrad Meyer Date: Mon, 9 Sep 2019 22:04:48 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Building world with gcc9 not working To: Rebecca Cran Cc: FreeBSD Current Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 46SCdV3rRwz3Pgl X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of csecem@gmail.com designates 209.85.166.45 as permitted sender) smtp.mailfrom=csecem@gmail.com X-Spamd-Result: default: False [-4.23 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[cem@freebsd.org]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; REPLYTO_ADDR_EQ_FROM(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FORGED_SENDER(0.30)[cem@freebsd.org,csecem@gmail.com]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; TAGGED_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_NEQ_ENVFROM(0.00)[cem@freebsd.org,csecem@gmail.com]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[45.166.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.23)[ip: (-5.50), ipnet: 209.85.128.0/17(-3.32), asn: 15169(-2.26), country: US(-0.05)]; RWL_MAILSPIKE_POSSIBLE(0.00)[45.166.85.209.rep.mailspike.net : 127.0.0.17]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 05:05:03 -0000 On Mon, Sep 9, 2019 at 8:25 PM Rebecca Cran wrote: > > Is building world with gcc still supported? I'm getting an error running > the following: > > make XCC=3D/usr/local/bin/gcc9 XCXX=3D/usr/local/bin/g++9 buildworld Hi Rebecca, Aspirationally, yes. I did a successful CROSS_TOOLCHAIN=3Damd64-gcc buildworld earlier this week. (It doesn't play well with binary pkg's built with Clang, so I ended up replacing it with a Clang-built world instead, but it compiled.) Unfortunately, amd64-xtoolchain-gcc is stuck on GCC 6.4.0, while you're running the much more recent GCC9. I think GCC6.4.0 is more or less expected to build world today, but I don't think many people are building with GCC9. I would love for amd64-xtoolchain to move to a newer version, but I don't know what is blocking that =E2=80=94 it seems li= ke it should be straightforward. Best regards, Conrad From owner-freebsd-current@freebsd.org Tue Sep 10 05:08:56 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CC260F1033 for ; Tue, 10 Sep 2019 05:08:56 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46SCjz57DRz3Pr9 for ; Tue, 10 Sep 2019 05:08:55 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x836.google.com with SMTP id g4so19256002qtq.7 for ; Mon, 09 Sep 2019 22:08:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=2O2WJjPCf9pHLKztQElG+QooeNNCmWs13v7CLFPbCPk=; b=DfS3UfG+uaA2YZfAPcoLXM5tk0xR2B/grkcSbxxPaeRbLgkG6WCMs/sVdq35vw69lA ia/4x5vBDdlx5rUInI1VQ4WEiydgMNfNX0ws+1lgm2lb/Mt8QxtQYcpf6nLrcFx9YO1E L4wWVXls1bwSIWzFOVvBcSDzEAaZYfUgot6nJhSbTxnmUyW7hZ07mm0EznzLgcrcdLko Joe4D6gvTOWY7DIK1ywQBDiJv9xTclrCxzEnZNgQLHuAeEltCZX5bIkPWkADfgV/uVEM 8SdENefGHXNQozuMzNougL7SjLChBJANXdrjkKnBLVD/6Eu5fLT6/AjjnzKktas0F6zv BdkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=2O2WJjPCf9pHLKztQElG+QooeNNCmWs13v7CLFPbCPk=; b=IbjYoMeHLmI49+MvMWLY0iu+jVbvrYTstYYiwObZS0s5qE9t1XNXQ12ksJBliPR3Q5 +CRmT66VK0stAiV0cINZmz88M4s5g37unAn0F37EaZ4BnC5CqnCmMSDg1umIo8DJVK1K uj5TESx0SYYo5wL9gCpoHGVRPrlVRR25Ryb8OI9bHSOrvBHbRSD1wqZhRpgDWCMRGZDW ch9+aic+juBINCJFaDkLl1Y5t6T1g6Z+AvLRgBrCa5hUjDAody1eK71L5JErKXW/lytz LH8FtfRjE59tfDJ77PdfZ+DrrnRoyOeIQTKwDpver//SiDRr707DkrhNIts+djK9LbKl E1Jg== X-Gm-Message-State: APjAAAXypQY6+E2wu+vlXfGMulRIXfkOTtfb9g+3YiuoULgEp72nk62Q NHHOCQphFqHZeXbIfRN8It+j89vLNs6G3v0aVTXt9J4o X-Google-Smtp-Source: APXvYqy4mWjj7+0UwwLTp0cVcSBSi8QQEazT40RARJgPhgRgTxPOndAZ+3sM5XhP8/oGjFYAQXu8EMU3OvSJ16avFDY= X-Received: by 2002:ac8:71cb:: with SMTP id i11mr25868833qtp.32.1568092134544; Mon, 09 Sep 2019 22:08:54 -0700 (PDT) MIME-Version: 1.0 References: <4c4d4ccf-4662-572a-d6ff-7ce9a0c36f38@bsdio.com> In-Reply-To: From: Warner Losh Date: Mon, 9 Sep 2019 23:08:43 -0600 Message-ID: Subject: Re: Building world with gcc9 not working To: "Conrad E. Meyer" Cc: Rebecca Cran , FreeBSD Current X-Rspamd-Queue-Id: 46SCjz57DRz3Pr9 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=DfS3UfG+; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::836) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-4.86 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[6.3.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; IP_SCORE(-2.86)[ip: (-9.26), ipnet: 2607:f8b0::/32(-2.74), asn: 15169(-2.26), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 05:08:56 -0000 On Mon, Sep 9, 2019 at 11:05 PM Conrad Meyer wrote: > On Mon, Sep 9, 2019 at 8:25 PM Rebecca Cran wrote: > > > > Is building world with gcc still supported? I'm getting an error runnin= g > > the following: > > > > make XCC=3D/usr/local/bin/gcc9 XCXX=3D/usr/local/bin/g++9 buildworld > > Hi Rebecca, > > Aspirationally, yes. I did a successful CROSS_TOOLCHAIN=3Damd64-gcc > buildworld earlier this week. (It doesn't play well with binary pkg's > built with Clang, so I ended up replacing it with a Clang-built world > instead, but it compiled.) > > Unfortunately, amd64-xtoolchain-gcc is stuck on GCC 6.4.0, while > you're running the much more recent GCC9. I think GCC6.4.0 is more or > less expected to build world today, but I don't think many people are > building with GCC9. I would love for amd64-xtoolchain to move to a > newer version, but I don't know what is blocking that =E2=80=94 it seems = like > it should be straightforward. > I did a gcc8 build about a year or so ago, though I had to turn off Werror to complete the build... Warner From owner-freebsd-current@freebsd.org Tue Sep 10 05:44:18 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A06F9F1DC4 for ; Tue, 10 Sep 2019 05:44:18 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46SDVn2r2Gz3wmq for ; Tue, 10 Sep 2019 05:44:17 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x8A5i8hB035919 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 10 Sep 2019 08:44:11 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x8A5i8hB035919 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x8A5i7WA035918; Tue, 10 Sep 2019 08:44:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 10 Sep 2019 08:44:07 +0300 From: Konstantin Belousov To: Neel Chauhan Cc: freebsd-current@freebsd.org Subject: Re: TSC-low clock slow on WhiskeyLake i7-8565U (HP Spectre x360 13-ap0043dx) Message-ID: <20190910054407.GW2559@kib.kiev.ua> References: <72978644df330013de27c75e6285ab4d@neelc.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <72978644df330013de27c75e6285ab4d@neelc.org> User-Agent: Mutt/1.12.1 (2019-06-15) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-Rspamd-Queue-Id: 46SDVn2r2Gz3wmq X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all:c]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(0.00)[ip: (-2.58), ipnet: 2001:470::/32(-4.46), asn: 6939(-3.17), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[gmail.com]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 05:44:18 -0000 On Mon, Sep 09, 2019 at 04:07:39PM -0400, Neel Chauhan wrote: > Hi, > > I recently got a HP Spectre x360 13-ap0043dx as a gift and by default > the clock runs slower on FreeBSD than Windows or Linux. Yes, I know the > ThinkPad is the best laptop for FreeBSD, but getting an X1 Carbon would > increase the price of the gift even more which couldn't be done. > > The kern.timecounter.hardware by default is set to TSC-low and the clock > is slow on the Spectre x360. Setting it to ACPI-slow resolves this > issue. > > The CPU is a Intel WhiskeyLake Core i7-8565U. > > Is the slow TSC-low an issue with WhiskeyLake in general or specifically > HP? Is it something else? > > I am considering writing a patch but before I write one, do other people > with WhiskeyLake laptops (or newer) have slow clocks (where one second > on the system is actually more in real life), or not. Start with providing full listing dmesg for verbose boot. Out of blue, try to set machdep.disable_tsc_calibration=1 in loader.conf and see if this improves things. From owner-freebsd-current@freebsd.org Tue Sep 10 06:10:05 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E6F17F2563 for ; Tue, 10 Sep 2019 06:10:05 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46SF4Y0rPJz3xvv for ; Tue, 10 Sep 2019 06:10:05 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x8A69oiY041839 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 10 Sep 2019 09:09:53 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x8A69oiY041839 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x8A69nS2041838; Tue, 10 Sep 2019 09:09:49 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 10 Sep 2019 09:09:49 +0300 From: Konstantin Belousov To: Warner Losh Cc: Ian Lepore , "Rodney W. Grimes" , Cy Schubert , Harlan Stenn , Vladimir Zakharov , FreeBSD Current Subject: Re: ntpd segfaults on start Message-ID: <20190910060949.GY2559@kib.kiev.ua> References: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net> <20190909184446.GU2559@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-Rspamd-Queue-Id: 46SF4Y0rPJz3xvv X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-0.50 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; TAGGED_RCPT(0.00)[]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all:c]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_SEVEN(0.00)[7]; IP_SCORE(0.00)[ip: (-2.54), ipnet: 2001:470::/32(-4.46), asn: 6939(-3.16), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[gmail.com]; SUSPICIOUS_RECIPS(1.50)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 06:10:06 -0000 On Mon, Sep 09, 2019 at 03:42:43PM -0600, Warner Losh wrote: > On Mon, Sep 9, 2019 at 3:12 PM Ian Lepore wrote: > > > On Mon, 2019-09-09 at 21:44 +0300, Konstantin Belousov wrote: > > > On Mon, Sep 09, 2019 at 12:13:24PM -0600, Ian Lepore wrote: > > > > On Mon, 2019-09-09 at 09:30 -0700, Rodney W. Grimes wrote: > > > > > > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote: > > > > > > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin > > > > > > > Belousov writes: > > > > > > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert > > > > > > > > wrote: > > > > > > > > > [...] > > > > > > > > > > Doesn't locking this memory down also protect ntpd from OOM kills? > > > > > If so that is a MUST preserve functionality, as IMHO killing ntpd > > > > > on a box that has it configured is a total no win situation. > > > > > > > > > > > > > Does it have that effect? I don't know. But I would argue that that's > > > > a separate issue, and we should make that happen by adding > > > > ntpd_oomprotect=YES to /etc/defaults/rc.conf > > > > > > Wiring process memory has no effect on OOM selection. More, because > > > all potentially allocated pages are allocated for real after mlockall(), > > > the size of the vmspace, as accounted by OOM, is the largest possible > > > size from the whole lifetime. > > > > > > On the other hand, the code execution times are not predictable if the > > > process's pages can be paged out. Under severe load next instruction > > > might take several seconds or even minutes to start. It is quite unlike > > > the scheduler delays. That introduces a jitter in the local time > > > measurements and their usage as done in userspace. Wouldn't this affect > > > the accuracy ? > > > > > > > IMO, there is a large gap between "in theory, paging could cause > > indeterminate delays in code execution" and "time will be inaccurate on > > your system". If there were a delay in a part of the code where it > > matters that amounted to "seconds or even minutes", what you'd end up > > with is a measurement that would be discarded by the median filter as > > an outlier. There would be some danger that if that kind of delay > > happened for too many polling cycles in a row, you'd end up with no > > usable measurements after a while and clock accuracy would suffer. > > Sub-second delays would be more worriesome because they might not be > > rejected as outliers. > > > > There are only a couple code paths in freebsd ntpd processing where a > > paging (or scheduling) delay could cause measurement inaccuracy: > > > > - When stepping the clock, the code that runs between calling > > clock_gettime() and calling clock_settime() to apply the step > > adjustment to the clock. > > > > - When beginning an exchange with or replying to a peer, the code that > > runs between obtaining system time for the outgoing Transmit Timestamp > > and actually transmitting that packet. > > > > Stepping the clock typically only happens once at startup. The ntpd > > code itself recognizes that this is a time-critical path (it has > > comments to that effect) but unfortunately the code that runs is > > scattered among several different .c files so it's hard to say what the > > likelyhood is that code in the critical section will all be in the same > > page (or be already-resident because other startup-time code faulted in > > those pages). IMO, the right fix for this would be a kernel interface > > that let you apply a step-delta to the clock with a single syscall > > (perhaps as an extension to the existing ntp_adjtime() using a new mode > > flag). > > > > On freebsd, the Receive timestamps are captured in the kernel and > > delivered along with the packet to userland, and are retrieved by the > > ntpd code from the SCM_BINTIME control message in the packet, so there > > is no latency problem in the receive path. There isn't a corresponding > > kernel mechanism for setting the outgoing timestamps, so whether it's > > originating a request to a peer or replying to a request from a peer, > > the transmit timestamp could be wrong due to: > > > > - paging delays > > - scheduler delays > > - network stack, outgoing queues, and driver delays > > > > So the primary vulnerability is on the transmit path between obtaining > > system time and the packet leaving the system. A quick glance at that > > code makes me think that most of the data being touched has already > > been referenced pretty recently during the process of assembling the > > outgoing packet, so it's unlikely that storing the timestamp into the > > outgoing packet or the other bit of work that happens after that > > triggers a pagein unless the system is pathologically overloaded. > > Naturally, obtaining the timestamp and putting it into the packet is > > one of the last things it does before sending, so the code path is > > relatively short, but it's not clear to me whether it's likely or not > > that the code involved all lives in the same page. Still, it's one of > > the heavily exercised paths within ntpd, which should increase the odds > > of the pages being resident because of recent use. > > > > So, I'm not disputing the point that a sufficiently overloaded system > > can lead to an indeterminate delay between *any* two instructions > > executed in userland. What I've said above is more along the lines of > > considering the usual situation, not the most pathlogical one. In the > > most pathological cases, either the delays introduced are fairly minor > > and you get some minor jitter in system time (ameliorated by the median > > filtering built in to ntpd), or the delays are major (a full second or > > more) and get rejected as outliers, not affecting system time at all > > unless the situation persists and prevents getting any good > > measurements for many hours. > > > > I've read through all this and agree with it as well. Paging delays can > happen, but if they do who cares: the measurements will be rejected as > outliers for long delays, but might introduce some noise if the delay is on > the order of tens of milliseconds. Shorter may won't matter long term: they > will average out. Longer will definitely be rejected. And it will likely be > just for the first packet in the exchange since the code path will be > paged/swapped into the working set for that. The loop is a combination of > PLL/FLL so that if we think there's a big phase step, we'll also think > there's a frequency error. Both are steered out the same way: by setting > the offset. But ntpd is wise enough to know that there will be noisy > measurements, so even if the measurements make it through the filters, we > only remove a portion of the error each polling interval anyway. Any over > or undershoot will be correct the next measurement interval. And if you > have so much memory pressure that ntpd is paged out every single > measurement interval, your system likely needs careful tuning anyway. > > In days of yore (like the mid 90s), these defaults were setup when it took > tens or even hundreds of milliseconds to page in it mattered. Today those > numbers are submillisecond to single digit milliseconds, so in the typical > case the disruption is much less severe than it was when things were > initially locked into memory. I'm guessing that's why Linux has moved to -1 > for default. In addition, ntpd's algorithms have improved somewhat since > then as well to cope with noise. These days, good ntpd performance is > submillisecond over the internet, so the noise has to be approximately on > that order to affect the filters in ntpd. On the LAN you're good to 10's of > microseconds, typically, so any noise > several 10's of microseconds would > be eliminated as an outlier anyway. I strongly suspect careful > measurements will declare no difference in performance except, maybe, on > the most overloaded of servers (and then it would need to be extremely > overloaded to have a delay in scheduling often enough to matter). > > For people that want to be sure, by all means lock it into memory. But as a > default, I'm extremely skeptical one could measure a difference, at all, > let alone measure a difference that matters. >From the overall system performance, I am all for stopping wiring ntpd. One of the unfortunate consequences of doing that is the wiring of rtld, libc, and libthr. Small note, besides large delays caused by real pageouts (hard faults), there are small jitter-like delays when pagedaemon emulates access and dirty bits on machines which lack them by unmapping still resident pages. The cost of reinstalling the pages is just the cost of locking and calculating PTEs, but locking makes it dependent on other system activities. I think that ARM is the largest suspect there. From owner-freebsd-current@freebsd.org Tue Sep 10 12:37:59 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2111DD5923 for ; Tue, 10 Sep 2019 12:37:59 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from new4-smtp.messagingengine.com (new4-smtp.messagingengine.com [66.111.4.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46SPh61wjlz4PBP for ; Tue, 10 Sep 2019 12:37:57 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailnew.nyi.internal (Postfix) with ESMTP id B40612A7D for ; Tue, 10 Sep 2019 08:37:55 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Tue, 10 Sep 2019 08:37:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yuripv.net; h=to :from:subject:message-id:date:mime-version:content-type :content-transfer-encoding; s=fm1; bh=+v53l4HjxJOtbWb2PL/isgF4la fR8IG/t73vecOIgsE=; b=IqLWHbVaZLa2oufoMj5VmDod4iKJOCgcM/9mpHCdjp mp98UOZ00ngfDVHCuru9WsUGfeuJzmkbR1u7hpnlTpnnAxkCLzGoWTQAFCk2wAEH rkUm4HV4kVuOc2Hpg+VmOPzxPxzninsvIeH/bKp2s4730BcWDStGLYm+y9szRW6c cm4OZJampYnuKyNLn6sU3IZwjgqFltlHfCkhcMHhhFzJ3ZxDm44iLJhgjs0RNia2 1G8jU2CHnSJdcRQPjbaCDbV0mQiMFLoMxNIZoetUlDV01DccgI9XylcdtYjs0aI4 xv0ZgKlLyaUyZ/NJqqRFS0p/+lNyzcs3wiEMbQQ8lpow== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=+v53l4 HjxJOtbWb2PL/isgF4lafR8IG/t73vecOIgsE=; b=kDkdc63KXVihXkn7fgp+BQ eXN+VkH7GYyNcLxNvJTbHrGS2KNLR/97Rn919+WJQv3vjI7pFu+qG0Mn4lkf+UvJ RKjkHUG/TkJ0Aw2/GZacuBGYQrx39c8ym9no8RmpoXXwzl/UQbbmylllcyWmJ9kp OBt88x6vRICPg3/58oa35dBk9OhAgKj4N5bfK9dejv/SC4J7hkTwBwGNz1iDKGSq 5fUpydnTgq4Gis1fSEDCy7SW7jVs7RAzM/IeFi1y199g5gU9MS0jL5Z6CyhyFTkv /u+WRM2R/br6mc0qE87gR5pjjgLNQ6x/4l9Q+M5DVg+jMEMcu/mGNUStO6zI5+2g == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedrtddtgdefudcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepvffhuffkffgfgggtgfesthejredttd efjeenucfhrhhomhepjghurhhiucfrrghnkhhovhcuoeihuhhrihhpvheshihurhhiphhv rdhnvghtqeenucffohhmrghinhepfhhrvggvsghsugdrohhrghdpphgrshhtvggsohgrrh gurdgtohenucfkphepuddtledrjedtrddugeegrdduhedtnecurfgrrhgrmhepmhgrihhl fhhrohhmpeihuhhrihhpvheshihurhhiphhvrdhnvghtnecuvehluhhsthgvrhfuihiivg eptd X-ME-Proxy: Received: from [192.168.1.11] (unknown [109.70.144.150]) by mail.messagingengine.com (Postfix) with ESMTPA id 03277D6005D for ; Tue, 10 Sep 2019 08:37:54 -0400 (EDT) To: freebsd-current From: Yuri Pankov Subject: panic: rcv_start < rcv_end Message-ID: <1601b17b-764f-ebaf-5b00-6ff26c82680e@yuripv.net> Date: Tue, 10 Sep 2019 15:37:54 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Thunderbird/68.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46SPh61wjlz4PBP X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yuripv.net header.s=fm1 header.b=IqLWHbVa; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=kDkdc63K; dmarc=none; spf=pass (mx1.freebsd.org: domain of yuripv@yuripv.net designates 66.111.4.230 as permitted sender) smtp.mailfrom=yuripv@yuripv.net X-Spamd-Result: default: False [-6.09 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[yuripv.net:s=fm1,messagingengine.com:s=fm3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.230]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[yuripv.net]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yuripv.net:+,messagingengine.com:+]; IP_SCORE(-3.49)[ip: (-9.87), ipnet: 66.111.4.0/24(-4.84), asn: 11403(-2.68), country: US(-0.05)]; RCVD_IN_DNSWL_LOW(-0.10)[230.4.111.66.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 12:37:59 -0000 Just seen this almost immediately after booting the system installed from amd64-20190906-r351901 snapshot, trying to do initial pkg bootstrap. Sadly, I didn't have the swap/dump device configured at the time, so no dump was saved. But it looks like I'm not alone, seeing the https://forums.freebsd.org/threads/kernel-panic-on-bhyve-virtualization.72222/ topic. Note that I'm running on bare metal, so bhyve isn't involved. My panic screenshot is at https://pasteboard.co/IwLaXXb.jpg. In (the most likely) case it's not helpful enough, I'm now running with dump device configured, and will update if/when the panic reproduces. From owner-freebsd-current@freebsd.org Tue Sep 10 14:20:45 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 008D9D7ACE for ; Tue, 10 Sep 2019 14:20:45 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46SRyh6BpKz4V6j for ; Tue, 10 Sep 2019 14:20:44 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from [10.211.20.102] (unknown [194.95.11.51]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTPSA id 67969721E2829; Tue, 10 Sep 2019 16:20:42 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: panic: rcv_start < rcv_end From: Michael Tuexen In-Reply-To: <1601b17b-764f-ebaf-5b00-6ff26c82680e@yuripv.net> Date: Tue, 10 Sep 2019 16:20:41 +0200 Cc: freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: References: <1601b17b-764f-ebaf-5b00-6ff26c82680e@yuripv.net> To: Yuri Pankov X-Mailer: Apple Mail (2.3445.104.11) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Rspamd-Queue-Id: 46SRyh6BpKz4V6j X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-2.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; ASN(0.00)[asn:680, ipnet:193.174.0.0/15, country:DE] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 14:20:45 -0000 > On 10. Sep 2019, at 14:37, Yuri Pankov wrote: >=20 > Just seen this almost immediately after booting the system installed = from amd64-20190906-r351901 snapshot, trying to do initial pkg = bootstrap. Sadly, I didn't have the swap/dump device configured at the = time, so no dump was saved. >=20 > But it looks like I'm not alone, seeing the = https://forums.freebsd.org/threads/kernel-panic-on-bhyve-virtualization.72= 222/ topic. Note that I'm running on bare metal, so bhyve isn't = involved. My panic screenshot is at https://pasteboard.co/IwLaXXb.jpg. >=20 > In (the most likely) case it's not helpful enough, I'm now running = with dump device configured, and will update if/when the panic = reproduces. This panic should be fixed by: https://svnweb.freebsd.org/changeset/base/352072 Please drop me a note if not. Best regards Michael > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to = "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@freebsd.org Tue Sep 10 14:23:17 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 741A6D7E19; Tue, 10 Sep 2019 14:23:17 +0000 (UTC) (envelope-from ler@FreeBSD.org) Received: from thebighonker.lerctr.org (ns-b.lerctr.org [IPv6:2001:470:1f0f:3ad::53:2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.lerctr.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46SS1d2VNzz4VW1; Tue, 10 Sep 2019 14:23:17 +0000 (UTC) (envelope-from ler@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=ler2019; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Transfer-Encoding:Content-Type:MIME-Version:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=3CDicv77rxfbInzeaIxweOfTNoRWHN6eVvNGqUf46UQ=; b=hHjn3Ty2/AmvfoteQ1+aaNic6j 7gfOCy57TDGGNNbeQRLiEUZPoDkvFha58vnY3Pg6uHDP13bTPfcN/m5c8gRb1ndO2bEPAMwXG2Vp2 ibSRqs1D0NUManJZD0Vy0aV34ANOhSoUJ7VcMtclKNTnHF21E6qKcvkMYXJR8SIjdWI3DutzAi0SY FMYWgALUlL2ki0FYHK1JquE7+OCJ/759N27W/34zTCltnsaO2y8/JQDFItJZqT8nD5HhkzTArROQD 0Ghd3jBZmjTjgTRXXnykUfjXzna+9WPsECqaIQWtDzqAhlpMZHqxt13u22K3sx7IhAR8p4mF+JFre XmSbY9YA==; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:bb:dcff:fe50:d900]:27726 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92.2 (FreeBSD)) (envelope-from ) id 1i7h35-0006RE-Ib; Tue, 10 Sep 2019 09:23:15 -0500 Received: from 2600:1700:210:b180:f434:7f3b:2aaf:25b9 by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Tue, 10 Sep 2019 09:23:15 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 10 Sep 2019 09:23:15 -0500 From: Larry Rosenman To: Michael Tuexen Cc: Yuri Pankov , freebsd-current , owner-freebsd-current@freebsd.org Subject: Re: panic: rcv_start < rcv_end In-Reply-To: References: <1601b17b-764f-ebaf-5b00-6ff26c82680e@yuripv.net> Message-ID: <88808aaf430b3184cd5e6921fc8c10ba@FreeBSD.org> X-Sender: ler@FreeBSD.org User-Agent: Roundcube Webmail/1.3.9 X-Rspamd-Queue-Id: 46SS1d2VNzz4VW1 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 14:23:17 -0000 On 09/10/2019 9:20 am, Michael Tuexen wrote: >> On 10. Sep 2019, at 14:37, Yuri Pankov wrote: >> >> Just seen this almost immediately after booting the system installed >> from amd64-20190906-r351901 snapshot, trying to do initial pkg >> bootstrap. Sadly, I didn't have the swap/dump device configured at >> the time, so no dump was saved. >> >> But it looks like I'm not alone, seeing the >> https://forums.freebsd.org/threads/kernel-panic-on-bhyve-virtualization.72222/ >> topic. Note that I'm running on bare metal, so bhyve isn't involved. >> My panic screenshot is at https://pasteboard.co/IwLaXXb.jpg. >> >> In (the most likely) case it's not helpful enough, I'm now running >> with dump device configured, and will update if/when the panic >> reproduces. > This panic should be fixed by: > https://svnweb.freebsd.org/changeset/base/352072 > > Please drop me a note if not. > > Best regards > Michael >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to >> "freebsd-current-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" is this the same panic: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240471 I *DO* have a core. -- Larry Rosenman http://people.freebsd.org/~ler Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 From owner-freebsd-current@freebsd.org Tue Sep 10 14:32:45 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5D1B4D8208 for ; Tue, 10 Sep 2019 14:32:45 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from new2-smtp.messagingengine.com (new2-smtp.messagingengine.com [66.111.4.224]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46SSDX3H0yz4W57; Tue, 10 Sep 2019 14:32:44 +0000 (UTC) (envelope-from yuripv@yuripv.net) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailnew.nyi.internal (Postfix) with ESMTP id 7CB06CC4; Tue, 10 Sep 2019 10:32:43 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Tue, 10 Sep 2019 10:32:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yuripv.net; h= subject:to:cc:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm1; bh=g nfYU3spXn0VXLLDwMHdoTGSPcEp2C0bbyUKqZ2i+3U=; b=HfH5/GPVzZqMepGeC a6qHl4w+QnXZ5hKF65FVXrpxYaMirDVz6q+h2gaY90gXbFyXhQVZezwJUUk3Wa9I h198m7JN7Fj7p9tC35QdiSCpTLFfM7QH9nkIrh7GjFeGSRtSsNDicdSRAYGHFUL+ iJEfxjWQ1UkbrAO4JQ1LdTjZvyXCgpxYsKxO/UXOAyTxMaj9DqUOx/bP53TXPSnT +9kRvuFTL9DHAT9Ju7pX9RTFZ8ZMLgtyuICCRCcIwiZnLx8rAKfC1kvrYyliUM8z DUfDacgtUfj9TXxWTTEii+mZpH9RQQ5WiIzY23GKQ9ZzERnL5eWZdcmjcTiZnmrI 1/WlQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=gnfYU3spXn0VXLLDwMHdoTGSPcEp2C0bbyUKqZ2i+ 3U=; b=v/O9wDpiZO+wlW3CDagu8AjxSl3WKTekJstCtX0SY/kAkmMnYux/xmrVG fW8NWBubrmkyKlmPoWtthNiEKgvV3TUUKBxV7iFo/EaF8Zj4uhXnKlsWt2Qe3iJq YXo5LfLgF+C7kANWlrlBrYMM3D8dO+l17ecLwT6SEHHBgFiqPndLg15gHIpXPa/w LqbReIeQ/qPJ7cjib27gBaI6vNB6UnIEImZEuWmV+Emlba+Wim26wm7pbHCdSMUR S8fElEs/8W8z6B3zIn4zLgwwe77momfWpGZyH7LIbEViuoPcE5QwCkFzFgVLht/c YTYPrIPeBtl9LWE848ZIMbAO6rpTQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedrtddtgdehgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenog fuphgrmhfkphdqohhuthculdehtddtmdenucfjughrpefuvfhfhffkffgfgggjtgfgseht jeertddtfeejnecuhfhrohhmpegjuhhrihcurfgrnhhkohhvuceohihurhhiphhvseihuh hrihhpvhdrnhgvtheqnecuffhomhgrihhnpehfrhgvvggsshgurdhorhhgpdhprghsthgv sghorghrugdrtghonecukfhppedvudejrddugeeirdekvddrudeggeenucfrrghrrghmpe hmrghilhhfrhhomhephihurhhiphhvseihuhhrihhpvhdrnhgvthenucevlhhushhtvghr ufhiiigvpedt X-ME-Proxy: Received: from [192.168.1.11] (unknown [217.146.82.144]) by mail.messagingengine.com (Postfix) with ESMTPA id 8223F80059; Tue, 10 Sep 2019 10:32:42 -0400 (EDT) Subject: Re: panic: rcv_start < rcv_end To: Michael Tuexen Cc: freebsd-current References: <1601b17b-764f-ebaf-5b00-6ff26c82680e@yuripv.net> From: Yuri Pankov Message-ID: Date: Tue, 10 Sep 2019 17:32:41 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Thunderbird/68.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46SSDX3H0yz4W57 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yuripv.net header.s=fm1 header.b=HfH5/GPV; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=v/O9wDpi; dmarc=none; spf=pass (mx1.freebsd.org: domain of yuripv@yuripv.net designates 66.111.4.224 as permitted sender) smtp.mailfrom=yuripv@yuripv.net X-Spamd-Result: default: False [-5.63 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[yuripv.net:s=fm1,messagingengine.com:s=fm3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.224]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[yuripv.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; IP_SCORE(-3.03)[ip: (-7.56), ipnet: 66.111.4.0/24(-4.84), asn: 11403(-2.68), country: US(-0.05)]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yuripv.net:+,messagingengine.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_LOW(-0.10)[224.4.111.66.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 14:32:45 -0000 Michael Tuexen wrote: >> On 10. Sep 2019, at 14:37, Yuri Pankov wrote: >> >> Just seen this almost immediately after booting the system installed from amd64-20190906-r351901 snapshot, trying to do initial pkg bootstrap. Sadly, I didn't have the swap/dump device configured at the time, so no dump was saved. >> >> But it looks like I'm not alone, seeing the https://forums.freebsd.org/threads/kernel-panic-on-bhyve-virtualization.72222/ topic. Note that I'm running on bare metal, so bhyve isn't involved. My panic screenshot is at https://pasteboard.co/IwLaXXb.jpg. >> >> In (the most likely) case it's not helpful enough, I'm now running with dump device configured, and will update if/when the panic reproduces. > This panic should be fixed by: > https://svnweb.freebsd.org/changeset/base/352072 > > Please drop me a note if not. Good to know, thanks! I only seen it once after installing the snapshot that didn't include the fix, and didn't notice that the fix is already committed. From owner-freebsd-current@freebsd.org Tue Sep 10 16:33:44 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 708B8DBA66 for ; Tue, 10 Sep 2019 16:33:44 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [IPv6:2a01:4f8:c17:6c4b::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46SVw74CcLz4dj9 for ; Tue, 10 Sep 2019 16:33:43 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.129.235]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 52B492602ED; Tue, 10 Sep 2019 18:33:40 +0200 (CEST) To: FreeBSD Current , Warner Losh From: Hans Petter Selasky Subject: Source tree has many empty directories? Message-ID: Date: Tue, 10 Sep 2019 18:32:54 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46SVw74CcLz4dj9 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of hps@selasky.org designates 2a01:4f8:c17:6c4b::2 as permitted sender) smtp.mailfrom=hps@selasky.org X-Spamd-Result: default: False [-3.87 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:mail.turbocat.net:c]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; SUBJECT_ENDS_QUESTION(1.00)[]; DMARC_NA(0.00)[selasky.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-2.57)[ip: (-9.07), ipnet: 2a01:4f8::/29(-1.97), asn: 24940(-1.78), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 16:33:44 -0000 Hi Developers, My -head source tree might be dirty over the years, but there appears to be some empty directories. Can these just be removed? --HPS find . -type d -empty ./sys/fs/nandfs ./sys/mips/gxemul ./sys/gnu/dts/include/dt-bindings/genpd ./sys/modules/drm/r128 ./sys/modules/drm/sis ./sys/modules/drm/via ./sys/modules/drm/drm ./sys/modules/drm/mach64 ./sys/modules/drm/mga ./sys/modules/drm/tdfx ./sys/modules/drm/savage ./sys/modules/if_tun ./sys/modules/nandfs ./sys/modules/nand ./sys/modules/nandsim ./sys/modules/drm2/drm2 ./sys/modules/drm2/radeonkmsfw/ARUBA_me ./sys/modules/drm2/radeonkmsfw/VERDE_ce ./sys/modules/drm2/radeonkmsfw/TURKS_pfp ./sys/modules/drm2/radeonkmsfw/HAINAN_mc ./sys/modules/drm2/radeonkmsfw/CAYMAN_pfp ./sys/modules/drm2/radeonkmsfw/HAINAN_me ./sys/modules/drm2/radeonkmsfw/BARTS_pfp ./sys/modules/drm2/radeonkmsfw/CAICOS_mc ./sys/modules/drm2/radeonkmsfw/CAICOS_me ./sys/modules/drm2/radeonkmsfw/CEDAR_pfp ./sys/modules/drm2/radeonkmsfw/RV710_pfp ./sys/modules/drm2/radeonkmsfw/RV630_pfp ./sys/modules/drm2/radeonkmsfw/R600_rlc ./sys/modules/drm2/radeonkmsfw/TAHITI_ce ./sys/modules/drm2/radeonkmsfw/RV670_pfp ./sys/modules/drm2/radeonkmsfw/BARTS_mc ./sys/modules/drm2/radeonkmsfw/ARUBA_rlc ./sys/modules/drm2/radeonkmsfw/RV635_pfp ./sys/modules/drm2/radeonkmsfw/BARTS_me ./sys/modules/drm2/radeonkmsfw/CYPRESS_pfp ./sys/modules/drm2/radeonkmsfw/PALM_pfp ./sys/modules/drm2/radeonkmsfw/HAINAN_rlc ./sys/modules/drm2/radeonkmsfw/RV710_me ./sys/modules/drm2/radeonkmsfw/OLAND_pfp ./sys/modules/drm2/radeonkmsfw/RV730_me ./sys/modules/drm2/radeonkmsfw/OLAND_ce ./sys/modules/drm2/radeonkmsfw/R200_cp ./sys/modules/drm2/radeonkmsfw/RV770_me ./sys/modules/drm2/radeonkmsfw/REDWOOD_pfp ./sys/modules/drm2/radeonkmsfw/SUMO2_pfp ./sys/modules/drm2/radeonkmsfw/JUNIPER_rlc ./sys/modules/drm2/radeonkmsfw/PITCAIRN_pfp ./sys/modules/drm2/radeonkmsfw/PITCAIRN_ce ./sys/modules/drm2/radeonkmsfw/SUMO_rlc ./sys/modules/drm2/radeonkmsfw/REDWOOD_me ./sys/modules/drm2/radeonkmsfw/TAHITI_pfp ./sys/modules/drm2/radeonkmsfw/CEDAR_me ./sys/modules/drm2/radeonkmsfw/SUMO_uvd ./sys/modules/drm2/radeonkmsfw/VERDE_rlc ./sys/modules/drm2/radeonkmsfw/HAINAN_ce ./sys/modules/drm2/radeonkmsfw/CAICOS_pfp ./sys/modules/drm2/radeonkmsfw/R300_cp ./sys/modules/drm2/radeonkmsfw/BTC_rlc ./sys/modules/drm2/radeonkmsfw/CAYMAN_rlc ./sys/modules/drm2/radeonkmsfw/CEDAR_rlc ./sys/modules/drm2/radeonkmsfw/RV610_pfp ./sys/modules/drm2/radeonkmsfw/VERDE_mc ./sys/modules/drm2/radeonkmsfw/VERDE_me ./sys/modules/drm2/radeonkmsfw/RV730_pfp ./sys/modules/drm2/radeonkmsfw/CYPRESS_rlc ./sys/modules/drm2/radeonkmsfw/R700_rlc ./sys/modules/drm2/radeonkmsfw/RS780_pfp ./sys/modules/drm2/radeonkmsfw/RV770_pfp ./sys/modules/drm2/radeonkmsfw/R600_pfp ./sys/modules/drm2/radeonkmsfw/RV710_uvd ./sys/modules/drm2/radeonkmsfw/JUNIPER_me ./sys/modules/drm2/radeonkmsfw/OLAND_rlc ./sys/modules/drm2/radeonkmsfw/ARUBA_pfp ./sys/modules/drm2/radeonkmsfw/TAHITI_mc ./sys/modules/drm2/radeonkmsfw/TAHITI_me ./sys/modules/drm2/radeonkmsfw/HAINAN_pfp ./sys/modules/drm2/radeonkmsfw/REDWOOD_rlc ./sys/modules/drm2/radeonkmsfw/RS780_me ./sys/modules/drm2/radeonkmsfw/CYPRESS_uvd ./sys/modules/drm2/radeonkmsfw/RV635_me ./sys/modules/drm2/radeonkmsfw/R600_me ./sys/modules/drm2/radeonkmsfw/R420_cp ./sys/modules/drm2/radeonkmsfw/PITCAIRN_rlc ./sys/modules/drm2/radeonkmsfw/PALM_me ./sys/modules/drm2/radeonkmsfw/OLAND_mc ./sys/modules/drm2/radeonkmsfw/OLAND_me ./sys/modules/drm2/radeonkmsfw/JUNIPER_pfp ./sys/modules/drm2/radeonkmsfw/TAHITI_rlc ./sys/modules/drm2/radeonkmsfw/RV620_pfp ./sys/modules/drm2/radeonkmsfw/SUMO2_me ./sys/modules/drm2/radeonkmsfw/CAYMAN_mc ./sys/modules/drm2/radeonkmsfw/TURKS_mc ./sys/modules/drm2/radeonkmsfw/PITCAIRN_mc ./sys/modules/drm2/radeonkmsfw/SUMO_pfp ./sys/modules/drm2/radeonkmsfw/CAYMAN_me ./sys/modules/drm2/radeonkmsfw/TURKS_me ./sys/modules/drm2/radeonkmsfw/PITCAIRN_me ./sys/modules/drm2/radeonkmsfw/RS600_cp ./sys/modules/drm2/radeonkmsfw/RV610_me ./sys/modules/drm2/radeonkmsfw/RV620_me ./sys/modules/drm2/radeonkmsfw/TAHITI_uvd ./sys/modules/drm2/radeonkmsfw/RV630_me ./sys/modules/drm2/radeonkmsfw/R100_cp ./sys/modules/drm2/radeonkmsfw/SUMO_me ./sys/modules/drm2/radeonkmsfw/RS690_cp ./sys/modules/drm2/radeonkmsfw/RV670_me ./sys/modules/drm2/radeonkmsfw/CYPRESS_me ./sys/modules/drm2/radeonkmsfw/R520_cp ./sys/modules/drm2/radeonkmsfw/VERDE_pfp ./sys/modules/drm2/i915kms ./sys/modules/drm2/radeonkms ./sys/modules/if_tap ./sys/dev/nand ./crypto/heimdal/lib/sqlite ./usr.bin/send-pr ./sbin/nandfs ./sbin/newfs_nandfs ./tools/tools/nanobsd/gateworks/Files/root ./tools/tools/nanobsd/gateworks/cfg/ssh ./tools/tools/nanobsd/rescue/Pkg ./contrib/traceroute/lbl ./contrib/ipfilter/net ./contrib/ipfilter/ipsd/Celler ./contrib/netbsd-tests/dev/usb/libhid ./contrib/netbsd-tests/dev/usb/t_hid ./contrib/netbsd-tests/crypto/libcrypto/x509v3 ./contrib/netbsd-tests/crypto/libcrypto/rsa ./contrib/netbsd-tests/crypto/libcrypto/rc2 ./contrib/netbsd-tests/crypto/libcrypto/bf ./contrib/netbsd-tests/crypto/libcrypto/rc4 ./contrib/netbsd-tests/crypto/libcrypto/rc5 ./contrib/netbsd-tests/crypto/libcrypto/dh ./contrib/netbsd-tests/crypto/libcrypto/lhash ./contrib/netbsd-tests/crypto/libcrypto/bn/exp ./contrib/netbsd-tests/crypto/libcrypto/bn/bn ./contrib/netbsd-tests/crypto/libcrypto/bn/div ./contrib/netbsd-tests/crypto/libcrypto/idea ./contrib/netbsd-tests/crypto/libcrypto/sha ./contrib/netbsd-tests/crypto/libcrypto/ecdsa ./contrib/netbsd-tests/crypto/libcrypto/ripemd ./contrib/netbsd-tests/crypto/libcrypto/md2 ./contrib/netbsd-tests/crypto/libcrypto/md4 ./contrib/netbsd-tests/crypto/libcrypto/rand ./contrib/netbsd-tests/crypto/libcrypto/md5 ./contrib/netbsd-tests/crypto/libcrypto/mdc2 ./contrib/netbsd-tests/crypto/libcrypto/ec ./contrib/netbsd-tests/crypto/libcrypto/cast ./contrib/netbsd-tests/crypto/libcrypto/evp ./contrib/netbsd-tests/crypto/libcrypto/threads ./contrib/netbsd-tests/crypto/libcrypto/sha1 ./contrib/netbsd-tests/crypto/libcrypto/ecdh ./contrib/netbsd-tests/crypto/libcrypto/srp ./contrib/netbsd-tests/crypto/libcrypto/engine ./contrib/netbsd-tests/crypto/libcrypto/dsa ./contrib/netbsd-tests/crypto/libcrypto/des ./contrib/netbsd-tests/crypto/libcrypto/hmac ./contrib/netbsd-tests/lib/libtre ./contrib/netbsd-tests/lib/libposix/posix2 ./contrib/netbsd-tests/lib/libposix/bsd ./contrib/netbsd-tests/lib/libposix/posix1 ./contrib/apr/include/private ./contrib/wpa/patches ./contrib/wpa/src/hlr_auc_gw ./contrib/wpa/wpa_supplicant/tests ./contrib/compiler-rt/lib/builtins/armv6m ./contrib/compiler-rt/lib/sancov ./contrib/llvm/tools/lldb/source/Plugins/OperatingSystem/Go ./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Go ./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Java ./contrib/llvm/tools/lldb/source/Plugins/ExpressionParser/Go ./contrib/llvm/tools/lldb/source/Plugins/Language/Go ./contrib/llvm/tools/lldb/source/Plugins/Language/Java ./contrib/llvm/tools/lldb/source/Plugins/Language/OCaml ./contrib/llvm/tools/llvm-mca/include/HardwareUnits ./contrib/llvm/tools/llvm-mca/include/Stages ./contrib/llvm/tools/llvm-mca/lib/HardwareUnits ./contrib/llvm/tools/llvm-mca/lib/Stages ./contrib/llvm/include/llvm/MC/MCAnalysis ./contrib/llvm/include/llvm/BinaryFormat/WasmRelocs ./contrib/llvm/include/llvm/TextAPI/MachO ./contrib/llvm/lib/ExecutionEngine/JIT ./contrib/llvm/lib/MC/MCAnalysis ./contrib/llvm/lib/Target/Nios2/MCTargetDesc ./contrib/llvm/lib/Target/Nios2/TargetInfo ./contrib/llvm/lib/Target/Nios2/InstPrinter ./contrib/llvm/lib/TextAPI/MachO ./contrib/libxo/m4 ./usr.sbin/nandsim ./usr.sbin/nandtool ./usr.sbin/bsdconfig/fdisk ./lib/libnandfs ./cddl/contrib/opensolaris/common/avl ./stand/sparc64/zfsloader From owner-freebsd-current@freebsd.org Tue Sep 10 16:59:02 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C998EDCB71 for ; Tue, 10 Sep 2019 16:59:02 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46SWTL1HkPz3D7k for ; Tue, 10 Sep 2019 16:59:01 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x842.google.com with SMTP id r5so21617022qtd.0 for ; Tue, 10 Sep 2019 09:59:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=vvArdjmWeDEfzaC1tGkfhOuL7e+sDPnvEEVm1RDnopU=; b=SWz2+AmXLxnpBzCHdtuggNuZacsUSSvDLKbUpu4I3SXv6UkDLplMb5H7II7G5N/qJ3 KmkqySdXvy8geSjoAsfdE1LGTY/rbgYeTi5eFS8VM/VO0FTzlJnQ8TLeZdUL31tAOkQG UXDnEf1ZxrGSkexLZvkx0ouVRSJVjo5jFLCPy14WLrmHTnA/uYe6WjhQpRXr55ku2IwM 8TAZGwUCr1ZFIKtyGm8AEL32fDJVN4WLbJ7ZRSrdvC2GyHAveMpjaW2FYn1id74+a6tc A850IqwJeGCqA9Pr8/Az6Q4z5UFiC9yLSk6HI+4S+mMSvUpMxELOpZS9HCJnoJsEezAA NWzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=vvArdjmWeDEfzaC1tGkfhOuL7e+sDPnvEEVm1RDnopU=; b=uVXMNksdL3EOZbJmaFghs158cgNatn2G5E9/ekVUjFe6mv6b8Ird6JRNq5DcoWvKQ8 ytKljkSlsX0TWgStSi5kZ55CXXpPL1DtR22N7iewxvbwzhMNP40bW/Hqquvlrmbmr2W/ rFSYrK5nJMeNfr9OVaCf7A08tXuVRVA+3EIyXi+4OC45pvs1ofwaET33H0ckZaKIo5aS +SrfWDCW/3KxsIWaKplsdTCr6Vy8P4UcG2d2BHIJngRp9tqifUWune+FP7mGGRBLdHPI 26UvyH9TobvWdFVH65b0Hx4snUJEGbhji56/EBOTAQuMrWnGdrfdUMLMG3lwSQFgXdNQ EXrw== X-Gm-Message-State: APjAAAUhcmSMvmZ9XD7pF0XmVAL1T7/yrrRSEw+4plWmTCU+yiKVjZ25 Fsx/Ov6bWooSUPwP8NNKh7uWctx+Db+NsTakf9EWH7n/hEQ= X-Google-Smtp-Source: APXvYqwfjxYv0pivL8oHXdNLOub3dlRqh4yMGRoHn7ml1wot7O3yybdWytV1owjpwz6ZbHWGC4k4CQKktI5mC4ZLX7U= X-Received: by 2002:ac8:71cb:: with SMTP id i11mr28754320qtp.32.1568134740878; Tue, 10 Sep 2019 09:59:00 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Tue, 10 Sep 2019 10:58:50 -0600 Message-ID: Subject: Re: Source tree has many empty directories? To: Hans Petter Selasky Cc: FreeBSD Current X-Rspamd-Queue-Id: 46SWTL1HkPz3D7k X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=SWz2+AmX; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::842) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-1.58 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; IP_SCORE(-0.58)[ip: (2.16), ipnet: 2607:f8b0::/32(-2.73), asn: 15169(-2.26), country: US(-0.05)]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2.4.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 16:59:02 -0000 On Tue, Sep 10, 2019 at 10:33 AM Hans Petter Selasky wrote: > Hi Developers, > > My -head source tree might be dirty over the years, but there appears to > be some empty directories. Can these just be removed? > I've removed the ones I know are safe to remove, trying to mirror the commits they were originally made empty. I can do the rest if nobody else objects if people would like... Warner > --HPS > > find . -type d -empty > ./sys/fs/nandfs > ./sys/mips/gxemul > ./sys/gnu/dts/include/dt-bindings/genpd > ./sys/modules/drm/r128 > ./sys/modules/drm/sis > ./sys/modules/drm/via > ./sys/modules/drm/drm > ./sys/modules/drm/mach64 > ./sys/modules/drm/mga > ./sys/modules/drm/tdfx > ./sys/modules/drm/savage > ./sys/modules/if_tun > ./sys/modules/nandfs > ./sys/modules/nand > ./sys/modules/nandsim > ./sys/modules/drm2/drm2 > ./sys/modules/drm2/radeonkmsfw/ARUBA_me > ./sys/modules/drm2/radeonkmsfw/VERDE_ce > ./sys/modules/drm2/radeonkmsfw/TURKS_pfp > ./sys/modules/drm2/radeonkmsfw/HAINAN_mc > ./sys/modules/drm2/radeonkmsfw/CAYMAN_pfp > ./sys/modules/drm2/radeonkmsfw/HAINAN_me > ./sys/modules/drm2/radeonkmsfw/BARTS_pfp > ./sys/modules/drm2/radeonkmsfw/CAICOS_mc > ./sys/modules/drm2/radeonkmsfw/CAICOS_me > ./sys/modules/drm2/radeonkmsfw/CEDAR_pfp > ./sys/modules/drm2/radeonkmsfw/RV710_pfp > ./sys/modules/drm2/radeonkmsfw/RV630_pfp > ./sys/modules/drm2/radeonkmsfw/R600_rlc > ./sys/modules/drm2/radeonkmsfw/TAHITI_ce > ./sys/modules/drm2/radeonkmsfw/RV670_pfp > ./sys/modules/drm2/radeonkmsfw/BARTS_mc > ./sys/modules/drm2/radeonkmsfw/ARUBA_rlc > ./sys/modules/drm2/radeonkmsfw/RV635_pfp > ./sys/modules/drm2/radeonkmsfw/BARTS_me > ./sys/modules/drm2/radeonkmsfw/CYPRESS_pfp > ./sys/modules/drm2/radeonkmsfw/PALM_pfp > ./sys/modules/drm2/radeonkmsfw/HAINAN_rlc > ./sys/modules/drm2/radeonkmsfw/RV710_me > ./sys/modules/drm2/radeonkmsfw/OLAND_pfp > ./sys/modules/drm2/radeonkmsfw/RV730_me > ./sys/modules/drm2/radeonkmsfw/OLAND_ce > ./sys/modules/drm2/radeonkmsfw/R200_cp > ./sys/modules/drm2/radeonkmsfw/RV770_me > ./sys/modules/drm2/radeonkmsfw/REDWOOD_pfp > ./sys/modules/drm2/radeonkmsfw/SUMO2_pfp > ./sys/modules/drm2/radeonkmsfw/JUNIPER_rlc > ./sys/modules/drm2/radeonkmsfw/PITCAIRN_pfp > ./sys/modules/drm2/radeonkmsfw/PITCAIRN_ce > ./sys/modules/drm2/radeonkmsfw/SUMO_rlc > ./sys/modules/drm2/radeonkmsfw/REDWOOD_me > ./sys/modules/drm2/radeonkmsfw/TAHITI_pfp > ./sys/modules/drm2/radeonkmsfw/CEDAR_me > ./sys/modules/drm2/radeonkmsfw/SUMO_uvd > ./sys/modules/drm2/radeonkmsfw/VERDE_rlc > ./sys/modules/drm2/radeonkmsfw/HAINAN_ce > ./sys/modules/drm2/radeonkmsfw/CAICOS_pfp > ./sys/modules/drm2/radeonkmsfw/R300_cp > ./sys/modules/drm2/radeonkmsfw/BTC_rlc > ./sys/modules/drm2/radeonkmsfw/CAYMAN_rlc > ./sys/modules/drm2/radeonkmsfw/CEDAR_rlc > ./sys/modules/drm2/radeonkmsfw/RV610_pfp > ./sys/modules/drm2/radeonkmsfw/VERDE_mc > ./sys/modules/drm2/radeonkmsfw/VERDE_me > ./sys/modules/drm2/radeonkmsfw/RV730_pfp > ./sys/modules/drm2/radeonkmsfw/CYPRESS_rlc > ./sys/modules/drm2/radeonkmsfw/R700_rlc > ./sys/modules/drm2/radeonkmsfw/RS780_pfp > ./sys/modules/drm2/radeonkmsfw/RV770_pfp > ./sys/modules/drm2/radeonkmsfw/R600_pfp > ./sys/modules/drm2/radeonkmsfw/RV710_uvd > ./sys/modules/drm2/radeonkmsfw/JUNIPER_me > ./sys/modules/drm2/radeonkmsfw/OLAND_rlc > ./sys/modules/drm2/radeonkmsfw/ARUBA_pfp > ./sys/modules/drm2/radeonkmsfw/TAHITI_mc > ./sys/modules/drm2/radeonkmsfw/TAHITI_me > ./sys/modules/drm2/radeonkmsfw/HAINAN_pfp > ./sys/modules/drm2/radeonkmsfw/REDWOOD_rlc > ./sys/modules/drm2/radeonkmsfw/RS780_me > ./sys/modules/drm2/radeonkmsfw/CYPRESS_uvd > ./sys/modules/drm2/radeonkmsfw/RV635_me > ./sys/modules/drm2/radeonkmsfw/R600_me > ./sys/modules/drm2/radeonkmsfw/R420_cp > ./sys/modules/drm2/radeonkmsfw/PITCAIRN_rlc > ./sys/modules/drm2/radeonkmsfw/PALM_me > ./sys/modules/drm2/radeonkmsfw/OLAND_mc > ./sys/modules/drm2/radeonkmsfw/OLAND_me > ./sys/modules/drm2/radeonkmsfw/JUNIPER_pfp > ./sys/modules/drm2/radeonkmsfw/TAHITI_rlc > ./sys/modules/drm2/radeonkmsfw/RV620_pfp > ./sys/modules/drm2/radeonkmsfw/SUMO2_me > ./sys/modules/drm2/radeonkmsfw/CAYMAN_mc > ./sys/modules/drm2/radeonkmsfw/TURKS_mc > ./sys/modules/drm2/radeonkmsfw/PITCAIRN_mc > ./sys/modules/drm2/radeonkmsfw/SUMO_pfp > ./sys/modules/drm2/radeonkmsfw/CAYMAN_me > ./sys/modules/drm2/radeonkmsfw/TURKS_me > ./sys/modules/drm2/radeonkmsfw/PITCAIRN_me > ./sys/modules/drm2/radeonkmsfw/RS600_cp > ./sys/modules/drm2/radeonkmsfw/RV610_me > ./sys/modules/drm2/radeonkmsfw/RV620_me > ./sys/modules/drm2/radeonkmsfw/TAHITI_uvd > ./sys/modules/drm2/radeonkmsfw/RV630_me > ./sys/modules/drm2/radeonkmsfw/R100_cp > ./sys/modules/drm2/radeonkmsfw/SUMO_me > ./sys/modules/drm2/radeonkmsfw/RS690_cp > ./sys/modules/drm2/radeonkmsfw/RV670_me > ./sys/modules/drm2/radeonkmsfw/CYPRESS_me > ./sys/modules/drm2/radeonkmsfw/R520_cp > ./sys/modules/drm2/radeonkmsfw/VERDE_pfp > ./sys/modules/drm2/i915kms > ./sys/modules/drm2/radeonkms > ./sys/modules/if_tap > ./sys/dev/nand > ./crypto/heimdal/lib/sqlite > ./usr.bin/send-pr > ./sbin/nandfs > ./sbin/newfs_nandfs > ./tools/tools/nanobsd/gateworks/Files/root > ./tools/tools/nanobsd/gateworks/cfg/ssh > ./tools/tools/nanobsd/rescue/Pkg > ./contrib/traceroute/lbl > ./contrib/ipfilter/net > ./contrib/ipfilter/ipsd/Celler > ./contrib/netbsd-tests/dev/usb/libhid > ./contrib/netbsd-tests/dev/usb/t_hid > ./contrib/netbsd-tests/crypto/libcrypto/x509v3 > ./contrib/netbsd-tests/crypto/libcrypto/rsa > ./contrib/netbsd-tests/crypto/libcrypto/rc2 > ./contrib/netbsd-tests/crypto/libcrypto/bf > ./contrib/netbsd-tests/crypto/libcrypto/rc4 > ./contrib/netbsd-tests/crypto/libcrypto/rc5 > ./contrib/netbsd-tests/crypto/libcrypto/dh > ./contrib/netbsd-tests/crypto/libcrypto/lhash > ./contrib/netbsd-tests/crypto/libcrypto/bn/exp > ./contrib/netbsd-tests/crypto/libcrypto/bn/bn > ./contrib/netbsd-tests/crypto/libcrypto/bn/div > ./contrib/netbsd-tests/crypto/libcrypto/idea > ./contrib/netbsd-tests/crypto/libcrypto/sha > ./contrib/netbsd-tests/crypto/libcrypto/ecdsa > ./contrib/netbsd-tests/crypto/libcrypto/ripemd > ./contrib/netbsd-tests/crypto/libcrypto/md2 > ./contrib/netbsd-tests/crypto/libcrypto/md4 > ./contrib/netbsd-tests/crypto/libcrypto/rand > ./contrib/netbsd-tests/crypto/libcrypto/md5 > ./contrib/netbsd-tests/crypto/libcrypto/mdc2 > ./contrib/netbsd-tests/crypto/libcrypto/ec > ./contrib/netbsd-tests/crypto/libcrypto/cast > ./contrib/netbsd-tests/crypto/libcrypto/evp > ./contrib/netbsd-tests/crypto/libcrypto/threads > ./contrib/netbsd-tests/crypto/libcrypto/sha1 > ./contrib/netbsd-tests/crypto/libcrypto/ecdh > ./contrib/netbsd-tests/crypto/libcrypto/srp > ./contrib/netbsd-tests/crypto/libcrypto/engine > ./contrib/netbsd-tests/crypto/libcrypto/dsa > ./contrib/netbsd-tests/crypto/libcrypto/des > ./contrib/netbsd-tests/crypto/libcrypto/hmac > ./contrib/netbsd-tests/lib/libtre > ./contrib/netbsd-tests/lib/libposix/posix2 > ./contrib/netbsd-tests/lib/libposix/bsd > ./contrib/netbsd-tests/lib/libposix/posix1 > ./contrib/apr/include/private > ./contrib/wpa/patches > ./contrib/wpa/src/hlr_auc_gw > ./contrib/wpa/wpa_supplicant/tests > ./contrib/compiler-rt/lib/builtins/armv6m > ./contrib/compiler-rt/lib/sancov > ./contrib/llvm/tools/lldb/source/Plugins/OperatingSystem/Go > ./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Go > ./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Java > ./contrib/llvm/tools/lldb/source/Plugins/ExpressionParser/Go > ./contrib/llvm/tools/lldb/source/Plugins/Language/Go > ./contrib/llvm/tools/lldb/source/Plugins/Language/Java > ./contrib/llvm/tools/lldb/source/Plugins/Language/OCaml > ./contrib/llvm/tools/llvm-mca/include/HardwareUnits > ./contrib/llvm/tools/llvm-mca/include/Stages > ./contrib/llvm/tools/llvm-mca/lib/HardwareUnits > ./contrib/llvm/tools/llvm-mca/lib/Stages > ./contrib/llvm/include/llvm/MC/MCAnalysis > ./contrib/llvm/include/llvm/BinaryFormat/WasmRelocs > ./contrib/llvm/include/llvm/TextAPI/MachO > ./contrib/llvm/lib/ExecutionEngine/JIT > ./contrib/llvm/lib/MC/MCAnalysis > ./contrib/llvm/lib/Target/Nios2/MCTargetDesc > ./contrib/llvm/lib/Target/Nios2/TargetInfo > ./contrib/llvm/lib/Target/Nios2/InstPrinter > ./contrib/llvm/lib/TextAPI/MachO > ./contrib/libxo/m4 > ./usr.sbin/nandsim > ./usr.sbin/nandtool > ./usr.sbin/bsdconfig/fdisk > ./lib/libnandfs > ./cddl/contrib/opensolaris/common/avl > ./stand/sparc64/zfsloader > From owner-freebsd-current@freebsd.org Tue Sep 10 17:01:45 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DF992DD194 for ; Tue, 10 Sep 2019 17:01:45 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46SWXT0Z5Tz3DcX for ; Tue, 10 Sep 2019 17:01:44 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x742.google.com with SMTP id f13so17740438qkm.9 for ; Tue, 10 Sep 2019 10:01:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=MzC7XonyX1sn+W0a/fTD2Ew+4sXvlQNu6/uqhcDr2Qg=; b=Lod1d2ms35aam0lML0b6nF3apZ46XFyYXoTqcVKXxK+8Tna9Qxzv58ZXkchl1+6VaA +MqOnZjLN2pwtNsrxRpBJF3yHZtmedzPJ76aKKyU8uEsaFeDRfy1w8Y8iKnDGLDDdx03 x07PNOa0Z5Dhc3TsSTxzfkOIH/jjgrzDxIcntofigq1bt8xwjonQiZIhe+HptZXjpSrg lNBMLcn+v6NDWeBdhW3xR8+ssw4bKKYtAsmTbBsUn+N0wv7Egs4hEaJfoYCiezKX4O1n aTRx6fA07UJ89sUlx4Yry0f21AGil+3YS06XMt8H4GLKRvVz1HWyHdORflfoP8mB+zKF klaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=MzC7XonyX1sn+W0a/fTD2Ew+4sXvlQNu6/uqhcDr2Qg=; b=sybMZl5yLIjiPpxeWREx2vGNbg3jKA6LAXmyMuMQQowpelLNGmiNfOqC2hvWrXNyAJ /CgU7qoKN/mQZshGQinUknN4CMITvBLejWfzOdveppUfwz6Sj9Lux6mYztn8pc4CAIYg nYVDI5Xwnqe5ERVyf7+bcElu+edyhUvZBXJWWYRs82ARVsSgnCapkpv/FbJuwf1zXQRg QZ6WbURK9pE+VQpXCuTh95O5e4NLC7rA8J3csxoqcHli6NNLunRyK4dW6tVvLveZHKjW mPg16q1r2cLZEHf0POWGEs90RTenLSRQubyW6Nb4uCATGwrB4fyccgzjKqlVEqOa27fp MkXg== X-Gm-Message-State: APjAAAVdUCHtocK+y2L3D1ki5LwJg7xMmAFLkpaHt+GdvfprFe/qKf+k 6AIFqsKcBKZYDYElgvhCNXxpxZykm4yu74s+DKouCb7bZS4TXg== X-Google-Smtp-Source: APXvYqyRT0pb3FpeZCiUwok8rHpglqqgypYK7mM05g4Shd7i9XFgrhrE9AwH9iESoVlAz3WsCfbXlkKCXGKPQQnjT+c= X-Received: by 2002:a05:620a:1671:: with SMTP id d17mr20273337qko.495.1568134903831; Tue, 10 Sep 2019 10:01:43 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Tue, 10 Sep 2019 11:01:32 -0600 Message-ID: Subject: Re: Source tree has many empty directories? To: Hans Petter Selasky Cc: FreeBSD Current X-Rspamd-Queue-Id: 46SWXT0Z5Tz3DcX X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=Lod1d2ms; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::742) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-1.39 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; IP_SCORE(-0.39)[ip: (3.10), ipnet: 2607:f8b0::/32(-2.73), asn: 15169(-2.26), country: US(-0.05)]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2.4.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 17:01:45 -0000 On Tue, Sep 10, 2019 at 10:58 AM Warner Losh wrote: > > On Tue, Sep 10, 2019 at 10:33 AM Hans Petter Selasky > wrote: > >> Hi Developers, >> >> My -head source tree might be dirty over the years, but there appears to >> be some empty directories. Can these just be removed? >> > > I've removed the ones I know are safe to remove, trying to mirror the > commits they were originally made empty. I can do the rest if nobody else > objects if people would like... > The rest being: /contrib/llvm/include/llvm/BinaryFormat/WasmRelocs ./contrib/llvm/include/llvm/MC/MCAnalysis ./contrib/llvm/include/llvm/TextAPI/MachO ./contrib/llvm/lib/Target/Nios2/MCTargetDesc ./contrib/llvm/lib/Target/Nios2/TargetInfo ./contrib/llvm/lib/Target/Nios2/InstPrinter ./contrib/llvm/lib/ExecutionEngine/JIT ./contrib/llvm/lib/MC/MCAnalysis ./contrib/llvm/lib/TextAPI/MachO ./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Go ./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Java ./contrib/llvm/tools/lldb/source/Plugins/OperatingSystem/Go ./contrib/llvm/tools/lldb/source/Plugins/ExpressionParser/Go ./contrib/llvm/tools/lldb/source/Plugins/Language/Go ./contrib/llvm/tools/lldb/source/Plugins/Language/Java ./contrib/llvm/tools/lldb/source/Plugins/Language/OCaml ./contrib/llvm/tools/llvm-mca/include/HardwareUnits ./contrib/llvm/tools/llvm-mca/include/Stages ./contrib/llvm/tools/llvm-mca/lib/HardwareUnits ./contrib/llvm/tools/llvm-mca/lib/Stages ./contrib/wpa/patches ./contrib/wpa/src/hlr_auc_gw ./contrib/wpa/wpa_supplicant/tests ./contrib/compiler-rt/lib/builtins/armv6m ./contrib/compiler-rt/lib/sancov ./contrib/libxo/m4 ./contrib/ipfilter/net ./contrib/ipfilter/ipsd/Celler However, please do *NOT* remove the sys/*/compile directories. Warner > Warner > > >> --HPS >> >> find . -type d -empty >> ./sys/fs/nandfs >> ./sys/mips/gxemul >> ./sys/gnu/dts/include/dt-bindings/genpd >> ./sys/modules/drm/r128 >> ./sys/modules/drm/sis >> ./sys/modules/drm/via >> ./sys/modules/drm/drm >> ./sys/modules/drm/mach64 >> ./sys/modules/drm/mga >> ./sys/modules/drm/tdfx >> ./sys/modules/drm/savage >> ./sys/modules/if_tun >> ./sys/modules/nandfs >> ./sys/modules/nand >> ./sys/modules/nandsim >> ./sys/modules/drm2/drm2 >> ./sys/modules/drm2/radeonkmsfw/ARUBA_me >> ./sys/modules/drm2/radeonkmsfw/VERDE_ce >> ./sys/modules/drm2/radeonkmsfw/TURKS_pfp >> ./sys/modules/drm2/radeonkmsfw/HAINAN_mc >> ./sys/modules/drm2/radeonkmsfw/CAYMAN_pfp >> ./sys/modules/drm2/radeonkmsfw/HAINAN_me >> ./sys/modules/drm2/radeonkmsfw/BARTS_pfp >> ./sys/modules/drm2/radeonkmsfw/CAICOS_mc >> ./sys/modules/drm2/radeonkmsfw/CAICOS_me >> ./sys/modules/drm2/radeonkmsfw/CEDAR_pfp >> ./sys/modules/drm2/radeonkmsfw/RV710_pfp >> ./sys/modules/drm2/radeonkmsfw/RV630_pfp >> ./sys/modules/drm2/radeonkmsfw/R600_rlc >> ./sys/modules/drm2/radeonkmsfw/TAHITI_ce >> ./sys/modules/drm2/radeonkmsfw/RV670_pfp >> ./sys/modules/drm2/radeonkmsfw/BARTS_mc >> ./sys/modules/drm2/radeonkmsfw/ARUBA_rlc >> ./sys/modules/drm2/radeonkmsfw/RV635_pfp >> ./sys/modules/drm2/radeonkmsfw/BARTS_me >> ./sys/modules/drm2/radeonkmsfw/CYPRESS_pfp >> ./sys/modules/drm2/radeonkmsfw/PALM_pfp >> ./sys/modules/drm2/radeonkmsfw/HAINAN_rlc >> ./sys/modules/drm2/radeonkmsfw/RV710_me >> ./sys/modules/drm2/radeonkmsfw/OLAND_pfp >> ./sys/modules/drm2/radeonkmsfw/RV730_me >> ./sys/modules/drm2/radeonkmsfw/OLAND_ce >> ./sys/modules/drm2/radeonkmsfw/R200_cp >> ./sys/modules/drm2/radeonkmsfw/RV770_me >> ./sys/modules/drm2/radeonkmsfw/REDWOOD_pfp >> ./sys/modules/drm2/radeonkmsfw/SUMO2_pfp >> ./sys/modules/drm2/radeonkmsfw/JUNIPER_rlc >> ./sys/modules/drm2/radeonkmsfw/PITCAIRN_pfp >> ./sys/modules/drm2/radeonkmsfw/PITCAIRN_ce >> ./sys/modules/drm2/radeonkmsfw/SUMO_rlc >> ./sys/modules/drm2/radeonkmsfw/REDWOOD_me >> ./sys/modules/drm2/radeonkmsfw/TAHITI_pfp >> ./sys/modules/drm2/radeonkmsfw/CEDAR_me >> ./sys/modules/drm2/radeonkmsfw/SUMO_uvd >> ./sys/modules/drm2/radeonkmsfw/VERDE_rlc >> ./sys/modules/drm2/radeonkmsfw/HAINAN_ce >> ./sys/modules/drm2/radeonkmsfw/CAICOS_pfp >> ./sys/modules/drm2/radeonkmsfw/R300_cp >> ./sys/modules/drm2/radeonkmsfw/BTC_rlc >> ./sys/modules/drm2/radeonkmsfw/CAYMAN_rlc >> ./sys/modules/drm2/radeonkmsfw/CEDAR_rlc >> ./sys/modules/drm2/radeonkmsfw/RV610_pfp >> ./sys/modules/drm2/radeonkmsfw/VERDE_mc >> ./sys/modules/drm2/radeonkmsfw/VERDE_me >> ./sys/modules/drm2/radeonkmsfw/RV730_pfp >> ./sys/modules/drm2/radeonkmsfw/CYPRESS_rlc >> ./sys/modules/drm2/radeonkmsfw/R700_rlc >> ./sys/modules/drm2/radeonkmsfw/RS780_pfp >> ./sys/modules/drm2/radeonkmsfw/RV770_pfp >> ./sys/modules/drm2/radeonkmsfw/R600_pfp >> ./sys/modules/drm2/radeonkmsfw/RV710_uvd >> ./sys/modules/drm2/radeonkmsfw/JUNIPER_me >> ./sys/modules/drm2/radeonkmsfw/OLAND_rlc >> ./sys/modules/drm2/radeonkmsfw/ARUBA_pfp >> ./sys/modules/drm2/radeonkmsfw/TAHITI_mc >> ./sys/modules/drm2/radeonkmsfw/TAHITI_me >> ./sys/modules/drm2/radeonkmsfw/HAINAN_pfp >> ./sys/modules/drm2/radeonkmsfw/REDWOOD_rlc >> ./sys/modules/drm2/radeonkmsfw/RS780_me >> ./sys/modules/drm2/radeonkmsfw/CYPRESS_uvd >> ./sys/modules/drm2/radeonkmsfw/RV635_me >> ./sys/modules/drm2/radeonkmsfw/R600_me >> ./sys/modules/drm2/radeonkmsfw/R420_cp >> ./sys/modules/drm2/radeonkmsfw/PITCAIRN_rlc >> ./sys/modules/drm2/radeonkmsfw/PALM_me >> ./sys/modules/drm2/radeonkmsfw/OLAND_mc >> ./sys/modules/drm2/radeonkmsfw/OLAND_me >> ./sys/modules/drm2/radeonkmsfw/JUNIPER_pfp >> ./sys/modules/drm2/radeonkmsfw/TAHITI_rlc >> ./sys/modules/drm2/radeonkmsfw/RV620_pfp >> ./sys/modules/drm2/radeonkmsfw/SUMO2_me >> ./sys/modules/drm2/radeonkmsfw/CAYMAN_mc >> ./sys/modules/drm2/radeonkmsfw/TURKS_mc >> ./sys/modules/drm2/radeonkmsfw/PITCAIRN_mc >> ./sys/modules/drm2/radeonkmsfw/SUMO_pfp >> ./sys/modules/drm2/radeonkmsfw/CAYMAN_me >> ./sys/modules/drm2/radeonkmsfw/TURKS_me >> ./sys/modules/drm2/radeonkmsfw/PITCAIRN_me >> ./sys/modules/drm2/radeonkmsfw/RS600_cp >> ./sys/modules/drm2/radeonkmsfw/RV610_me >> ./sys/modules/drm2/radeonkmsfw/RV620_me >> ./sys/modules/drm2/radeonkmsfw/TAHITI_uvd >> ./sys/modules/drm2/radeonkmsfw/RV630_me >> ./sys/modules/drm2/radeonkmsfw/R100_cp >> ./sys/modules/drm2/radeonkmsfw/SUMO_me >> ./sys/modules/drm2/radeonkmsfw/RS690_cp >> ./sys/modules/drm2/radeonkmsfw/RV670_me >> ./sys/modules/drm2/radeonkmsfw/CYPRESS_me >> ./sys/modules/drm2/radeonkmsfw/R520_cp >> ./sys/modules/drm2/radeonkmsfw/VERDE_pfp >> ./sys/modules/drm2/i915kms >> ./sys/modules/drm2/radeonkms >> ./sys/modules/if_tap >> ./sys/dev/nand >> ./crypto/heimdal/lib/sqlite >> ./usr.bin/send-pr >> ./sbin/nandfs >> ./sbin/newfs_nandfs >> ./tools/tools/nanobsd/gateworks/Files/root >> ./tools/tools/nanobsd/gateworks/cfg/ssh >> ./tools/tools/nanobsd/rescue/Pkg >> ./contrib/traceroute/lbl >> ./contrib/ipfilter/net >> ./contrib/ipfilter/ipsd/Celler >> ./contrib/netbsd-tests/dev/usb/libhid >> ./contrib/netbsd-tests/dev/usb/t_hid >> ./contrib/netbsd-tests/crypto/libcrypto/x509v3 >> ./contrib/netbsd-tests/crypto/libcrypto/rsa >> ./contrib/netbsd-tests/crypto/libcrypto/rc2 >> ./contrib/netbsd-tests/crypto/libcrypto/bf >> ./contrib/netbsd-tests/crypto/libcrypto/rc4 >> ./contrib/netbsd-tests/crypto/libcrypto/rc5 >> ./contrib/netbsd-tests/crypto/libcrypto/dh >> ./contrib/netbsd-tests/crypto/libcrypto/lhash >> ./contrib/netbsd-tests/crypto/libcrypto/bn/exp >> ./contrib/netbsd-tests/crypto/libcrypto/bn/bn >> ./contrib/netbsd-tests/crypto/libcrypto/bn/div >> ./contrib/netbsd-tests/crypto/libcrypto/idea >> ./contrib/netbsd-tests/crypto/libcrypto/sha >> ./contrib/netbsd-tests/crypto/libcrypto/ecdsa >> ./contrib/netbsd-tests/crypto/libcrypto/ripemd >> ./contrib/netbsd-tests/crypto/libcrypto/md2 >> ./contrib/netbsd-tests/crypto/libcrypto/md4 >> ./contrib/netbsd-tests/crypto/libcrypto/rand >> ./contrib/netbsd-tests/crypto/libcrypto/md5 >> ./contrib/netbsd-tests/crypto/libcrypto/mdc2 >> ./contrib/netbsd-tests/crypto/libcrypto/ec >> ./contrib/netbsd-tests/crypto/libcrypto/cast >> ./contrib/netbsd-tests/crypto/libcrypto/evp >> ./contrib/netbsd-tests/crypto/libcrypto/threads >> ./contrib/netbsd-tests/crypto/libcrypto/sha1 >> ./contrib/netbsd-tests/crypto/libcrypto/ecdh >> ./contrib/netbsd-tests/crypto/libcrypto/srp >> ./contrib/netbsd-tests/crypto/libcrypto/engine >> ./contrib/netbsd-tests/crypto/libcrypto/dsa >> ./contrib/netbsd-tests/crypto/libcrypto/des >> ./contrib/netbsd-tests/crypto/libcrypto/hmac >> ./contrib/netbsd-tests/lib/libtre >> ./contrib/netbsd-tests/lib/libposix/posix2 >> ./contrib/netbsd-tests/lib/libposix/bsd >> ./contrib/netbsd-tests/lib/libposix/posix1 >> ./contrib/apr/include/private >> ./contrib/wpa/patches >> ./contrib/wpa/src/hlr_auc_gw >> ./contrib/wpa/wpa_supplicant/tests >> ./contrib/compiler-rt/lib/builtins/armv6m >> ./contrib/compiler-rt/lib/sancov >> ./contrib/llvm/tools/lldb/source/Plugins/OperatingSystem/Go >> ./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Go >> ./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Java >> ./contrib/llvm/tools/lldb/source/Plugins/ExpressionParser/Go >> ./contrib/llvm/tools/lldb/source/Plugins/Language/Go >> ./contrib/llvm/tools/lldb/source/Plugins/Language/Java >> ./contrib/llvm/tools/lldb/source/Plugins/Language/OCaml >> ./contrib/llvm/tools/llvm-mca/include/HardwareUnits >> ./contrib/llvm/tools/llvm-mca/include/Stages >> ./contrib/llvm/tools/llvm-mca/lib/HardwareUnits >> ./contrib/llvm/tools/llvm-mca/lib/Stages >> ./contrib/llvm/include/llvm/MC/MCAnalysis >> ./contrib/llvm/include/llvm/BinaryFormat/WasmRelocs >> ./contrib/llvm/include/llvm/TextAPI/MachO >> ./contrib/llvm/lib/ExecutionEngine/JIT >> ./contrib/llvm/lib/MC/MCAnalysis >> ./contrib/llvm/lib/Target/Nios2/MCTargetDesc >> ./contrib/llvm/lib/Target/Nios2/TargetInfo >> ./contrib/llvm/lib/Target/Nios2/InstPrinter >> ./contrib/llvm/lib/TextAPI/MachO >> ./contrib/libxo/m4 >> ./usr.sbin/nandsim >> ./usr.sbin/nandtool >> ./usr.sbin/bsdconfig/fdisk >> ./lib/libnandfs >> ./cddl/contrib/opensolaris/common/avl >> ./stand/sparc64/zfsloader >> > From owner-freebsd-current@freebsd.org Tue Sep 10 18:14:11 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7DF89E0074 for ; Tue, 10 Sep 2019 18:14:11 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound2m.ore.mailhop.org (outbound2m.ore.mailhop.org [54.149.155.156]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46SY832G7Qz3Ln9 for ; Tue, 10 Sep 2019 18:14:11 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1568139250; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=FePgCyzd8qNil9tzva7MfF6JpB2dYkuycu0QhHXHdCKsaR9cE143TvVyzwibOc8GdXJqjA+0Y4GgX qTEs9Ib/Q7cC2Ob07mttJTmgzHi7BdHHMR1iKbNHD7TL8EskxHbvDgpAfD3TxciKhtXwHCQCiorJlZ mB2weAXi9QZ3kNh/nfRVJMb7iEzlsfk4vOG+0EZbmek2Uc+Z8oSa+9FQXzFdqdXezZDpkG3dQcCUkI ppw+L7Ie4uEqL1BlbNeatQMko6BD8PlwAlgXzuiAOtAqZmDumWMPYwcNLPFozJf2wpR71vzveFc94v yMP7Qe2oRL5pVOs0RgmhAzubzeXt4YQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=g2BMxC2/jtidTqEEAP8W6x/rtNAvYl6lDBABqtKcbLE=; b=eUjZcn2o0sV+ZzufpDUkghq3gCrqngsufiRsIkevAOBxiyrWTSWub5xMokYV43UyMuo3nIlwG8ci6 LpJzF6CtzfC1FGx0nPfB8tLizqauh2aZfMium1RhRxk+ZlataruUf/ZaWkBVIyqR0GFc0xMU6OEmTG bemLWeh2RAkqgep8gkqilRwDtx9AUnhP3YQTZse5h4zbOkvxT9nwdnB/SmptIYIUFaNDiKZkAxhHCF uQ35fAyHQnVu4bhK06E+UxwAZm7T/y9hWlLClotPnxL05hGSC/OJKBk/s67eNaP7WMevUdMswp+I6k AONkhY1ZYvi6XFylzUL7YP/nLL2Z3hA== ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=g2BMxC2/jtidTqEEAP8W6x/rtNAvYl6lDBABqtKcbLE=; b=bTGiiRi7cOu8rmAjPU+NEym85FhvcKkPUVWA5Flu8RMTfLbdELTC/0BwiRW8VCnBELkqFSOL1wg2k c+VQAuBNkgND0SLZZRRHUW75eKGHvxaoNuVLMUiTuSBNWoMwsJDFfIssINqwAmYWy6OVTo85TZ62tw 90/SY1VGIszHfLZS9mWtIH5YYpmVTYKBgMgfCcw4SyIbCbmVXDi9O7Iz3fYtttl9Ly7CG2iHOc4wSZ bgkgi6Yybg8do0rFe87uCiDcwvbICF0DwoF87zH2cewFqeOGIX+UT15npbR3qULImFU9o9d3J5Uzip MESEzF9D9r/P8ooJJOEWso5RpRmV0dA== X-MHO-RoutePath: aGlwcGll X-MHO-User: c6a1ac29-d3f6-11e9-85ed-13b9aae3a1d2 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound4.ore.mailhop.org (Halon) with ESMTPSA id c6a1ac29-d3f6-11e9-85ed-13b9aae3a1d2; Tue, 10 Sep 2019 18:14:07 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x8AIE5BH067623; Tue, 10 Sep 2019 12:14:05 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: Subject: Re: Source tree has many empty directories? From: Ian Lepore To: Warner Losh Cc: FreeBSD Current Date: Tue, 10 Sep 2019 12:14:05 -0600 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46SY832G7Qz3Ln9 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-2.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:16509, ipnet:54.148.0.0/15, country:US] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 18:14:11 -0000 On Tue, 2019-09-10 at 11:01 -0600, Warner Losh wrote: > However, please do *NOT* remove the sys/*/compile directories. > > Warner Uhhh... that's interesting. I just nuked one of those on my system yesterday, because it had been hanging around since 2013 and I had no idea what was -- I just assumed the build machinery created it because I had accidentally done a make in a wrong directory once. So what are those directories about? I'm not used to seeing mystery directories appear inside a source tree. -- Ian From owner-freebsd-current@freebsd.org Tue Sep 10 19:41:08 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E6407E225D for ; Tue, 10 Sep 2019 19:41:08 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46Sb4N5qMjz3QYG; Tue, 10 Sep 2019 19:41:08 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [IPv6:2001:470:7a58:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "tensor.andric.com", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 92F5416BD2; Tue, 10 Sep 2019 19:41:08 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from [IPv6:2001:470:7a58::74d0:76b2:f541:2f0c] (unknown [IPv6:2001:470:7a58:0:74d0:76b2:f541:2f0c]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 5625147C77; Tue, 10 Sep 2019 21:41:07 +0200 (CEST) From: Dimitry Andric Message-Id: <289ADD78-FD39-435B-80D5-91901FE065DD@FreeBSD.org> Content-Type: multipart/signed; boundary="Apple-Mail=_BB7FCB97-51CC-4187-9732-B98C4374371D"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: Source tree has many empty directories? Date: Tue, 10 Sep 2019 21:41:03 +0200 In-Reply-To: Cc: Warner Losh , FreeBSD Current To: Ian Lepore References: X-Mailer: Apple Mail (2.3445.104.11) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 19:41:09 -0000 --Apple-Mail=_BB7FCB97-51CC-4187-9732-B98C4374371D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 10 Sep 2019, at 20:14, Ian Lepore wrote: >=20 > On Tue, 2019-09-10 at 11:01 -0600, Warner Losh wrote: >> However, please do *NOT* remove the sys/*/compile directories. >>=20 >> Warner >=20 > Uhhh... that's interesting. I just nuked one of those on my system > yesterday, because it had been hanging around since 2013 and I had no > idea what was -- I just assumed the build machinery created it because > I had accidentally done a make in a wrong directory once. >=20 > So what are those directories about? I'm not used to seeing mystery > directories appear inside a source tree. There's not much mystery to be found. Subversion does not warn you when = you remove the last files from a directory, and it also does not = automatically remove such empty directories, like Git. Hence, those directories tend = to stick around, because every simply forgets about them. With regards to those empty directories under contrib/llvm, those were = actually imported from upstream. But since the llvm project will switch to Git = soon, this problem will automagically disappear. :-) -Dimitry --Apple-Mail=_BB7FCB97-51CC-4187-9732-B98C4374371D Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.2 iF0EARECAB0WIQR6tGLSzjX8bUI5T82wXqMKLiCWowUCXXf8TwAKCRCwXqMKLiCW o7cGAKDOzpwbyGGwYSL74TSUjC1JrYpdOwCgsDHOh2aYTppN0uAQ2qqBGWwKWPI= =6CWy -----END PGP SIGNATURE----- --Apple-Mail=_BB7FCB97-51CC-4187-9732-B98C4374371D-- From owner-freebsd-current@freebsd.org Tue Sep 10 19:43:02 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D23BBE23AD for ; Tue, 10 Sep 2019 19:43:02 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound2m.ore.mailhop.org (outbound2m.ore.mailhop.org [54.149.155.156]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46Sb6Z3HsKz3Qv0 for ; Tue, 10 Sep 2019 19:43:02 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1568144581; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=I8A6Aic5xitGiZHELEXfsyCqEhUUU70CQW3G4LcBCgdNBy4V2MUSlnZYakZq7XVBqhkj1AYQnrXwy kkK+r0ijdPSbm7b39ba38CkMNpjC/Pg9G3NnOUfa+BwVvgAOI83qUNFwGOvVpxVNdITbIqdJIHDWKE WHKefgUA+yNIM3fkc85ZEAgqyA/AXeMd/XN9jTxD1Lb7Ow2YVBYEYnc+1hXBaiI1Blets7jRdBxCtn +AE/GszY/r2xUI7cNbwSRPsTfalflej4+Q88zlLv0NgmF84wHwSEKxenSimR05oAz1nGKuCu6071MG Xwq6eeACxlVPWO+CDdjpJ0CCnzcnH2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=nkYiFlUuSdrBn728NaspgElxgeWnZGSp9RjrSsDKuAc=; b=u6rUUgpVs7SuCBtaARxz2aP+B+DDwPeLt7ErXYjoyp5Ztmp3OI2By2e83zSlCMhvVnFrCu18XN7ea dbhsv7j/KcYJSXQFJZmlBEZkubbFd100ifbOprKY+KFhceDmuXF1Idgod7LQ1lbU/cqrtzzP/TaKja f7HLPAH0htx+l7vjLHBj+F2ntRgfFqLy78BMgFj4sC4zdIp1jeplvjk3CPowwR48v37Sdi1S2fwDuk 7bfEk+XhPu42B+JaMMX/IPfn/9lhD/NvGtg8ucMxzeokoG/CxlcB4v7HbwIBIBhhr+4G89rIvPfwUn Gq1SPk1QWiGuPqhrfBD2ioTZgqDGy4Q== ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=nkYiFlUuSdrBn728NaspgElxgeWnZGSp9RjrSsDKuAc=; b=mtRagBa2FXW5Nv7DiiagYg6CPdgNZ3sPRRbTAOjyN6x6X4KMajGjrqU59crG0UM2zD1TrklRao2ax DvKHx4I0qmlpDEplKKuX8eXb2GvPYD820pzu0Sem0ZDyyoPNTl9EBit9aj0W/0dCC+sDw2qdW6XfC7 kOBgaj0o/VCOqPj71CaYJEvq+/Yd4WuKOf4vbrFwtwfdRKe7q4BioJMNMASR5EH8F6jBpn+buv+yoU dXUXGC8gBruy1AcrCRSmKMxZzcwG/6gCkNoItQEkpJYucDu2h7x80uUFiMdl43P+5kMBa5XcmvpCrG BcxbZdpjmiN16hRfzsH2sG02qC59Rug== X-MHO-RoutePath: aGlwcGll X-MHO-User: 313be3de-d403-11e9-85ed-13b9aae3a1d2 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound4.ore.mailhop.org (Halon) with ESMTPSA id 313be3de-d403-11e9-85ed-13b9aae3a1d2; Tue, 10 Sep 2019 19:43:00 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x8AJgwCa067863; Tue, 10 Sep 2019 13:42:58 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <6140ae514e02b029e7451065da08c49c749f029e.camel@freebsd.org> Subject: Re: Source tree has many empty directories? From: Ian Lepore To: Dimitry Andric Cc: Warner Losh , FreeBSD Current Date: Tue, 10 Sep 2019 13:42:58 -0600 In-Reply-To: <289ADD78-FD39-435B-80D5-91901FE065DD@FreeBSD.org> References: <289ADD78-FD39-435B-80D5-91901FE065DD@FreeBSD.org> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46Sb6Z3HsKz3Qv0 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-2.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:16509, ipnet:54.148.0.0/15, country:US] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 19:43:02 -0000 On Tue, 2019-09-10 at 21:41 +0200, Dimitry Andric wrote: > On 10 Sep 2019, at 20:14, Ian Lepore wrote: > > > > On Tue, 2019-09-10 at 11:01 -0600, Warner Losh wrote: > > > However, please do *NOT* remove the sys/*/compile directories. > > > > > > Warner > > > > Uhhh... that's interesting. I just nuked one of those on my system > > yesterday, because it had been hanging around since 2013 and I had no > > idea what was -- I just assumed the build machinery created it because > > I had accidentally done a make in a wrong directory once. > > > > So what are those directories about? I'm not used to seeing mystery > > directories appear inside a source tree. > > There's not much mystery to be found. Subversion does not warn you when you > remove the last files from a directory, and it also does not automatically > remove such empty directories, like Git. Hence, those directories tend to > stick around, because every simply forgets about them. > > With regards to those empty directories under contrib/llvm, those were actually > imported from upstream. But since the llvm project will switch to Git soon, > this problem will automagically disappear. :-) > > -Dimitry > I was referring specifically to the sys/*/compile directories Warner mentioned. -- Ian From owner-freebsd-current@freebsd.org Tue Sep 10 19:48:30 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 85A9BE2620 for ; Tue, 10 Sep 2019 19:48:30 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46SbDs5DlCz3R8W for ; Tue, 10 Sep 2019 19:48:29 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x741.google.com with SMTP id x5so18314000qkh.5 for ; Tue, 10 Sep 2019 12:48:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=oYinMlE0C7pCJgPKN/9oA5BzF+lHzM/G/B/7N9eucAw=; b=Pu6VliFNoOD64Q9hbgGGEwCdp8G9SW8my80hKX8uqDhCk9UtjbCPqrN55ilgAaKRyu WyrQKjkpymXei+e3sLwnqRmEB8EP8oHOqwR6azKpgBfuzzDZ20n+2d6wSq6mCrDHJKcJ VVu6PHtwYVwTVS3TByvuK50ptIoEa0q1k6oJ22jx6Lb1cnYS77vtdUQs2Ucq2FOZ343n O7LLctWyAe/9ZyGeF+1kgRjWrupTCY/AnbYAatdLG2SqcWpz0FVX3URMZIwUvrmHsUBx SXhq/+9oxsScMdfDrOs/T3OFzNYvGZwgs1ZBHS17coW4xH6S3LxeGoHgzYjylCCRYdF7 cwBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=oYinMlE0C7pCJgPKN/9oA5BzF+lHzM/G/B/7N9eucAw=; b=iq+keyhYU6j0SeUIs9rtUI6ifqgl3QgcW8lHJ3XTR9uUWBIw6MJ8yrtoh/heVWH+vh c4BgEwGQ3oDwLx3THFhJtodQUqFUKxsMIzzp24MKq1LN9AeTjnC8bJGHHws9pmP3pfk0 SxVfAj9FALhPImUWOr3KrtHA0nwb61h4hUWeouFT31dN+fV67xYCPE20Q1Fmc5jsRO2J nf3OIzQB2t9suODm13br8VDM6bhBjbdlAR9IRvUe00ikoT3w6WO/qxKztcI7DPx7mTRd bVE9sfvD2CslTB4Y1JHbTNwIEm39djXTNNwCCvNyimkEM7IhpG6yQoTj7TrTwaJEztv7 8e1Q== X-Gm-Message-State: APjAAAUZAeYxY1XsKFvpETQHUQ1v66lmNPSkr+HtaxpxtIfDulhUgfga +x/JSz1yjN2DWcd0j32PcdVqGgTklVktEXNO95s3zw== X-Google-Smtp-Source: APXvYqzyFE/3r8ZgjhjaigItbRssGzdmoiDS+w8cr54E5RKSzzAu43TJqh/Ndgw5Y6IDC1daPyWyRmzMRidoqJJejJo= X-Received: by 2002:a05:620a:16ab:: with SMTP id s11mr32239864qkj.215.1568144908285; Tue, 10 Sep 2019 12:48:28 -0700 (PDT) MIME-Version: 1.0 References: <289ADD78-FD39-435B-80D5-91901FE065DD@FreeBSD.org> <6140ae514e02b029e7451065da08c49c749f029e.camel@freebsd.org> In-Reply-To: <6140ae514e02b029e7451065da08c49c749f029e.camel@freebsd.org> From: Warner Losh Date: Tue, 10 Sep 2019 13:48:16 -0600 Message-ID: Subject: Re: Source tree has many empty directories? To: Ian Lepore Cc: Dimitry Andric , FreeBSD Current X-Rspamd-Queue-Id: 46SbDs5DlCz3R8W X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=Pu6VliFN; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::741) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-1.52 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; IP_SCORE(-0.52)[ip: (2.42), ipnet: 2607:f8b0::/32(-2.73), asn: 15169(-2.26), country: US(-0.05)]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[1.4.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 19:48:30 -0000 On Tue, Sep 10, 2019, 1:43 PM Ian Lepore wrote: > On Tue, 2019-09-10 at 21:41 +0200, Dimitry Andric wrote: > > On 10 Sep 2019, at 20:14, Ian Lepore wrote: > > > > > > On Tue, 2019-09-10 at 11:01 -0600, Warner Losh wrote: > > > > However, please do *NOT* remove the sys/*/compile directories. > > > > > > > > Warner > > > > > > Uhhh... that's interesting. I just nuked one of those on my system > > > yesterday, because it had been hanging around since 2013 and I had no > > > idea what was -- I just assumed the build machinery created it because > > > I had accidentally done a make in a wrong directory once. > > > > > > So what are those directories about? I'm not used to seeing mystery > > > directories appear inside a source tree. > > > > There's not much mystery to be found. Subversion does not warn you when > you > > remove the last files from a directory, and it also does not > automatically > > remove such empty directories, like Git. Hence, those directories tend > to > > stick around, because every simply forgets about them. > > > > With regards to those empty directories under contrib/llvm, those were > actually > > imported from upstream. But since the llvm project will switch to Git > soon, > > this problem will automagically disappear. :-) > > > > -Dimitry > > > > I was referring specifically to the sys/*/compile directories Warner > mentioned. > They are for old school builds. Config(8) doesn't create them, so we have them in the tree. Because svn does this, we deleted the keep me files after the cut over. Warner > From owner-freebsd-current@freebsd.org Tue Sep 10 22:22:14 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A5E2AE66B7 for ; Tue, 10 Sep 2019 22:22:14 +0000 (UTC) (envelope-from rebecca@bsdio.com) Received: from muon.bsdio.com (muon.bluestop.org [65.103.231.193]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46SffF5pK5z46XH; Tue, 10 Sep 2019 22:22:13 +0000 (UTC) (envelope-from rebecca@bsdio.com) Received: from muon.bsdio.com (localhost [127.0.0.1]) by muon.bsdio.com (Postfix) with ESMTP id 8DF44109D09; Tue, 10 Sep 2019 16:22:18 -0600 (MDT) Received: from muon.bsdio.com ([127.0.0.1]) by muon.bsdio.com (muon.bsdio.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id VtFfv6kJCTyT; Tue, 10 Sep 2019 16:22:18 -0600 (MDT) Received: from photon.int.bluestop.org (unknown [10.0.10.120]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by muon.bsdio.com (Postfix) with ESMTPSA; Tue, 10 Sep 2019 16:22:18 -0600 (MDT) Subject: Re: Building world with gcc9 not working To: Warner Losh , "Conrad E. Meyer" Cc: FreeBSD Current References: <4c4d4ccf-4662-572a-d6ff-7ce9a0c36f38@bsdio.com> From: Rebecca Cran Autocrypt: addr=rebecca@bsdio.com; keydata= mQINBFrUMZ4BEADI1yUEGeZeXeTCPay1ZpTBdDEpGPAw1dq2VCSTc1VhsnrEBa1iZxAfaeSv Uu5Ti7jlhQ/3sQMl0bJMKGB/RtmIW7k8h2w476oZmG8gChk8su5ZEx/pV1gdqInyFmmJKTYc gabJz8pL+m82w07qPv+oalepZ4dbj+HF++RAK/iEju+q9UHlsjj8e3mMNsvtrOz1K6bnpveO jZ+ms/2H3Hs5a4k8y6buwe2RvwhJQaXa13cR3LhzL+nwj4B9PHZZEa2WpEyYpw/bI0V9YSQN QgC1CYRzDyakZge6BCM6wHOgZSUzRPufGilrNKUwIVbRoIBR9/85+0wR+PlFUOUOfOc6ox7T dWcIx6PuPhek48rh4uwmmwsPtPiH4Z3T5p+GmWQ9NLFZKA1YnEdaSkWtYZsDxwVZZeYG2plt MfhXP0Hj4rf9Y3eoUenCaGioxAbUOBCtXdTGNAhNjz1g5NGDBVyhjKkzwJQvt9UrYTseERit 5dX2CMTy8hYLvSXd/Ivy+HylUS5IslfZxW5z9LgWX7Z97kILgkH3N0ewtLkygkG+Y+x7uaAV dFqp9ASOyzaiwKbJdeOI+WxRSh+AqeCR0S+bpkcLudLmbjrPmaFwjKycy1H85Z5R2J3YHyXY oT6OYjD8vLbUU2GWp6Onkcy1Pu8EMbRuzKil6HnpYg3BexbPFwARAQABtCNSZWJlY2NhIENy YW4gPHJlYmVjY2FAYmx1ZXN0b3Aub3JnPokCVwQTAQgAQQIbIwUJCWYBgAULCQgHAgYVCgkI CwIEFgIDAQIeAQIXgBYhBB+5fZtkTdO940Yr4g0CK1MRvhAgBQJa2B9zAhkBAAoJEA0CK1MR vhAgzJEQAJUqVmTRO9OqCSS2CVKjrqnEWJMvyo0K8B+WiXo0nSQg9+uyoVU7h2s/kkWVGy4u IWbGy2Qe8LiXzBJjHC3TadGvOvakfdMeKKXcgxgX6KlhA9hA2LW6tg22aHUk7Flr/8diHpgf qIwrXhqJXZmK72GR1QfhgoHsOsTJ9GWPswo1kUMc0cJowq0qP1RDdua6BwvDHHPJwu9OmC/i oQlMNm9gkBDq8H2B+m125ANwCnqBizXaiTTLQdewTMbCSuxbsni2icDqwBfFXzEgcJGaYYfB cQeFsfCmtXQK3JUd4Myx128Dxk9P3X64I93SB7QzB0nmWlyvmCFBNoCp0PCLA4qbwbw2sMRX Wx4BqYa8nI/jg+Nqo+Ut2BfltNZIlsHxK+XhxejfLqAjRCZeLnu1otvFnFuGLaAVYx9x1Y1q J8VizZxq6ujio62Qpultp6KNhlkJ+OKoGwA0k4NHh26SxvlsNxlfg/2v9b1LqWRzNujnwbcF 8g4902XjyBLxV+9YpXZEa8H6zzEHxpeDPWT3QfvrT8JuoHa1IyYnUKvG674UKW5zEGEwkQc9 cuQwR1RHd1ZrKtH1duXzaLr/caMp8ZDfGDDxFpenJTRxNRlg4+K7HSdhpac7sBVMUA8uVdE+ iuTThOmdf0c4DorL3BIh6Yv3FV4/NSqT1Wn3CG2fgG1guQINBFrUMZ4BEADkc4mvMcMcDF1t dNxNQuIBE1F243oZamG3LACCKfc1Yur3CPzHwIk5LXCUmbq23iE5bowxMWw3mlVT0p5xM0Wn UidIBwCKu4kRyy/fY4NyWWBuwy9srpTdmUcKRBRNB8zEZE8xIlidD1ijjgqLBfeM7n9ylawA xHLxwU96sdpdHFzb7Z0yKY2e/bzDaHiG0fUvcCmkgLf+uwKKZid1j8zR5PzKpgPqfy/PF01e KyGV3MNu8Y90xMoiEMWfCI2IB1m+hTuzZoboFvGV54SiMuvfWK/VMQjhsL6K2ddOqwVuy2nI MI4G3xDQW/v8KVyn43OSIAyW1eaklhzu0Ir2sO60PXRkvbTUrouvmSvpJfIQS49rU0M/X6FS DgXQLKrZ3my94+g8ptz9KoVml6s4OAwYVz+sb49nuSxipFKkU5FwhKOLmzbsBxCtytcUJoLm juJPJPDQue6YJiIXyc86GVY2pH3DjemKdbB4dSgqAJIp+lCzKSJzz7bgueh2Ox8vzx1tSxKj 7V8Nal+UTKKbkxPmMh+e20YZ4esAVifO3bS6IJP/aDnfagghB71vA7+aWGXPbjPlc2UHpCBi RSsl+IgoQXvdvZBsKRyfBx8neODa2C6JIE5vcaCjilSeKF8SzsFXvimnndhQNhAPU/DwQwSX dCl4gTsFVi5d8Oxq1sce+wARAQABiQI8BBgBCAAmFiEEH7l9m2RN073jRiviDQIrUxG+ECAF AlrUMZ4CGwwFCQlmAYAACgkQDQIrUxG+ECAWnRAAsmZX+KgNxW3v7R/76Tz4Wjmh4AGeE+Ji 3p5QsdTYny1B6vYBL9vCzPJ/AK8pgKMDRaweUP5eZQpfrdWC8Q7SNGgi4Q+97KEs+i2xZLQ+ WJb8a+WEEIc716u0y4ITiHfOgM5jWcFO4MXQATbJgv0drLLesa+LQCvZgPBqupt307EsCubQ s+Sxt+RVjf6rOUolp1GJXEQYwGsKklVd6yqLC8M1BSG53/WE5tSv5GzBZ8fp6EtmjT7leuid FtEvKYHQz4DqG9ELpHUF0X0UUCBK/MgXe3kCVLKE060UrJ4M6uPSx57rmVFA2MvwQR8M7GsW C5UsSM4PYwPWBhwxE7vcx0691YKAHT/5q8LxRVBdUyzPSprMhSQFttsBt+ygm6wRi3Pi3TuC EARNubPkQefyeC34yr40SAUCkOl3eWxSXPF4NfXFQb4AAzZSE5hv3qbDuwo3lrL0LqpIpEQP Az+JZ1QZ6mMFQ5/JD9Gukj54kZc0X8w3sQt0a8vyE/qrJg8vKgv2rCHrPc5MeDkEUEFiiJiC EDdkJtMyoRlU3S4NrnbyLOLEcHE8fGe3hStPX8hY62id2ecdQ5WZ7vLZW5SFeLarbUciuHIk VL6MHnUjbV7XlY50N7ebeFCIdlCWhdum2FJs/Ni+SSxbZC564vrokwlBBGSo6WTPQTa8IWx1 DtU= Message-ID: Date: Tue, 10 Sep 2019 16:22:11 -0600 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.1.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 46SffF5pK5z46XH X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of rebecca@bsdio.com has no SPF policy when checking 65.103.231.193) smtp.mailfrom=rebecca@bsdio.com X-Spamd-Result: default: False [-2.93 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.991,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[bsdio.com]; AUTH_NA(1.00)[]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:209, ipnet:65.103.224.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(-1.84)[ip: (-9.09), asn: 209(-0.05), country: US(-0.05)] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2019 22:22:14 -0000 On 2019-09-09 23:08, Warner Losh wrote: > >> Aspirationally, yes. I did a successful CROSS_TOOLCHAIN=amd64-gcc >> buildworld earlier this week. (It doesn't play well with binary pkg's >> built with Clang, so I ended up replacing it with a Clang-built world >> instead, but it compiled.) >> >> Unfortunately, amd64-xtoolchain-gcc is stuck on GCC 6.4.0, while >> you're running the much more recent GCC9. I think GCC6.4.0 is more or >> less expected to build world today, but I don't think many people are >> building with GCC9. I would love for amd64-xtoolchain to move to a >> newer version, but I don't know what is blocking that — it seems like >> it should be straightforward. >> > I did a gcc8 build about a year or so ago, though I had to turn off Werror > to complete the build... Thanks. I kept running into build errors with gcc 8 and 9 (even with WERROR= in src.conf), but building with gcc 6 from the amd64-gcc port worked. -- Rebecca Cran From owner-freebsd-current@freebsd.org Wed Sep 11 05:58:14 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B155FEF70B for ; Wed, 11 Sep 2019 05:58:14 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic307-10.consmr.mail.ne1.yahoo.com (sonic307-10.consmr.mail.ne1.yahoo.com [66.163.190.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46SrmP40Sbz4Sq9 for ; Wed, 11 Sep 2019 05:58:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: _au2af4VM1lqpJ0A4OskBEOOs.7EvmIwIN4YohKajFTSAq2hnAIjO1P2GFt08oi rEfT8EI.WwPwxT1kut1qhkh.gNi_OPuBEgwU_Qf517JGzasHOfZoUg3Dt_WOD2WKMN_cIwZ2HIga e7Qijx6ULs5n1IWT.v.woZYMLBrKccsXOxmNQt7uP7_SvgDHzS9bSQiYX37I2rjNA8npQO2PGv66 gHSWvXiiuLrrcPXcP_HxbgPVYasO072RxP0LHQs5EUjSQShVpHvhlGeAjmwN5m13Ancrmgmxef5q ZQ7Fre4YgJ5rfkesLnxGbdsX3hlYW_zkckyfKqh64Hk_1k.OD0qs5WutVEFAqZPhrmk6kConUMa0 l2jdHjCPfwdzqa_o8bmX3LbnZ1kDyps6ILOlE5gJfES4Vh93BcB_qAGCatQX0h.SwEN6UET1j846 ZKcY1GPpc1BRrtZIR3CSyYDLSbQPyXlj4xtqGlLCuZwckvagnGWL4l_avA0umuw4vgNDzKIV7Ab7 KBl95eYZYWnDJhuC6kAlDwHMTlUX6DKFZegZ3NZPOm0A3PIYxBJSom1dOdXBD_8jd_aGenZDhcUe DOK3MpaBpNOorhOS9JDzWE1KnQS234Ordl3G.kDhrtMQXAQoNFjxolqF7w.4jGdCpdhBVdLU0Ct9 y74oOrt6Z.xpIW.i9XaHxVIjI8oTNUDirVJLM451vhVzGubdg1bNCt3zyvbudhC5FyzdqecRF8T7 qZLZHJ_WADVUpALDPBpaOJQM1Pui5gy7C3F43P84R9TFJfGNfe138mQvBCDpmRyH8jSH2sWmAbA3 THLOlfcM4uErz1ZMiNpYY.qPlULtDwsqwf89EUJ2mZKjYdnc1CO59pftq6fTkviqp_DSSBoRaOqp ZChM62Bcn.QZGtJbrBdpIm0bwTe_8ixa96gagTNrHE.Lh3C99OzXj1A7DIi4Ty2gM.P9jCP9xY0Q rnxQ027f.17VnDY4XmTmWKAlckwvihZUkr3L0oOuJAAX.SY8PlfGW_tAULKX.sEdt0a_SBgWq0nL BhcGdatZVr8bAryAQnfVIp.aFqZPztWFrns42QkEFFKbkd.1UU3XU4d5N1yfihlfLbGOMCeX5syE 5uF9Wln6FkEzJA__ouJCuZ4.A2IEfKv8Hlvx6A.5kM0qWMjk9d07_XWplwh8FQ_rsHsr6.7Jpqir clasZRFPicaboeSWJaciNsdS8xIfoLAfF2_EFmNSZ0.jxlp_bVMU8OFmWC9wtx3e7m4xnjotCHWI telyi6qplRILDda_RphLs_BQJM.wD1mZNsn0rqasRqFrXwg-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.ne1.yahoo.com with HTTP; Wed, 11 Sep 2019 05:58:11 +0000 Received: by smtp426.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID eb6d57ddc44af5fcdf86d8632c91fff3; Wed, 11 Sep 2019 05:58:06 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: "cpuset -n prefer:?" --what values for "?" are supposed to be allowed? (only 1 is, despite two numa domains) Message-Id: Date: Tue, 10 Sep 2019 22:58:05 -0700 To: FreeBSD Current X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 46SrmP40Sbz4Sq9 X-Spamd-Bar: - X-Spamd-Result: default: False [-1.50 / 15.00]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.86)[-0.859,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.14)[-0.139,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(0.00)[ip: (3.85), ipnet: 66.163.184.0/21(1.30), asn: 36646(1.04), country: US(-0.05)]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; SUBJECT_HAS_QUESTION(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[33.190.163.66.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Sep 2019 05:58:14 -0000 In a context with: # cpuset -g pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, = 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27 pid -1 domain policy: first-touch mask: 0, 1 I get: # cpuset -l0 -n prefer:0 COMMAND cpuset: setdomain: Invalid argument # cpuset -l0 -n prefer:2 COMMAND cpuset: setdomain: Invalid argument But one prefer:? value does allow the COMMAND to run: # cpuset -l0 -n prefer:1 COMMAND This seem odd to me. Am I missing something? For reference: I'm using a ThreadRipper 1950X with a head -r351227 based context for this activity. The above happens to have been run in a Windows 10 Pro HyperV session, instead of in a native-boot of the same media. (A native-boot would have had 32 CPUs.) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-current@freebsd.org Wed Sep 11 06:28:44 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4E96BF056E for ; Wed, 11 Sep 2019 06:28:44 +0000 (UTC) (envelope-from clay.daniels.jr@gmail.com) Received: from mail-ua1-x941.google.com (mail-ua1-x941.google.com [IPv6:2607:f8b0:4864:20::941]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46SsRb3C42z4VHs for ; Wed, 11 Sep 2019 06:28:43 +0000 (UTC) (envelope-from clay.daniels.jr@gmail.com) Received: by mail-ua1-x941.google.com with SMTP id n6so6408587uaq.3 for ; Tue, 10 Sep 2019 23:28:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=FzaLkuM7iGciEFr7QmjPzfPSYFFgdY3+qWXSfX8B/gI=; b=OlDWOCFLR4tfUgb0aalv5Y44v+Fm25rsZSUdDCwFoBY6SIbk2zspNX4e0GiAJsDNZs 6RqogL0wSc+m0HDQv2dJzHwTWf5JK4vtBCwncQZPS69ADgwCkldYRO2kFo370kXOlE6G UCKHRyvPxwpznV6HqSPaD7Thv+jsqZfAGJcShMg1ewFgYa0gOqjtHI9trFvwdYZI4goP 5lNFqV3bJfZVDXl9SpVqmG8nnzTnEdcYjq2yZAnEm35Ivs/stmD9N0MedVVzHT+A4nk9 avIv+COTjDVqGHaAfFJEQAPudHNeJSUsCef/AsEmtWCxij0vBdyNibIlnUWQ28JwZfqK LWUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=FzaLkuM7iGciEFr7QmjPzfPSYFFgdY3+qWXSfX8B/gI=; b=aCtAItGzVcjkFLA6Hqj9kfsGsH7rSbDO7zAIUtXMsihfd4up8CJvYVYUEeurf50o3L ZHxYBVpQSHl16Hpp7VxKnbXhU4tdB2yeiS4pOHD12CQVE3fwY5+jshaFYqZ8owp54Vr9 CyunXCdmbBD7KxvQz1eVaYjSItBS+lzaXvTnF4/MWl9hCSBE9xy4Jq76tzrKxEOkSR2V cvyvKYo0t5LY1arTGGpxDXEfw1b+/YCJQwkDgGulQtr1A7g4+YRKBYXbuhCCSYd8TtYz ETTWNAYk9/ujYHaz06MZ/VofFlshbgdrcSM2Cz/PHDdS4dANIWop7R9hLYzp/EBnfrvx rYrw== X-Gm-Message-State: APjAAAWKzO/XiYdxNHCgKbSSJ+hBtIhX9qDNU9+HuCBiJYZdsgzTC/V2 /zAaN45Cf4Bk2/DlBXeiezakKJySOzZOtEBuqw== X-Google-Smtp-Source: APXvYqz1zev3kZT4DYzdcDYPQqnNOdaD935siS3BmEAn++iMYQNi1ZJDub834U2dqRwlcMBrCDGbPCO/0b7DbkxAWmU= X-Received: by 2002:ab0:5b1a:: with SMTP id u26mr15849532uae.125.1568183321754; Tue, 10 Sep 2019 23:28:41 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: "Clay Daniels Jr." Date: Wed, 11 Sep 2019 01:28:30 -0500 Message-ID: Subject: Re: "cpuset -n prefer:?" --what values for "?" are supposed to be allowed? (only 1 is, despite two numa domains) To: Mark Millard Cc: FreeBSD Current X-Rspamd-Queue-Id: 46SsRb3C42z4VHs X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=OlDWOCFL; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of claydanielsjr@gmail.com designates 2607:f8b0:4864:20::941 as permitted sender) smtp.mailfrom=claydanielsjr@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; URI_COUNT_ODD(1.00)[7]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FREEMAIL_TO(0.00)[yahoo.com]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (2.78), ipnet: 2607:f8b0::/32(-2.73), asn: 15169(-2.26), country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[1.4.9.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Sep 2019 06:28:44 -0000 Mark, this is what I get on my machine: root@new:~ # cpuset -g pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 pid -1 domain policy: first-touch mask: 0 root@new:~ # cpuset -l0 -n prefer:0 COMMAND cpuset: COMMAND: No such file or directory root@new:~ # cpuset -l0 -n prefer:2 COMMAND cpuset: setdomain: Invalid argument root@new:~ # cpuset -l0 -n prefer:1 COMMAND cpuset: setdomain: Invalid argument >From dmesg: FreeBSD 13.0-CURRENT r351901 GENERIC amd64 CPU: AMD Ryzen 7 3700X 8-Core Processor(3600.08-MHz K8-class CPU) Similar, Clay On Wed, Sep 11, 2019 at 12:58 AM Mark Millard wrote: > In a context with: > > # cpuset -g > pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, > 18, 19, 20, 21, 22, 23, 24, 25, 26, 27 > pid -1 domain policy: first-touch mask: 0, 1 > > I get: > > # cpuset -l0 -n prefer:0 COMMAND > cpuset: setdomain: Invalid argument > > # cpuset -l0 -n prefer:2 COMMAND > cpuset: setdomain: Invalid argument > > But one prefer:? value does allow the COMMAND > to run: > > # cpuset -l0 -n prefer:1 COMMAND > > This seem odd to me. Am I missing something? > > For reference: I'm using a ThreadRipper 1950X > with a head -r351227 based context for this > activity. The above happens to have been run > in a Windows 10 Pro HyperV session, instead > of in a native-boot of the same media. (A > native-boot would have had 32 CPUs.) > > > === > Mark Millard > marklmi at yahoo.com > ( dsl-only.net went > away in early 2018-Mar) > > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-current@freebsd.org Wed Sep 11 13:15:44 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3BE97D37CC; Wed, 11 Sep 2019 13:15:44 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lj1-f171.google.com (mail-lj1-f171.google.com [209.85.208.171]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46T2TB6q1qz3Qyr; Wed, 11 Sep 2019 13:15:42 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lj1-f171.google.com with SMTP id 7so19933007ljw.7; Wed, 11 Sep 2019 06:15:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:openpgp:autocrypt:message-id :date:user-agent:mime-version:content-language :content-transfer-encoding; bh=qQcFrJ7yQyPK1T9347wZLUod9c/b5wL2lS43WxHE1Wg=; b=EdAcXtHTDvB2cKcNdHMLcXhHgy1p1c7Si9eJWQUGSoivPS9FzT4Erg+bvHAdE1tptS 2oPWYmrNgrr0EbWKPBT0pHxAbFpmnxInLsuvl3ptDucsdQ4I+sT/UGOCdfFRNnoxMKb0 SrXNrBXNvpJnNXaBj6n5OgOBS/FSVXMGeh9bXzuAIR+pYEG6JfI1FVGzYlgksmCoF2JM YdZzQornnKgs3ESaKY07hc+yNXlRv8lfF1cQq/Y4F9trP6jQGIuQNOddZKvqotUHMim/ 1ORIE71Wyg+CaZ5jenAkM/RTYfdWtfgrcFGdyOQ6i0mtmnB8n+ASFBcK9ys1EWQhNvZZ 1JyQ== X-Gm-Message-State: APjAAAUCDJxyESjyI1wJVlBphAjlkefPE1fpBxUFh7XAx/y0cSIvOPRZ L7z9lGV93OnMy5gdVHaJazvEFU/p X-Google-Smtp-Source: APXvYqxzaAH++zrLj4oRkGVLSjUjhFfJRiNPIq/ZbE053Nh6YZtZBa+ZH/woQ3M4Brv9/53Shd3ubA== X-Received: by 2002:a2e:7318:: with SMTP id o24mr22831096ljc.111.1568207740728; Wed, 11 Sep 2019 06:15:40 -0700 (PDT) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id l3sm5155923lfc.31.2019.09.11.06.15.38 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 11 Sep 2019 06:15:39 -0700 (PDT) To: freebsd-current@FreeBSD.org, freebsd-stable@FreeBSD.org From: Andriy Gapon Subject: ixv + RSS Openpgp: preference=signencrypt Autocrypt: addr=avg@FreeBSD.org; prefer-encrypt=mutual; keydata= mQINBFm4LIgBEADNB/3lT7f15UKeQ52xCFQx/GqHkSxEdVyLFZTmY3KyNPQGBtyvVyBfprJ7 mAeXZWfhat6cKNRAGZcL5EmewdQuUfQfBdYmKjbw3a9GFDsDNuhDA2QwFt8BmkiVMRYyvI7l N0eVzszWCUgdc3qqM6qqcgBaqsVmJluwpvwp4ZBXmch5BgDDDb1MPO8AZ2QZfIQmplkj8Y6Z AiNMknkmgaekIINSJX8IzRzKD5WwMsin70psE8dpL/iBsA2cpJGzWMObVTtCxeDKlBCNqM1i gTXta1ukdUT7JgLEFZk9ceYQQMJJtUwzWu1UHfZn0Fs29HTqawfWPSZVbulbrnu5q55R4PlQ /xURkWQUTyDpqUvb4JK371zhepXiXDwrrpnyyZABm3SFLkk2bHlheeKU6Yql4pcmSVym1AS4 dV8y0oHAfdlSCF6tpOPf2+K9nW1CFA8b/tw4oJBTtfZ1kxXOMdyZU5fiG7xb1qDgpQKgHUX8 7Rd2T1UVLVeuhYlXNw2F+a2ucY+cMoqz3LtpksUiBppJhw099gEXehcN2JbUZ2TueJdt1FdS ztnZmsHUXLxrRBtGwqnFL7GSd6snpGIKuuL305iaOGODbb9c7ne1JqBbkw1wh8ci6vvwGlzx rexzimRaBzJxlkjNfMx8WpCvYebGMydNoeEtkWldtjTNVsUAtQARAQABtB5BbmRyaXkgR2Fw b24gPGF2Z0BGcmVlQlNELm9yZz6JAlQEEwEIAD4WIQS+LEO7ngQnXA4Bjr538m7TUc1yjwUC WbgsiAIbIwUJBaOagAULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRB38m7TUc1yj+JAEACV l9AK/nOWAt/9cufV2fRj0hdOqB1aCshtSrwHk/exXsDa4/FkmegxXQGY+3GWX3deIyesbVRL rYdtdK0dqJyT1SBqXK1h3/at9rxr9GQA6KWOxTjUFURsU7ok/6SIlm8uLRPNKO+yq0GDjgaO LzN+xykuBA0FlhQAXJnpZLcVfPJdWv7sSHGedL5ln8P8rxR+XnmsA5TUaaPcbhTB+mG+iKFj GghASDSfGqLWFPBlX/fpXikBDZ1gvOr8nyMY9nXhgfXpq3B6QCRYKPy58ChrZ5weeJZ29b7/ QdEO8NFNWHjSD9meiLdWQaqo9Y7uUxN3wySc/YUZxtS0bhAd8zJdNPsJYG8sXgKjeBQMVGuT eCAJFEYJqbwWvIXMfVWop4+O4xB+z2YE3jAbG/9tB/GSnQdVSj3G8MS80iLS58frnt+RSEw/ psahrfh0dh6SFHttE049xYiC+cM8J27Aaf0i9RflyITq57NuJm+AHJoU9SQUkIF0nc6lfA+o JRiyRlHZHKoRQkIg4aiKaZSWjQYRl5Txl0IZUP1dSWMX4s3XTMurC/pnja45dge/4ESOtJ9R 8XuIWg45Oq6MeIWdjKddGhRj3OohsltKgkEU3eLKYtB6qRTQypHHUawCXz88uYt5e3w4V16H lCpSTZV/EVHnNe45FVBlvK7k7HFfDDkryLkCDQRZuCyIARAAlq0slcsVboY/+IUJdcbEiJRW be9HKVz4SUchq0z9MZPX/0dcnvz/gkyYA+OuM78dNS7Mbby5dTvOqfpLJfCuhaNYOhlE0wY+ 1T6Tf1f4c/uA3U/YiadukQ3+6TJuYGAdRZD5EqYFIkreARTVWg87N9g0fT9BEqLw9lJtEGDY EWUE7L++B8o4uu3LQFEYxcrb4K/WKmgtmFcm77s0IKDrfcX4doV92QTIpLiRxcOmCC/OCYuO jB1oaaqXQzZrCutXRK0L5XN1Y1PYjIrEzHMIXmCDlLYnpFkK+itlXwlE2ZQxkfMruCWdQXye syl2fynAe8hvp7Mms9qU2r2K9EcJiR5N1t1C2/kTKNUhcRv7Yd/vwusK7BqJbhlng5ZgRx0m WxdntU/JLEntz3QBsBsWM9Y9wf2V4tLv6/DuDBta781RsCB/UrU2zNuOEkSixlUiHxw1dccI 6CVlaWkkJBxmHX22GdDFrcjvwMNIbbyfQLuBq6IOh8nvu9vuItup7qemDG3Ms6TVwA7BD3j+ 3fGprtyW8Fd/RR2bW2+LWkMrqHffAr6Y6V3h5kd2G9Q8ZWpEJk+LG6Mk3fhZhmCnHhDu6CwN MeUvxXDVO+fqc3JjFm5OxhmfVeJKrbCEUJyM8ESWLoNHLqjywdZga4Q7P12g8DUQ1mRxYg/L HgZY3zfKOqcAEQEAAYkCPAQYAQgAJhYhBL4sQ7ueBCdcDgGOvnfybtNRzXKPBQJZuCyIAhsM BQkFo5qAAAoJEHfybtNRzXKPBVwQAKfFy9P7N3OsLDMB56A4Kf+ZT+d5cIx0Yiaf4n6w7m3i ImHHHk9FIetI4Xe54a2IXh4Bq5UkAGY0667eIs+Z1Ea6I2i27Sdo7DxGwq09Qnm/Y65ADvXs 3aBvokCcm7FsM1wky395m8xUos1681oV5oxgqeRI8/76qy0hD9WR65UW+HQgZRIcIjSel9vR XDaD2HLGPTTGr7u4v00UeTMs6qvPsa2PJagogrKY8RXdFtXvweQFz78NbXhluwix2Tb9ETPk LIpDrtzV73CaE2aqBG/KrboXT2C67BgFtnk7T7Y7iKq4/XvEdDWscz2wws91BOXuMMd4c/c4 OmGW9m3RBLufFrOag1q5yUS9QbFfyqL6dftJP3Zq/xe+mr7sbWbhPVCQFrH3r26mpmy841ym dwQnNcsbIGiBASBSKksOvIDYKa2Wy8htPmWFTEOPRpFXdGQ27awcjjnB42nngyCK5ukZDHi6 w0qK5DNQQCkiweevCIC6wc3p67jl1EMFY5+z+zdTPb3h7LeVnGqW0qBQl99vVFgzLxchKcl0 R/paSFgwqXCZhAKMuUHncJuynDOP7z5LirUeFI8qsBAJi1rXpQoLJTVcW72swZ42IdPiboqx NbTMiNOiE36GqMcTPfKylCbF45JNX4nF9ElM0E+Y8gi4cizJYBRr2FBJgay0b9Cp Message-ID: <11438596-7ac3-e5c2-b963-6e0690e94265@FreeBSD.org> Date: Wed, 11 Sep 2019 16:15:38 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46T2TB6q1qz3Qyr X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of agapon@gmail.com designates 209.85.208.171 as permitted sender) smtp.mailfrom=agapon@gmail.com X-Spamd-Result: default: False [-3.22 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[FreeBSD.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.22)[ip: (-0.47), ipnet: 209.85.128.0/17(-3.32), asn: 15169(-2.25), country: US(-0.05)]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[171.208.85.209.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FORGED_SENDER(0.30)[avg@FreeBSD.org,agapon@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[171.208.85.209.rep.mailspike.net : 127.0.0.17]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[avg@FreeBSD.org,agapon@gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[96.151.72.93.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Sep 2019 13:15:44 -0000 Has anyone ever tested a kernel with option RSS and an ixv interface? Just looking for some data points. Thanks! -- Andriy Gapon From owner-freebsd-current@freebsd.org Wed Sep 11 14:31:33 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4A6ACD5F13 for ; Wed, 11 Sep 2019 14:31:33 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd36.google.com (mail-io1-xd36.google.com [IPv6:2607:f8b0:4864:20::d36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46T48h27Dqz41MN for ; Wed, 11 Sep 2019 14:31:32 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-io1-xd36.google.com with SMTP id r4so46266452iop.4 for ; Wed, 11 Sep 2019 07:31:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=HY78RraKo+uBYn5dUskyEQdRVQaBrexZduCSjVPl7Pk=; b=EVH9196pfG6gN81SzWOYMNV2fpjfxOWVcizKAhad+tAl/tnnfMkHf6ARdj8rNjt6GZ q/ixw9o7wfZY8me8m/aF5PeFxkJm7Qhjh2D3WuUInkqzoDeGKgltI6bYF+rklk9DWkiW qTOFKJntN1zlOXr3dYWqTSgpTo8yqkcLrl86AANKvanpTOfxjaBleuS0s+k6TVBvUiFX ojVpEtbQztjfUOMidE4TZwzkymWPYETykHGafTPJ0EiXgf8Rz9cAyq6O3LEOeIw9+6RA pDFisGrsj4iWneZCLX5OBR4h7h/gmtJDiUbaRsWe+2FyeAL9BB8gg1/aprSRuly52KFo TIrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=HY78RraKo+uBYn5dUskyEQdRVQaBrexZduCSjVPl7Pk=; b=dFpE4xWJon6cvQVXHS/EXsKbFVFQ759nN+tlIrbXvuy/QqBr/yPh4VFgtoHu/6wjjY oFiQjahnw5XCn7umjMxhLZGVk8vliRr5xY5WaMAiaVmWWQlVt2ELVX60Wm2xaKk7Jpya quix5SWvGXYmtLhC66xXwjwHc5WmOKGwYh8gCnXKieNlHHofavu6652Vi8BTCWE4Sqwb oRQE7hyVUkjSxedOVNbEN0hlGcvyRvrLU9GyHp+aWmeZcFpZguJFE10uuDFn6NctHMe6 JfiLygdkh3rlqWH2fz4nQ7IuQddtpLDEjsgjC6BgTJi99t5nJfKCZ6lDuS7doCiM98tT bThg== X-Gm-Message-State: APjAAAUkhgZRTb5Ntfj/OJNzDJGFfK/QOi8Tl8l/9oY/cYAwfrEbdKhN XFtVIb28CtD4eDC/wMm4uEA= X-Google-Smtp-Source: APXvYqwRPFDKIRNDh8Ej3i3HFEnN1QnzympQ6jjcObqHecAeLwjWEumxq64UVwGj0IUCTScKCbahVA== X-Received: by 2002:a05:6638:928:: with SMTP id 8mr37214168jak.124.1568212290972; Wed, 11 Sep 2019 07:31:30 -0700 (PDT) Received: from raichu (toroon0560w-lp140-01-69-159-39-167.dsl.bell.ca. [69.159.39.167]) by smtp.gmail.com with ESMTPSA id c4sm16575099ioa.76.2019.09.11.07.31.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2019 07:31:30 -0700 (PDT) Sender: Mark Johnston Date: Wed, 11 Sep 2019 10:31:25 -0400 From: Mark Johnston To: Mark Millard Cc: FreeBSD Current Subject: Re: "cpuset -n prefer:?" --what values for "?" are supposed to be allowed? (only 1 is, despite two numa domains) Message-ID: <20190911143125.GA17992@raichu> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 46T48h27Dqz41MN X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=EVH9196p; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::d36 as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-3.80 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; FREEMAIL_TO(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-2.10)[ip: (-5.45), ipnet: 2607:f8b0::/32(-2.72), asn: 15169(-2.25), country: US(-0.05)]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; SUBJECT_HAS_QUESTION(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[6.3.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Sep 2019 14:31:33 -0000 On Tue, Sep 10, 2019 at 10:58:05PM -0700, Mark Millard wrote: > In a context with: > > # cpuset -g > pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27 > pid -1 domain policy: first-touch mask: 0, 1 > > I get: > > # cpuset -l0 -n prefer:0 COMMAND > cpuset: setdomain: Invalid argument > > # cpuset -l0 -n prefer:2 COMMAND > cpuset: setdomain: Invalid argument > > But one prefer:? value does allow the COMMAND > to run: > > # cpuset -l0 -n prefer:1 COMMAND > > This seem odd to me. Am I missing something? > > For reference: I'm using a ThreadRipper 1950X > with a head -r351227 based context for this > activity. The above happens to have been run > in a Windows 10 Pro HyperV session, instead > of in a native-boot of the same media. (A > native-boot would have had 32 CPUs.) Can you please show the output of "sysctl vm.phys_segs" from this setup? From owner-freebsd-current@freebsd.org Wed Sep 11 14:57:31 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4F4B5D6804 for ; Wed, 11 Sep 2019 14:57:31 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-21.consmr.mail.gq1.yahoo.com (sonic310-21.consmr.mail.gq1.yahoo.com [98.137.69.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46T4kf1x5lz42Vq for ; Wed, 11 Sep 2019 14:57:29 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: RQgq3n8VM1lZeiY4pzg2EFgC1q1Jd7cvbMH3BSJfbpkB85rba3nYa5nTf8eCaJz Yscsdvua0WCzj6Nv0roRgPJ3BAcm2R3CnsXqMezgc4ei.ahBNhRd9heHPx.vX8V8vudO9ZFw1hYn 1fBboZQlT6EIbtFNipmeArs2M38DXPhaPCt52B77_1POxikg78K.j3gPYyt3jVhMB.33DijBZsQh adojktyIRYvB_x8z5bJNSvAB4CiRh66D0FG.qbYa_pGQfvfTXhHOpALwj0_bvAlaiDgnCAmREeIF NM73qL4O8n7Bh7fzoi1s4KpIA36k4EF6zR5EAINX5WWrGrC2Wqt0nLwUPi8ygB9Hsqi3z1_efzAP 2IpijJ3x215IG8F1jZWb.3J.4tpU8bqja837nXSmlIG_rkqG1ghDD0Ztmssbj946kk4vhntmrgcB 7HXGfxwBo_l7hB8Rw2dmfwwIykQhHrinZ6vMiu1l10B3MW_GzehwxYOgATqAZiNWjJCxp1yq7IKs a.cVXyrKVh_BA8urIaBlh3cKNLt1NoEfYu4uftjcsqA.ISYHN_UqCrwPEJSWWNsiOA_3Xm9x6qN7 Sf2.wi4z.1xnj813WLKEhcCfYFUsiJdRVqb91qrUW8cKZryHFfW87x_FdZ3sVc2PkHO.fCXpm1s6 fObmNBEmvgfsB4lpISR2DYAb60hIw.Ml7Cnjb6XdWxOQPvQfW6akdpjsCj05SwCAoq_.eRpBh1zi UqYgphuhtU6n1R69fA.91uuOWzuG9tDMRo2MXXVbgth0BQY184XI8Hwn1PI0zW.gObRdMfHLDfVO UlgM3Xg0FNnst1NwQVz0pTuRT0xyKQ9a2_68sSBbiBSyVzgOVObtiDI9DwJP9opTEUrUaR9ZRfxx jJr2XJZ0FVl4wzlaJECXu78UEiaEG5KzV.xn0VnQgCjqW8vNodMo3WqHU9SxVqo45UG3nWM94qTO kARzAA2nHqxW2ocPBZxYOjxnzPaYqjQpKZNm2.uoYzqlhpwTHE0Sg7NrALWbT0.l.twZJFx83kka HPI9gGwLynBfVx_T9Ms6nKPt1EM2vWnhOttBfGyqscYP_N_k7bZsnYhliLQyM4PbcCUs59Tsk3YT LeDivAwCtlUoL596yJsqPiGUBXYAE53gqGuw2._Vk.FX7RVuKXGsQytuW8vQpDrjSjPIyJp.F1b3 Yq1wnvY1nVf3MgiD1SKqgbrGy9rPDOm6AFrknoeKw0MPgWVCe7R5M6EkA6E_ryHpymSzJMtPkYNz n_gaVYeprl_eq.up45sMlFIp988UjY_9kGnRnatyb0gpMaX96Gn2wjmuUse4- Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Wed, 11 Sep 2019 14:57:28 +0000 Received: by smtp430.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 40a50c2fca182dcd8f557fc7a02c9985; Wed, 11 Sep 2019 14:57:27 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: "cpuset -n prefer:?" --what values for "?" are supposed to be allowed? (only 1 is, despite two numa domains) From: Mark Millard In-Reply-To: <20190911143125.GA17992@raichu> Date: Wed, 11 Sep 2019 07:57:26 -0700 Cc: FreeBSD Current Content-Transfer-Encoding: quoted-printable Message-Id: <99BB5653-1F42-4309-9892-24029FD02E39@yahoo.com> References: <20190911143125.GA17992@raichu> To: Mark Johnston X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 46T4kf1x5lz42Vq X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.14 / 15.00]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (3.91), ipnet: 98.137.64.0/21(0.92), asn: 36647(0.74), country: US(-0.05)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.85)[-0.851,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; NEURAL_HAM_LONG(-0.79)[-0.793,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[147.69.137.98.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Sep 2019 14:57:31 -0000 On 2019-Sep-11, at 07:31, Mark Johnston wrote: > On Tue, Sep 10, 2019 at 10:58:05PM -0700, Mark Millard wrote: >> In a context with: >>=20 >> # cpuset -g >> pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, = 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27 >> pid -1 domain policy: first-touch mask: 0, 1 >>=20 >> I get: >>=20 >> # cpuset -l0 -n prefer:0 COMMAND >> cpuset: setdomain: Invalid argument >>=20 >> # cpuset -l0 -n prefer:2 COMMAND >> cpuset: setdomain: Invalid argument >>=20 >> But one prefer:? value does allow the COMMAND >> to run: >>=20 >> # cpuset -l0 -n prefer:1 COMMAND >>=20 >> This seem odd to me. Am I missing something? >>=20 >> For reference: I'm using a ThreadRipper 1950X >> with a head -r351227 based context for this >> activity. The above happens to have been run >> in a Windows 10 Pro HyperV session, instead >> of in a native-boot of the same media. (A >> native-boot would have had 32 CPUs.) >=20 > Can you please show the output of "sysctl vm.phys_segs" from this > setup? Sure: # sysctl vm.phys_segs vm.phys_segs:=20 SEGMENT 0: start: 0x1000 end: 0x9f000 domain: 0 free list: 0xffffffff8281daa0 SEGMENT 1: start: 0x103000 end: 0x1000000 domain: 0 free list: 0xffffffff8281daa0 SEGMENT 2: start: 0x1000000 end: 0x2ee1000 domain: 0 free list: 0xffffffff8281d830 SEGMENT 3: start: 0x2eea000 end: 0x2f23000 domain: 0 free list: 0xffffffff8281d830 SEGMENT 4: start: 0x3000000 end: 0xf7ff0000 domain: 0 free list: 0xffffffff8281d830 SEGMENT 5: start: 0x100002000 end: 0x9c5562000 domain: 0 free list: 0xffffffff8281d5c0 SEGMENT 6: start: 0xa07c00000 end: 0xa07d50000 domain: 0 free list: 0xffffffff8281d5c0 SEGMENT 7: start: 0xa08001000 end: 0xf9ee00000 domain: 1 free list: 0xffffffff8281dd10 SEGMENT 8: start: 0x1000000000 end: 0x1427fe6000 domain: 1 free list: 0xffffffff8281dd10 And confirming the oddity is still the case (I'd rebooted since the report): # cpuset -g pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, = 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27 pid -1 domain policy: first-touch mask: 0, 1 # cpuset -l0 -n prefer:0 COMMAND cpuset: setdomain: Invalid argument # cpuset -l0 -n prefer:2 COMMAND cpuset: setdomain: Invalid argument # cpuset -l0 -n prefer:1 COMMAND cpuset: COMMAND: No such file or directory =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-current@freebsd.org Wed Sep 11 15:15:18 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 57A16D6FEC for ; Wed, 11 Sep 2019 15:15:18 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd34.google.com (mail-io1-xd34.google.com [IPv6:2607:f8b0:4864:20::d34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46T5791dVHz43M4 for ; Wed, 11 Sep 2019 15:15:16 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-io1-xd34.google.com with SMTP id f4so45935979ion.2 for ; Wed, 11 Sep 2019 08:15:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=gz3AXwftLEOIeRZOrHwHQgPzlZX9AlkUhYF7P1FtR/g=; b=RsketEWgKSLyOwOOiksc6zGYnHn0COjpulD+GP8qi31ISajuvnpB2yekTrJAn/jMDf hVkHwMnF3JQyTM/Xent6MXLow3UntpBMmh7PLNnNsVAcvUskif6+8mbyEBFUncRkH6/K QOUeG+SjwukokJO6y4s++PLv97BfxZNO6IH/HJ/n0gAuYdNFm6M2HeEsO7oN5jE2jx+I p1aFSSjaCFPj97FQDSuM3wUxnZ43C6mo1+hPCiq3kw0XCe7kv6jHP5ks+NUCgY5zZdWq MrvvZ8QUvkjvMx0GuOOc7SfnWuuUGjy1KSWeRLUDPFOowHrFuPMRXonuOHaC25luEfS9 ortw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=gz3AXwftLEOIeRZOrHwHQgPzlZX9AlkUhYF7P1FtR/g=; b=drXRN+8rl/PC3ltc9K5CjqgamE4+oVxct884AWuLsepTKhbziPG2RkOgsvTmhA7LOb 0AiyCoMuv0NUe2BeSBJaekboLfmlOkNgQIu0aoPqChmxd9frEzuOAY4OZrTDTGA0FS83 YeKfREP1FiTRzgl0CCO28SQ+qU5HlKP9rS2dJosPrga8MlPnrsBow9mpomIekEsmm6Lp nBi1sGTKX/O1yeubnGMoq17L5liP5eIsAG8OzckMZi1LITX0xirwXFEDJV/1Q3WliieT JV2Iz0uKdkmPZ8k13BUD3O0ybg5tRw2YKMO5cxrutUO14v/xT7EZOyrMhpw3ysTizqnn qEOQ== X-Gm-Message-State: APjAAAVKRkSg9xnG2QaSejYmuhSv4ZCAGcfmLBtQcmsN2uIoWbgYSaJS tfTZl7c1E/Vw6ld5clOzbIVITK6C X-Google-Smtp-Source: APXvYqypTQhylPboRqz13W/wSraaCAB0se7lzKOMf6iizx4Dzfy6Kw1Div7QJHOAAQvkixyMruZpLw== X-Received: by 2002:a02:6601:: with SMTP id k1mr39161122jac.47.1568214915680; Wed, 11 Sep 2019 08:15:15 -0700 (PDT) Received: from raichu (toroon0560w-lp140-01-69-159-39-167.dsl.bell.ca. [69.159.39.167]) by smtp.gmail.com with ESMTPSA id f7sm16080128ioc.31.2019.09.11.08.15.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2019 08:15:14 -0700 (PDT) Sender: Mark Johnston Date: Wed, 11 Sep 2019 11:15:12 -0400 From: Mark Johnston To: Mark Millard Cc: FreeBSD Current Subject: Re: "cpuset -n prefer:?" --what values for "?" are supposed to be allowed? (only 1 is, despite two numa domains) Message-ID: <20190911151512.GB17992@raichu> References: <20190911143125.GA17992@raichu> <99BB5653-1F42-4309-9892-24029FD02E39@yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <99BB5653-1F42-4309-9892-24029FD02E39@yahoo.com> User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 46T5791dVHz43M4 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=RsketEWg; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::d34 as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-3.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; FREEMAIL_TO(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-1.80)[ip: (-3.95), ipnet: 2607:f8b0::/32(-2.72), asn: 15169(-2.25), country: US(-0.05)]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; SUBJECT_HAS_QUESTION(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[4.3.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Sep 2019 15:15:18 -0000 On Wed, Sep 11, 2019 at 07:57:26AM -0700, Mark Millard wrote: > > > On 2019-Sep-11, at 07:31, Mark Johnston wrote: > > > On Tue, Sep 10, 2019 at 10:58:05PM -0700, Mark Millard wrote: > >> In a context with: > >> > >> # cpuset -g > >> pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27 > >> pid -1 domain policy: first-touch mask: 0, 1 > >> > >> I get: > >> > >> # cpuset -l0 -n prefer:0 COMMAND > >> cpuset: setdomain: Invalid argument > >> > >> # cpuset -l0 -n prefer:2 COMMAND > >> cpuset: setdomain: Invalid argument > >> > >> But one prefer:? value does allow the COMMAND > >> to run: > >> > >> # cpuset -l0 -n prefer:1 COMMAND > >> > >> This seem odd to me. Am I missing something? > >> > >> For reference: I'm using a ThreadRipper 1950X > >> with a head -r351227 based context for this > >> activity. The above happens to have been run > >> in a Windows 10 Pro HyperV session, instead > >> of in a native-boot of the same media. (A > >> native-boot would have had 32 CPUs.) > > > > Can you please show the output of "sysctl vm.phys_segs" from this > > setup? > > Sure: I was wondering if you had only one domain populated, but it seems not to be the case. Could you try updating to r351672 or later and see if the behaviour persists? From owner-freebsd-current@freebsd.org Wed Sep 11 17:11:52 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A08F1DA134 for ; Wed, 11 Sep 2019 17:11:52 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-2.consmr.mail.bf2.yahoo.com (sonic306-2.consmr.mail.bf2.yahoo.com [74.6.132.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46T7jg6B0vz4BHT for ; Wed, 11 Sep 2019 17:11:51 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: .Curh5wVM1la4tlyPFhbOv2vzZbxo9MFdbsLAT0MwfcRQSYHaVSjSdGjEC515nc 6vWJB5Xeflzk27VRPCgNaPG8aH352tSMTL3OWydjdCZc2LAAHVPr2NHUyCDUq0g5o72uJHXRL4lX S33aJZob3GgS_.RCUNk6L0rG8QvWe9y4J7lFj1Recj_tn.43nuOlgmz1xrE0YtSabrd3ocMCHeSC ZtmESgYxB1ZtjHkimSZOq05aVSXNH5hASUJOVDYATVJeiojnB49itF68.k0h2QFw_EpD5FJDxCAC wkV3MJoHKkdpyaEhRxr7vJl2KwWN6mT2kM7XvAm1ct11zbbBiugIP31Nx.mHIR.YtxMIoXo35ZA8 qOnTw1Un7ZhxCfNhH_hAFQTs8851HasL8I4pfsDRVtK6q5F9lUjvps5VjtSLpTKTZkM2ZaGTmfBJ VxdAgF6TucKTvv302R37bEAGgyOnHgEf9bGlzkHvgJnkJV.6e1Azs3Qav0y8yUWHZqiMW34bsq8L Y8YC7StIsl1jLzynKmwdTgDpchvQ2MJvY49sglz84uYKInYKufKk1iRATPzKD6.nKEHucxl_fcWC sFlo3F3MWA6LK.zIdv.wV3202bfkYj9HgkGy4FfO0tzCZIqal1JWca32QBqzAU7vIMzTIMcUB5fr I73z_jbune5S8mAcHTXKnSm6Jx3w1T.5Uz0ZFyJyE9Ykx.OC.gsqUkhzmvK2WMSJk8x7.suCtmjW DjlIiRtxG0P9pmee5pREMuPDfLLsXwpZV8kgrbxp_K3Qec6NSi9Nh.LujTbJ_88EX0woBa7Ui7GV Q.SFDFNmUQkJkva8tNKcw2ijx6Vh5Mi4n3RJ2yq26h95qQO1cE6RueDpDGXEDWvlseOOYRCUzzIM gEVyJqT6tS1xtPhjJEa7hA15xYnfPColOgmQwPoGBR2.mLSZ_D2bKiRwS9uPki1dWEGF2F2luCZe RLi6_PRan3FmSY4zp1txLVpWp85olci3MlWkOvJTEh_VIpf7Xj2pEaiYZXmSK4wmw.ff_3k9JCb7 OYhV.CttP7C9GxjK1_gWbIG1laKPpcrgUwPGQkHMmlGh0CezNo2ADVuJ_3DzYakpONe9_er.7_5V MGBt7_DvJG9xeaXDioENu3mUUIzxUcGw.zXf12qmFvmYSfrLQfzfZV1ywCM7tijYRNxdkrQKo3t_ UyPPW0SeTVR1di.3xlteJYPTd.RqX4_hMO_aMcRMn_LmJh4Y9UfjgzXDjKq6nC3ngzsFZpk7VnNk 8CEOXhbAwtbyOSK4cGrmPvBaQ6PdtKkyvwXbn8cTE5ZFDKdu6uH9PMbq7NYLS0Q-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.bf2.yahoo.com with HTTP; Wed, 11 Sep 2019 17:11:50 +0000 Received: by smtp406.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID c844bdba0cb144dafb0c5a2a48b13cca; Wed, 11 Sep 2019 17:11:47 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: "cpuset -n prefer:?" --what values for "?" are supposed to be allowed? (only 1 is, despite two numa domains) From: Mark Millard In-Reply-To: <20190911151512.GB17992@raichu> Date: Wed, 11 Sep 2019 10:11:45 -0700 Cc: FreeBSD Current Content-Transfer-Encoding: quoted-printable Message-Id: <3CE4AEB7-E32C-49BD-8C75-71AB8739BAEC@yahoo.com> References: <20190911143125.GA17992@raichu> <99BB5653-1F42-4309-9892-24029FD02E39@yahoo.com> <20190911151512.GB17992@raichu> To: Mark Johnston X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 46T7jg6B0vz4BHT X-Spamd-Bar: - X-Spamd-Result: default: False [-1.82 / 15.00]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (3.75), ipnet: 74.6.128.0/21(1.45), asn: 26101(1.16), country: US(-0.05)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.74)[-0.739,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; NEURAL_HAM_LONG(-0.58)[-0.583,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[41.132.6.74.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Sep 2019 17:11:52 -0000 On 2019-Sep-11, at 08:15, Mark Johnston wrote: > On Wed, Sep 11, 2019 at 07:57:26AM -0700, Mark Millard wrote: >>=20 >>=20 >> On 2019-Sep-11, at 07:31, Mark Johnston wrote: >>=20 >>> On Tue, Sep 10, 2019 at 10:58:05PM -0700, Mark Millard wrote: >>>> In a context with: >>>>=20 >>>> # cpuset -g >>>> pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, = 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27 >>>> pid -1 domain policy: first-touch mask: 0, 1 >>>>=20 >>>> I get: >>>>=20 >>>> # cpuset -l0 -n prefer:0 COMMAND >>>> cpuset: setdomain: Invalid argument >>>>=20 >>>> # cpuset -l0 -n prefer:2 COMMAND >>>> cpuset: setdomain: Invalid argument >>>>=20 >>>> But one prefer:? value does allow the COMMAND >>>> to run: >>>>=20 >>>> # cpuset -l0 -n prefer:1 COMMAND >>>>=20 >>>> This seem odd to me. Am I missing something? >>>>=20 >>>> For reference: I'm using a ThreadRipper 1950X >>>> with a head -r351227 based context for this >>>> activity. The above happens to have been run >>>> in a Windows 10 Pro HyperV session, instead >>>> of in a native-boot of the same media. (A >>>> native-boot would have had 32 CPUs.) >>>=20 >>> Can you please show the output of "sysctl vm.phys_segs" from this >>> setup? >>=20 >> Sure: >=20 > I was wondering if you had only one domain populated, but it seems not > to be the case. Could you try updating to r351672 or later and see if > the behaviour persists? It may be a bit before I do that. FYI: I had set MAXMEMDOM to match the number of actual domains for the context: /usr/src/sys/amd64/conf/GENERIC-DBG:options MAXMEMDOM=3D2 /usr/src/sys/amd64/conf/GENERIC-NODBG:options MAXMEMDOM=3D2 (These kernel configuration files include GENERIC.) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-current@freebsd.org Wed Sep 11 18:14:51 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 35ED5DB6F4 for ; Wed, 11 Sep 2019 18:14:51 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-9.consmr.mail.ne1.yahoo.com (sonic308-9.consmr.mail.ne1.yahoo.com [66.163.187.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46T96L2mNlz4FYW for ; Wed, 11 Sep 2019 18:14:50 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: VDTiFN8VM1nza.XakPnpb2SIZP1vQILOgi6Zfj06XgE3jc0b3rDDr7j5kQoAUKc M46R6UTTtJwEix_vrou9byobso_tVYKtHDF3xUhIF52XIbhp4nPk_rVG9iUwLqlDGF3rlWYCnjPQ khrD7cK1J2_7goy8ZpOh9iJ_4kRFXg6HM187qyAIPE.76LYlO64MqdBvNVKCOpkplS_ypwr13ylN J34UrGzqps1ZPGsxgeR3JeRgZ.VIaNG5ZI_u6mU1WPaDoqMnlCY4a0Qkd5k7ma9zHd3Sdhk9Kdwm KCZQqgk77HwgYesxxynpM3EJsnhRlDxEGXEX.WKreQIgidvs8jNkG.87DiS9iQKuyTvU7yW9VAo1 fAVYcQOCyTgNKg5f1EIbnSJsTPUwHu3j8_gXkuWmpBrJ6MI69DBuREQE.NUgRO2GEQ5h0UwY9LvM NZRCRrrZVsn0n.vdqMZM_ANnLPcxXpYSJ5J0KOIg1VbMUf9kGYIn9IcHm33C4eg8NmjgIY0sFub1 gHjn_7rbWBWAWdIrmf0Of6qjFifrbZziFSSvKKpP5b8Stc8sjhmHkV.FgcBRCLcX9hSgq0UD_tLh uoSLETf8BIEUF8.yBPu2MSKrhPDpcQ1DsrQaWACwu9RS60eyPCieCsxwA.OOMhnpvqS4SX.BgMF6 lX9XgmOCWXmdy88G7k.QMUkf0aX5xex_bWvAHarSmQHnpU6MddemBPxMhQTyGWBTaNaNkDuZgJti _qr5l3Lvz66TffilCVwaKdVNg5t2ebJMHYAY8bQScUXHx5yHIYy.6YxFI1tgqkEKyqVK2nLtYK1g 25eqHLbNjUtWf2EKqjDClU2SgwsnWZe3G8IhqaLKZNa7B3GQNDyptU7m0cxcqWeQ5nEuV4cNyfzA lHD4UyEdUdlxipqyMjGrGnzZ1ifdnKP.PE22.pIajrKV1haexoJiOg2HISorEF3w9ayyfibSqiaO YGWXSBMlsa49rQghBisTlS3eIYOApqekv4GJYzCwcjgGY77ZDwNFea93JqWMkykVMApjOkepiXgp teKQXba7SbzZzVvGX2BpKBcxubqrGjc6TMBm7RLoX8NoCGBZ1o.G_6YauJq7ukk_FKubJZDosqFQ LOUAZsmszWOls0Y7.lTSwuD5ibZ8kjXLMXHfwAJSRjLu0oNqxTgo3uEEpuBYRLaXv7eIoVMSIDsY N.CIfLVO8fxVnn6tSLiGZsRz08KQgTgddplwhgrcAfsMFJUQZ3WIT5IefZuO6YeieIB8wpxf1J35 uVj6tECKMmDYd0Cr7qOBSVHOQ1RUUo8PBMf4D1ZNFyL.D3Ff8BjMzuq1a0fZAIGcyYE6R Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.ne1.yahoo.com with HTTP; Wed, 11 Sep 2019 18:14:48 +0000 Received: by smtp430.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID df8eb85ede1bb96da59f2499f775d362; Wed, 11 Sep 2019 18:14:44 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: "cpuset -n prefer:?" --what values for "?" are supposed to be allowed? (only 1 is, despite two numa domains) From: Mark Millard In-Reply-To: <3CE4AEB7-E32C-49BD-8C75-71AB8739BAEC@yahoo.com> Date: Wed, 11 Sep 2019 11:14:42 -0700 Cc: FreeBSD Current Content-Transfer-Encoding: quoted-printable Message-Id: <54F53CA8-6BEC-4B31-9662-C6854CDE0A08@yahoo.com> References: <20190911143125.GA17992@raichu> <99BB5653-1F42-4309-9892-24029FD02E39@yahoo.com> <20190911151512.GB17992@raichu> <3CE4AEB7-E32C-49BD-8C75-71AB8739BAEC@yahoo.com> To: Mark Johnston X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 46T96L2mNlz4FYW X-Spamd-Bar: - X-Spamd-Result: default: False [-1.64 / 15.00]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (4.65), ipnet: 66.163.184.0/21(1.30), asn: 36646(1.04), country: US(-0.05)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.69)[-0.687,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; NEURAL_HAM_LONG(-0.46)[-0.458,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[32.187.163.66.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Sep 2019 18:14:51 -0000 On 2019-Sep-11, at 10:11, Mark Millard wrote: > On 2019-Sep-11, at 08:15, Mark Johnston wrote: >=20 >> On Wed, Sep 11, 2019 at 07:57:26AM -0700, Mark Millard wrote: >>>=20 >>>=20 >>> On 2019-Sep-11, at 07:31, Mark Johnston = wrote: >>>=20 >>>> On Tue, Sep 10, 2019 at 10:58:05PM -0700, Mark Millard wrote: >>>>> In a context with: >>>>>=20 >>>>> # cpuset -g >>>>> pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, = 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27 >>>>> pid -1 domain policy: first-touch mask: 0, 1 >>>>>=20 >>>>> I get: >>>>>=20 >>>>> # cpuset -l0 -n prefer:0 COMMAND >>>>> cpuset: setdomain: Invalid argument >>>>>=20 >>>>> # cpuset -l0 -n prefer:2 COMMAND >>>>> cpuset: setdomain: Invalid argument >>>>>=20 >>>>> But one prefer:? value does allow the COMMAND >>>>> to run: >>>>>=20 >>>>> # cpuset -l0 -n prefer:1 COMMAND >>>>>=20 >>>>> This seem odd to me. Am I missing something? >>>>>=20 >>>>> For reference: I'm using a ThreadRipper 1950X >>>>> with a head -r351227 based context for this >>>>> activity. The above happens to have been run >>>>> in a Windows 10 Pro HyperV session, instead >>>>> of in a native-boot of the same media. (A >>>>> native-boot would have had 32 CPUs.) >>>>=20 >>>> Can you please show the output of "sysctl vm.phys_segs" from this >>>> setup? >>>=20 >>> Sure: >>=20 >> I was wondering if you had only one domain populated, but it seems = not >> to be the case. Could you try updating to r351672 or later and see = if >> the behaviour persists? >=20 > It may be a bit before I do that. >=20 > FYI: I had set MAXMEMDOM to match the number of > actual domains for the context: >=20 > /usr/src/sys/amd64/conf/GENERIC-DBG:options MAXMEMDOM=3D2 > /usr/src/sys/amd64/conf/GENERIC-NODBG:options MAXMEMDOM=3D2 >=20 > (These kernel configuration files include GENERIC.) Not that the below is the problem that I reported, but cpuset_modify_domain has an oddity. In the below, note the "root->" use followed by the "root &&" test: the root-> use would have failed first. Should the && be "dset &&" instead? Should "root &&" just be removed for being redundant? 793 root =3D cpuset_getroot(set); 794 mtx_lock_spin(&cpuset_lock); 795 dset =3D root->cs_domain; 796 /* 797 * Verify that we have access to this set of = domains. 798 */ 799 if (root && !domainset_valid(dset, domain)) { 800 error =3D EINVAL; 801 goto out; 802 } =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-current@freebsd.org Wed Sep 11 18:26:46 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 22EA8DC08F; Wed, 11 Sep 2019 18:26:46 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46T9N607WRz4GV2; Wed, 11 Sep 2019 18:26:46 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: by freefall.freebsd.org (Postfix, from userid 1129) id D7B3712E65; Wed, 11 Sep 2019 18:26:45 +0000 (UTC) Date: Wed, 11 Sep 2019 18:26:45 +0000 From: Li-Wen Hsu To: freebsd-testing@freebsd.org Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: FreeBSD CI Weekly Report 2019-09-08 Message-ID: <20190911182645.GA64832@freefall.freebsd.org> Reply-To: freebsd-testing@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.11.4 (2019-03-13) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Sep 2019 18:26:46 -0000 (Please send the followup to freebsd-testing@ and note Reply-To is set.) FreeBSD CI Weekly Report 2019-09-08 =================================== Here is a summary of the FreeBSD Continuous Integration results for the period from 2019-09-02 to 2019-09-08. During this period, we have: * 2122 builds (99.4% (+2) passed, 0.6% (-2) failed) were executed on aarch64, amd64, armv6, armv7, i386, mips, mips64, powerpc, powerpc64, powerpcspe, riscv64, sparc64 architectures for head, stable/12, stable/11 branches. * 373 test runs (59.6% (-0.2) passed, 39.1% (-1.1) unstable, 1.3% (+1.3) exception) were executed on amd64, i386, riscv64 architectures for head, stable/12, stable/11 branches. * 50 doc builds (100% (+2) passed) Test case status (on 2019-09-08 23:59): | Branch/Architecture | Total | Pass | Fail | Skipped | | ------------------- | ---------- | ---------- | --------- | -------- | | head/amd64 | 7556 (+6) | 7495 (+11) | 0 (-1) | 61 (-4) | | head/i386 | 7554 (+6) | 7484 (+7) | 2 (0) | 68 (-1) | | 12-STABLE/amd64 | 7393 (+1) | 7341 (-7) | 11 (+11) | 41 (-3) | | 12-STABLE/i386 | 7391 (+1) | 7329 (-10) | 11 (+11) | 51 (0) | | 11-STABLE/amd64 | 6846 (+1) | 6802 (+4) | 0 (0) | 44 (-3) | | 11-STABLE/i386 | 6844 (+1) | 6767 (-7) | 34 (0) | 43 (-6) | (The statistics from experimental jobs are omitted) If any of the issues found by CI are in your area of interest or expertise please investigate the PRs listed below. The latest web version of this report is available at https://hackmd.io/@FreeBSD-CI/report-20190908 and archive is available at https://hackmd.io/@FreeBSD-CI/, any help is welcome. ## News * Weekly CI report archive has been moved to https://hackmd.io/@FreeBSD-CI * [FCP 20190401-ci_policy: CI policy](https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md) is in "feedback" state, please check and provide comments on -fcp@ and -hackers@ mailing lists. ## Fixed Tests * https://ci.freebsd.org/job/FreeBSD-head-amd64-test/ * (flakey) usr.bin.procstat.procstat_test.environment * (flakey) usr.bin.procstat.procstat_test.command_line_arguments Fixed in https://svnweb.freebsd.org/changeset/base/351819 ## Failing Tests * https://ci.freebsd.org/job/FreeBSD-stable-12-amd64-test/ * https://ci.freebsd.org/job/FreeBSD-stable-12-i386-test/ * sys.netipsec.tunnel.aes_cbc_128_hmac_sha1.v6 * sys.netipsec.tunnel.aes_cbc_256_hmac_sha2_256.v6 * sys.netipsec.tunnel.aes_gcm_128.v6 * sys.netipsec.tunnel.aes_gcm_256.v6 * sys.netipsec.tunnel.aesni_aes_cbc_128_hmac_sha1.v6 * sys.netipsec.tunnel.aesni_aes_cbc_256_hmac_sha2_256.v6 * sys.netipsec.tunnel.aesni_aes_gcm_128.v6 * sys.netipsec.tunnel.aesni_aes_gcm_256.v6 * sys.netipsec.tunnel.empty.v6 * sys.netpfil.pf.fragmentation.v6 * sys.netpfil.pf.pass_block.v6 * https://bugs.freebsd.org/240511 * https://ci.freebsd.org/job/FreeBSD-stable-11-i386-test/ * local.kyua.* (31 cases) * local.lutok.* (3 cases) ## Failing and Flaky Tests (from experimental jobs) * https://ci.freebsd.org/job/FreeBSD-head-amd64-dtrace_test/ * cddl.usr.sbin.dtrace.common.misc.t_dtrace_contrib.tst_dynopt_d * https://bugs.freebsd.org/237641 * cddl.usr.sbin.dtrace.amd64.arrays.t_dtrace_contrib.tst_uregsarray_d * https://bugs.freebsd.org/240358 * https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/ * There are ~60 failing cases, including flakey ones, see https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/lastCompletedBuild/testReport/ for more details ## Disabled Tests * lib.libc.sys.mmap_test.mmap_truncate_signal https://bugs.freebsd.org/211924 Patch available: https://reviews.freebsd.org/D21566 * sys.fs.tmpfs.mount_test.large https://bugs.freebsd.org/212862 * sys.fs.tmpfs.link_test.kqueue https://bugs.freebsd.org/213662 * sys.kqueue.libkqueue.kqueue_test.main https://bugs.freebsd.org/233586 * sys.kern.ptrace_test.ptrace__PT_KILL_competing_stop https://bugs.freebsd.org/220841 * lib.libc.regex.exhaust_test.regcomp_too_big (i386 only) https://bugs.freebsd.org/237450 * sys.netinet.socket_afinet.socket_afinet_bind_zero (new) https://bugs.freebsd.org/238781 * sys.netpfil.pf.names.names * sys.netpfil.pf.synproxy.synproxy https://bugs.freebsd.org/238870 * sys.kern.ptrace_test.ptrace__follow_fork_child_detached_unrelated_debugger https://bugs.freebsd.org/239292 * sys.netpfil.pf.forward.v4 (i386 only) * sys.netpfil.pf.forward.v6 (i386 only) * sys.netpfil.pf.set_tos.v4 (i386 only) https://bugs.freebsd.org/239380 (updating net/scapy to 2.4.3 may fix this) * sys.kern.ptrace_test.ptrace__follow_fork_both_attached_unrelated_debugger https://bugs.freebsd.org/239397 * sys.kern.ptrace_test.ptrace__parent_sees_exit_after_child_debugger https://bugs.freebsd.org/239399 * sys.kern.ptrace_test.ptrace__follow_fork_parent_detached_unrelated_debugger https://bugs.freebsd.org/239425 * lib.libc.gen.getmntinfo_test.getmntinfo_test https://bugs.freebsd.org/240049 * sys.sys.qmath_test.qdivq_s64q https://bugs.freebsd.org/240219 ## Issues ### Cause build fails * https://bugs.freebsd.org/233735 Possible build race: genoffset.o /usr/src/sys/sys/types.h: error: machine/endian.h: No such file or directory * https://bugs.freebsd.org/233769 Possible build race: ld: error: unable to find library -lgcc_s ### Cause kernel panics * https://bugs.freebsd.org/238870 sys.netpfil.pf.names.names and sys.netpfil.pf.synproxy.synproxy cause panic Patch exists: * https://reviews.freebsd.org/D20868 * https://reviews.freebsd.org/D20869 ### Open * https://bugs.freebsd.org/237403 Tests in sys/opencrypto should be converted to Python3 * https://bugs.freebsd.org/237641 Flakey test case: common.misc.t_dtrace_contrib.tst_dynopt_d * https://bugs.freebsd.org/237656 "Freed UMA keg (rtentry) was not empty (18 items). Lost 1 pages of memory." seen when running sys/netipsec tests * https://bugs.freebsd.org/238781 sys.netinet.socket_afinet.socket_afinet_bind_zero does not work when mac_portacl(4) loaded * https://bugs.freebsd.org/239292 Flakey test case: sys.kern.ptrace_test.ptrace__follow_fork_child_detached_unrelated_debugger * https://bugs.freebsd.org/239380 sys.netpfil.pf.forward.{v4,v6} and sys.netpfil.pf.set_tos.v4 fail on i386 * https://bugs.freebsd.org/239397 Flakey test case: sys.kern.ptrace_test.ptrace__follow_fork_both_attached_unrelated_debugger * https://bugs.freebsd.org/239399 Flakey test case: sys.kern.ptrace_test.ptrace__parent_sees_exit_after_child_debugger * https://bugs.freebsd.org/239425 Flakey test case: sys.kern.ptrace_test.ptrace__follow_fork_parent_detached_unrelated_debugger * https://bugs.freebsd.org/240085 Failing test: sys.netpfil.common.forward.pf_v4 on i386 * https://bugs.freebsd.org/240086 Failing test: sys.netpfil.common.tos.pf_tos on i386 ### Others * [Tickets related to testing@](https://preview.tinyurl.com/y9maauwg) From owner-freebsd-current@freebsd.org Thu Sep 12 04:40:14 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 17787EA8E5 for ; Thu, 12 Sep 2019 04:40:14 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46TQzw4mfcz3Jyx for ; Thu, 12 Sep 2019 04:40:12 +0000 (UTC) (envelope-from ohartmann@walstatt.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1568263201; bh=EX4hJECsDx6GLN9rGvl97z+dh9WIHCf4bA1SYWHtrio=; h=X-UI-Sender-Class:Date:From:To:Subject; b=U/ofKeyKTbc++9CYT4RZ0KWW9iXb/zqYHT7RykFrbvaG/U8MBXbX989oZYv2PgmUD vcntQBDqqkHIRZcMAoBWOO7vOR9ZPpx/C1YHX+4M2H4teHPR9P3ALBrFY6rbLaprwW OZPrNau4lVeeSmlL3f5zARF7JgIRVTOAYk3Zpw1Q= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from freyja ([46.88.81.15]) by mail.gmx.com (mrgmx002 [212.227.17.190]) with ESMTPSA (Nemesis) id 0MPlsg-1i4RCr20o6-0055GI for ; Thu, 12 Sep 2019 06:27:07 +0200 Date: Thu, 12 Sep 2019 06:27:00 +0200 From: "O. Hartmann" To: freebsd-current Subject: r352239: install failure: make[10]: exec(btxld) failed (No such file or directory) Message-ID: <20190912062656.7afa3816@freyja> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K1:vupM7kRy1J4A8HBijF5v+7Tp2BMiHLbPNDCYcpbB4f5DANRKr5B CbERc9Fnr7QFJmuSKneKHIYMUKa+Wb5L+8bf/YveN+n9QXgFe88fY2YS7YQ5cHfv83H66ZY hsqErgeifJFh/b/6I299hBl3M3OdP3WNzsrOSWDncnscFjGZqDJuoXg5hpWLAYzDMRuE+XM dihAIjHSfI4PQTA+0kK/Q== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:V/Xs9PJPaMA=:ijeGU6bsFiE2mSZH2bah77 YNLYX9RbPUEZSNRY+M4MmNMDaUAcoqrZnRsVINwJHPWjF42kjlD67ZF/1GyN3C8eqtZmH+BlW p9aQTlhUmuIEX5dotZFkwnWCYsv/JxPqPCt6VbaoueSdibSJ7JgAov85blgGBox3n9atbgklB fSmvPqCbGrhL+cf8FIc0J+TzfOKaeCWZJeu7pQBWl+nUk2Ud6N+oblbNq/Iha/QzePdumFORI vp22lksmsuetYj69Eh5krioxaMKLGhy10kVcj2pzOEgH3tstDUp8sCzYPNfI0Yp7YPshCFc4c UJ/7oAbaKdPmE6IpsyKcXxxb75QjZyyvJUdjdN5BTihlgTw4zPUcVpVVjj9DKu6RygtnKUn6p tHH8zXQd4IySDpdSYZxkWOUvlnPwM8dVoW4p+bzio3VZVhChIJsMo5y+kBTBBSdnKjxtSoWz9 f5vhELb0a6mPMYDwBO2tSG3hxOdQ04pYpiMmRFceqlKG2zQ5hY6hhyuO/WLeCSolKfJhO9nrq c6eCia4AMzbKxuoL2ZdAZBkuWBrp+j6oqQwmd55hcFLwKF/8XKxtNNuWYCRBxmUpxrA6pjS1w Gw4wEKsidOICBArV2OOXNPtXTi7ax+q0jqQlcr9R5jn8eb1ee5oz1ZM9LKcTX/8LrEqd18CtI CX3Btw+gBF3nC+6PG9W0fEOphzqE7U5a25SUvcEYc6F2oNpXJqfHS71ojMsxj3DJFZ8xzKFRg AablgItid84dHZBQzPaYJKz2pWkP5wZl9G3G5g9a7I+WTf4ZHraXC6GEBMGC4T+syFQahaSY3 15hS7WvrD5uD5MWsk2PUwDYc47yIv+12fsr/Rvjkjj/o5wHG6TXnFvdus4bDrT3IAehipAqGx fpRVcOacG7/Q6bo/XDeZEl5hGPbMVKLKRHX1QYJNAxGzyBfPk1HGQ0IqYYGupBQd2UHU2XQsL 2/gA0niBzZyLXXCPaptZkXrXKpR41b26EHqIqjxrLTAJRD23Z7fxlaXP4AKa+S3PM1eYvsjQA BDNZyvmWyZJSsKh+AicqeMC+V325x/ljcribHlrm7YQ2V3N8AYvFA6ypK3UzxV/Cwey6lYu8E MAPAo5DoUmu4agDz147En3b6t5+Uoy6QKI3AFpQLDnSO0JQ1UFdBCf3xFqrvfrUU88VX+aomM XqItO9aWNFCa8jQ89ftCbtwrXzJ27exf2+Outht+nppqEjGlYNdS4rLr2zvJr2E2D63Sz+gOq lMYoxvlJk7woaMEoD X-Rspamd-Queue-Id: 46TQzw4mfcz3Jyx X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=U/ofKeyK; dmarc=none; spf=none (mx1.freebsd.org: domain of ohartmann@walstatt.org has no SPF policy when checking 212.227.15.19) smtp.mailfrom=ohartmann@walstatt.org X-Spamd-Result: default: False [-2.92 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[walstatt.org]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-1.12)[ip: (-6.38), ipnet: 212.227.0.0/16(-1.37), asn: 8560(2.14), country: DE(-0.01)]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmx.net:+]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[15.81.88.46.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Sep 2019 04:40:14 -0000 Hello, we install several pkg-based systems and poudriere from a dedicated tree of sources, instead of /usr/src it is in our case /pool/sources/CURRENT/src and 12-STABLE/src. Compilation of the sources is done within a JAIL! For a couple of days now, both trees, CURRENT (r352239 now) and 12-STABLE (r352239) fail at the exact same point, when compiling and further packaging: [...] install -U -M /pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/worldstage//METALOG -D /pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/worldstage -T package=utilities -d -m 0755 -o root -g wheel /pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/worldstage/boot objcopy -S -O binary boot2.out boot2.bin btxld -v -E 0x2000 -f bin -b /pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/stand/i386/btx/btx/btx -l boot2.ldr -o boot2.ld -P 1 boot2.bin make[10]: exec(btxld) failed (No such file or directory) *** Error code 1 [...] For reduction of the installed binaries and stuff, we use customized src.conf and each build process is delegated to its appropriate src.conf by setting the variabel SRCCONF accordingly; poudriere also uses the same src.conf by linking the jailname-src.conf file into poudriere's config folder; the content of src.conf is as follows: [...] WITH_OFED= YES #WITH_CTF= YES # #WITH_BEARSSL= YES # WITH_SVN= YES # WITH_SORT_THREADS= YES # MALLOC_PRODUCTION= YES # #WITHOUT_ASSERT_DEBUG= YES #WITHOUT_DEBUG_FILES= YES #WITHOUT_TESTS= YES WITHOUT_PROFILE= YES # WITHOUT_REPRODUCIBLE_BUILD= YES # # mitigation for CVE-2017-5715 in the kernel build WITH_RETPOLINE= YES [...] Building poudriere jails from such sources also fails since a couple of days on all platforms with a weird message thata folder and/or file atf-check is missing (this happens when using command sequence: poudriere jail -j jailname -u -b and the install method of the appropriate jail is src=/path/to/source/src. Thanks for helping, oh From owner-freebsd-current@freebsd.org Thu Sep 12 11:38:57 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 05354F33A0 for ; Thu, 12 Sep 2019 11:38:57 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46TcH36v7xz48fD for ; Thu, 12 Sep 2019 11:38:55 +0000 (UTC) (envelope-from ohartmann@walstatt.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1568288325; bh=p/XzAgmdRPgfsJu25R2INsLgZFQxapYmGKrzeDs8udI=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:In-Reply-To:References; b=eAFJJC5BuyF+as/npP4hiN70s8Fo0X9wiqCBX3KXQHQNXhD24rJVUk/mzKv7Izb3G riyn4rtboemOYUa11+1zrvHkBYH5CF2i2tA8gLi3rSeVl2dfNt2lbWcaSMTmDLFP9N sgAx+VzYBETD/HKSu9UjEtyPYMsrL7MoQSO33HuQ= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from freyja ([46.88.81.15]) by mail.gmx.com (mrgmx103 [212.227.17.168]) with ESMTPSA (Nemesis) id 0Le69A-1iTOBo1tj2-00pvsW; Thu, 12 Sep 2019 13:26:00 +0200 Date: Thu, 12 Sep 2019 13:25:55 +0200 From: "O. Hartmann" To: "O. Hartmann" Cc: freebsd-current Subject: Re: r352239: install failure: make[10]: exec(btxld) failed (No such file or directory) Message-ID: <20190912132555.7126215c@freyja> In-Reply-To: <20190912062656.7afa3816@freyja> References: <20190912062656.7afa3816@freyja> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K1:Y+3eYBUoevXwFVdbbV0+3vdPpXj/VWbN37WSo/vKtfeyUd/ZkPt TJRmNHlNVonVd3JqC2c1w/RfhgQZ/hkxZECkQb+t6kn4yZqha8BFAEo7OHxIvZ0P6b2wdxs 6xp71iok1qMplFlCJHKrrWBdEnxcfRhfDTORReLKhyoSZuxO8z0l12o7j7sKuPc0OhNgplz oLvCARQGqY64mbUNZtJWQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:UUKVZRs6HSA=:85AQ0mcCwJALB5gEpZUZ4L uYsESGSIX4RszAIOp8SZR+nu4RbYfJfBpkyVrG9lyn3p0MsItQxtfJGdf5e+OL7RkB1kbAUzn oFJkZUJD0AjgXFea/OAsX82pfcCYrb13F/b4HCjhHnhlkLOZDTH1BPo3nqL+sYyD3KZUH1S9A VCP1gTt1tysdGVgQHx2ijvDxAMghc9bNUK8IHHq+nAqAsiZQo8qi32elLXxz62H8FEC5wL2lU 2urZOBK6IkiwqfxYiTJ1dmCUvVeqnjkd99Iv9kw9Sc883oN98Hy9+rn3y7wWCjjoQz0yMbMJs 1yBM6LgRd/4DgcErHZH5IsoztPkkraxiuwFZ13AS1u9QAt2samf9pE0b3VN+zlNPl/OuC5pRd Sy6ajjZT9Nk+4PcPvCTsC79voYCaagRTW3JcGvNnpBGRwYhrAyx/u4/6rkY5ueyCj6tToOLMP 2bxGsM2yWT3Y7XoD4Nb+8IGqp0BYCdh+kUZQajJL3Dbju4SLhSoaqUqtlehWaerRb5CRRpjVb IpQtzY8lG8xv7/uYjt2XdtWJNNDCy/15if+79lFj51mLzfDXnYgjh4dhyw+Vur07slabdHnIA akumSvkDsdVO4JB0xxDZssH9j9fqKmQZrEgKqY75jO7MgVCMnQ3oyYCG5UB7daVNGFBZgeCdY GX+6Q3hvioFFI0pbxWtgUR8ZOHF5x3oB73ZDQzT0azarIDrYtGTtQROEZfhgYTjdYEOvSI7bq enneJvMwmEGtmRYv0mW4tbo6NxScIRkvcFo5sZoq2+UC9WMKEYurRnOtGKDdUMgbmj2OY3AEH kntEKSPVMoGHxIDKwwBU9+Icy0V+bfq7F9hPtJidBeiwUQNUFReOIyzJHLR+fh133/RlyQ5UN vFXs9QBhkvyYnDMHIxYT0p8ZeMdarfDJKrABBoef7fZbAKqClS6A3W2+5xz7ZvWQvbltopXx0 R1OdxmNgg3rkJ9KKUttHUfMnNWo2haUy/N2DmhGMV8olT1/pSouZSuNGNa2RMT/Ua0i46CRW5 nkU/Hn64syN5y5TG/HjMCYJQW8zQNO0yaNH0kyhqucfnYmesnh9AFtgCcP2E0Mevomf9YW1cL DIw+5hsKEMeqLZQxFjm9JeS2KXjx7Iqh3fI5ZDcbi4ykZ8jo8aJEXhcH7W/cwFgJ8pbXlhz1/ M3jTwYzyCoJFnua4nf+8sWInlh8f37a+9W/TZY+7ZRvemqfzteCbAWXnfs+QJVE5MdZUoCQ3n U/NRKsXAH+10b3vUzgw+KceK5JdusdXGSKFe5HCbvjRv04jGCioYYVRW1CrQ= X-Rspamd-Queue-Id: 46TcH36v7xz48fD X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=eAFJJC5B; dmarc=none; spf=none (mx1.freebsd.org: domain of ohartmann@walstatt.org has no SPF policy when checking 212.227.17.21) smtp.mailfrom=ohartmann@walstatt.org X-Spamd-Result: default: False [-3.17 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450]; RECEIVED_SPAMHAUS_PBL(0.00)[15.81.88.46.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; IP_SCORE(-1.27)[ip: (-7.12), ipnet: 212.227.0.0/16(-1.37), asn: 8560(2.16), country: DE(-0.01)]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[walstatt.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmx.net:+]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[21.17.227.212.list.dnswl.org : 127.0.3.1]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Sep 2019 11:38:57 -0000 On Thu, 12 Sep 2019 06:27:00 +0200 "O. Hartmann" wrote: > Hello, > > we install several pkg-based systems and poudriere from a dedicated tree= of > sources, instead of /usr/src it is in our case /pool/sources/CURRENT/src= and > 12-STABLE/src. Compilation of the sources is done within a JAIL! > > For a couple of days now, both trees, CURRENT (r352239 now) and 12-STABL= E > (r352239) fail at the exact same point, when compiling and further packa= ging: > > [...] > install -U -M > /pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/worldstag= e//METALOG > -D /pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/worlds= tage > -T package=3Dutilities -d -m 0755 -o root -g wheel > /pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/worldstag= e/boot > objcopy -S -O binary boot2.out boot2.bin btxld -v -E 0x2000 -f bin -b > /pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/stand/i38= 6/btx/btx/btx > -l boot2.ldr -o boot2.ld -P 1 boot2.bin make[10]: exec(btxld) failed (N= o such > file or directory) *** Error code 1 > [...] > > For reduction of the installed binaries and stuff, we use customized src= .conf > and each build process is delegated to its appropriate src.conf by setti= ng the > variabel SRCCONF accordingly; poudriere also uses the same src.conf by l= inking > the jailname-src.conf file into poudriere's config folder; the content o= f > src.conf is as follows: > > [...] > WITH_OFED=3D YES > #WITH_CTF=3D YES > # > #WITH_BEARSSL=3D YES > # > WITH_SVN=3D YES > # > WITH_SORT_THREADS=3D YES > # > MALLOC_PRODUCTION=3D YES > # > #WITHOUT_ASSERT_DEBUG=3D YES > #WITHOUT_DEBUG_FILES=3D YES > #WITHOUT_TESTS=3D YES > WITHOUT_PROFILE=3D YES > # > WITHOUT_REPRODUCIBLE_BUILD=3D YES > # > # mitigation for CVE-2017-5715 in the kernel build > WITH_RETPOLINE=3D YES > > [...] > > Building poudriere jails from such sources also fails since a couple of = days > on all platforms with a weird message thata folder and/or file atf-check= is > missing (this happens when using command sequence: poudriere jail -j jai= lname > -u -b and the install method of the appropriate jail is > src=3D/path/to/source/src. > > Thanks for helping, > > oh > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.or= g" After today's update of CURRENT's source tree and "poudriere jail -j jailn= ame -u -b", which should result in a successful build, the error is: [...] cc -target x86_64-unknown-freebsd13.0 =2D-sysroot=3D/pool/sources/CURRENT/obj/pool/poudriere/jails/headamd64/usr= /src/amd64.amd64/tmp -B/pool/sources/CURRENT/obj/pool/poudriere/jails/headamd64/usr/src/amd64.a= md64/tmp/usr/bin -O2 -pipe -O3 -DNDEBUG -I. -I/pool/poudriere/jails/headamd64/usr/src/contrib/elftoolchain/libelf -I/pool/poudriere/jails/headamd64/usr/src/contrib/elftoolchain/common -mretpoline -g -MD -MF.depend.elf_update.o -MTelf_update.o -std=3Dgnu99 -Wno-format-zero-length -fstack-protector-strong -Wsystem-headers -Werror = -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-str= ings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winli= ne -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sig= n -Wmissing-variable-declarations -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -c /pool/poudriere/jails/headamd64/usr/src/contrib/elftoolchain/libelf/elf_up= date.c -o elf_update.o /pool/poudriere/jails/headamd64/usr/src/contrib/elftoolchain/libelf/elf_up= date.c:841:67: error: unused parameter 'ex' [-Werror,-Wunused-parameter] _libelf_write_ehdr(Elf *e, unsigned char *nf, struct _Elf_Extent *ex) [...] As I mentioned earlier, the build is performed within a jail considered to= run poudriere and the task has been performed successful earlier (a couple of = days ago, but didn't memorised the revision number). On non-jailed hosts, this task works as expected, both on 12-STABLE (recen= t version 12.1-PRE) and CURRENT. Also did I remove the object's path and sta= rted a fresh build, so remnants from an earlier build can be excluded. /usr/local/etc/poudriere.d/jailname-poudriere.conf has a valid setting lik= e this: export MAKEOBJDIRPREFIX=3D/pool/sources/CURRENT/obj/ Another strategy also fails. Building all the binaries under ./obj from th= e base host with an objetctree of a valid and successful build and then jexe= 'ing into the poudriere jail, which has all the infrastructure on ZFS already mounted and typing there poudriere jail -j jailname -u which should result in a correct and successfuil installation, fails immediately with install -N /pool/sources/CURRENT/src/etc -C -o root -g wheel -m 444 libpythagoras.a /pool/poudriere/jails/headamd64/usr/tests/libexec/rtld-elf= / install -N /pool/sources/CURRENT/src/etc -C -o root -g wheel -m 444 libpythagoras_p.a /pool/poudriere/jails/headamd64/usr/tests/libexec/rtld-e= lf/ =2D-- ld_library_pathfds.install --- --- _proginstall --- --- realinstall_subdir_libexec/rtld-elf/tests/libpythagoras --- install: libpythagoras_p.a: No such file or directory *** [_libinstall] Error code = 71 I feel a bit lost here ... is there something special to jails? Kind regards, oh From owner-freebsd-current@freebsd.org Thu Sep 12 15:54:36 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7889FD24E7 for ; Thu, 12 Sep 2019 15:54:36 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46Tjy34mWXz4QXv for ; Thu, 12 Sep 2019 15:54:35 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-io1-xd2f.google.com with SMTP id j4so55895793iog.11 for ; Thu, 12 Sep 2019 08:54:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=UjayjPjs2x6sLHh4cI8QCU2bPWZJMuQjTyXQA7vJvW8=; b=p954PMDk1ETBqhRprS9GP6j9fZ3GlIE2aMR+0RmphN/q4wz6+diJXJW9ps9EYZyjm2 OGGSXzXILzf32YUce4CF6ZXA+QsJeLLQHRUWq5kDpeSvPvTG1OAvrwd6OCsys19CWX9V WZRsWVIaNJVrUTCvxQ0vJmYWv9iaJD2oBPopa3kN3sRt7hfORkxdYce3d6IWO+L6sYhM LlC2eQF77M1IKut9ES5p2l11jdFb8rLqUkuQZapWxLB9u4RmUoQrq5KWdHYcf9PrKvrP RcBaGoZXVQKezfsdJlWNIoGQPRFJlVRpj/vHwycO99oa6QSJdTZnzHr1NyvzPJ3W8tD+ Y0XA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=UjayjPjs2x6sLHh4cI8QCU2bPWZJMuQjTyXQA7vJvW8=; b=FqpXQneCo1GqVf+SxkW4xIwica7uxVYkKC5bXThz5Ovd2Z/En+Rht6b4aacbBAmJ/H 9QD5Dz5HSnALjnAUM+TRyDgly390wfzy4U0Otq6xgbffLAe114Zci7kh+NpOO8ZlD89Q Mbmdnz1Un8KDlh9iLnmkHZZ2ljjIXd6gOEhFjgBu9JPzjZ9PDucrH0I/Wv9nkGGzPZUV Ux6LcWPFhaD9ei2O3U2x+eegH/CToADOU0nIYw5HmsebSvy9TGU0ln9EHL3ScxpGwJup LGbJ2S1FO7HIXHnTTkqBEUEsfY3+SD/sc5b07km8bXtyNl7u/zBZ+XeKEouiEHMU/0Hc y36A== X-Gm-Message-State: APjAAAVebRfydLbTpPBIe2ZdKdndaradc0Zpl/gM7aTcKpehwX7EHe/3 ARVqQ+t1wcMZ3/pT/Acbp9U5yngo X-Google-Smtp-Source: APXvYqycYrXQtuPCB45KUaw2Dp5P7+tOMgzcSndDr4+D+J+RmJ38ql8I2YN7zKo1NZ6mkfTsyNTGIA== X-Received: by 2002:a05:6602:2256:: with SMTP id o22mr2311350ioo.104.1568303674164; Thu, 12 Sep 2019 08:54:34 -0700 (PDT) Received: from raichu (toroon0560w-lp140-01-69-159-39-167.dsl.bell.ca. [69.159.39.167]) by smtp.gmail.com with ESMTPSA id y19sm23423476ioq.69.2019.09.12.08.54.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Sep 2019 08:54:33 -0700 (PDT) Sender: Mark Johnston Date: Thu, 12 Sep 2019 11:54:28 -0400 From: Mark Johnston To: Mark Millard Cc: FreeBSD Current Subject: Re: "cpuset -n prefer:?" --what values for "?" are supposed to be allowed? (only 1 is, despite two numa domains) Message-ID: <20190912155428.GA8397@raichu> References: <20190911143125.GA17992@raichu> <99BB5653-1F42-4309-9892-24029FD02E39@yahoo.com> <20190911151512.GB17992@raichu> <3CE4AEB7-E32C-49BD-8C75-71AB8739BAEC@yahoo.com> <54F53CA8-6BEC-4B31-9662-C6854CDE0A08@yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54F53CA8-6BEC-4B31-9662-C6854CDE0A08@yahoo.com> User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 46Tjy34mWXz4QXv X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=p954PMDk; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::d2f as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-3.92 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; FREEMAIL_TO(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-2.22)[ip: (-6.09), ipnet: 2607:f8b0::/32(-2.71), asn: 15169(-2.25), country: US(-0.05)]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; SUBJECT_HAS_QUESTION(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[f.2.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Sep 2019 15:54:36 -0000 On Wed, Sep 11, 2019 at 11:14:42AM -0700, Mark Millard wrote: > > > On 2019-Sep-11, at 10:11, Mark Millard wrote: > > > > > On 2019-Sep-11, at 08:15, Mark Johnston wrote: > > > >> On Wed, Sep 11, 2019 at 07:57:26AM -0700, Mark Millard wrote: > >>> > >>> > >>> On 2019-Sep-11, at 07:31, Mark Johnston wrote: > >>> > >>>> On Tue, Sep 10, 2019 at 10:58:05PM -0700, Mark Millard wrote: > >>>>> In a context with: > >>>>> > >>>>> # cpuset -g > >>>>> pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27 > >>>>> pid -1 domain policy: first-touch mask: 0, 1 > >>>>> > >>>>> I get: > >>>>> > >>>>> # cpuset -l0 -n prefer:0 COMMAND > >>>>> cpuset: setdomain: Invalid argument > >>>>> > >>>>> # cpuset -l0 -n prefer:2 COMMAND > >>>>> cpuset: setdomain: Invalid argument > >>>>> > >>>>> But one prefer:? value does allow the COMMAND > >>>>> to run: > >>>>> > >>>>> # cpuset -l0 -n prefer:1 COMMAND > >>>>> > >>>>> This seem odd to me. Am I missing something? > >>>>> > >>>>> For reference: I'm using a ThreadRipper 1950X > >>>>> with a head -r351227 based context for this > >>>>> activity. The above happens to have been run > >>>>> in a Windows 10 Pro HyperV session, instead > >>>>> of in a native-boot of the same media. (A > >>>>> native-boot would have had 32 CPUs.) > >>>> > >>>> Can you please show the output of "sysctl vm.phys_segs" from this > >>>> setup? > >>> > >>> Sure: > >> > >> I was wondering if you had only one domain populated, but it seems not > >> to be the case. Could you try updating to r351672 or later and see if > >> the behaviour persists? > > > > It may be a bit before I do that. > > > > FYI: I had set MAXMEMDOM to match the number of > > actual domains for the context: > > > > /usr/src/sys/amd64/conf/GENERIC-DBG:options MAXMEMDOM=2 > > /usr/src/sys/amd64/conf/GENERIC-NODBG:options MAXMEMDOM=2 > > > > (These kernel configuration files include GENERIC.) Ok, that helps. I believe you are hitting a bug that will be fixed by r351672 and a couple of preceding commits to the same area. > Not that the below is the problem that I reported, but > cpuset_modify_domain has an oddity. In the below, note > the "root->" use followed by the "root &&" test: the > root-> use would have failed first. Should the && be > "dset &&" instead? Should "root &&" just be removed for > being redundant? Good catch. I believe cpusets are not allowed to have a NULL domain set, so dset should never be NULL either. From owner-freebsd-current@freebsd.org Thu Sep 12 19:37:38 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 684A0D83AC for ; Thu, 12 Sep 2019 19:37:38 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com [IPv6:2607:f8b0:4864:20::72c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46TpvP53wmz4fn1 for ; Thu, 12 Sep 2019 19:37:37 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-qk1-x72c.google.com with SMTP id u184so22809013qkd.4 for ; Thu, 12 Sep 2019 12:37:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=nBW+ufIGymAIdGqAv2jrxFR4u0H5dz8jvADK0YDjK2U=; b=rDmmqF565oSelmPNFMcZkXrbiW/t4zqBl0nessFDCL0O+XdX5srFWhf4niALdVct5b A8KtWouJUKDvT03CtpXMO937R2e0W6BespNvCR3ClPHJuv8RLlALGqQM8B5iBDRWXkvb IOhbZx05hSvgRknsH4ohjjq8mVEbLMORl4w16DklPTLo9BX8SiIglZjMWKmVrMKMeG9d KoV+xYIcjnhwsrM08jFy0VTIZUJk7FskTmh4wDIP8YUdf2bSk0O/EcVglThWedco01ej RRiVsa5HcDPyzivhHzGWSyTYZUHGHkNuPF3XFRQx54pGD6Tf+f8fVFruNL+bwhDVZhNn JdYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=nBW+ufIGymAIdGqAv2jrxFR4u0H5dz8jvADK0YDjK2U=; b=JnGd+hlkgiI7bL3huL9CXnY+B/liSLR5fKszB4InaMiERu6HmffKgsm+uOuHND67/A 8KBw/mpxBT+eKoHL/W1ep+e8wevv8IvBHwOk8bpBZraWmWTVfWRT9ZJCtbagsQ8xwelX 0mOVFItM4iD2SkDwYgqNYx/sRL+Ckfod+lBlvQcZTg8saqwADAFK9Wf90CLPEYRc9+px 81lAShDGrhV7IhE93ZhTEK7DznkHuP2IX7pWrtZxQsIJMeGn/X0fi004onY7VP+n2dFD AhlL5eHKwfFHFXk6vmo2niq5+jkB94qrZoGUYVYrmLpWLwBpaHW3qXV0KOX/DfbjDPnq nt4Q== X-Gm-Message-State: APjAAAXWQw2o+K656+VBM0V8/T402p91Jv+pwfrEsFUCRoi0NRihXgrx jEZevdarJosOqIgPwhsl0BQHPNY8uPIWiet299FcL4UG X-Google-Smtp-Source: APXvYqws6+yHBERuAnLhKZmFRpeJVJuvzgVpQQZ1p2KRyLfUhFG/ojLwMvujhykXM9D78IGYIsPOLLj2bpN1kEMnQ8k= X-Received: by 2002:a37:b045:: with SMTP id z66mr19402085qke.428.1568317056311; Thu, 12 Sep 2019 12:37:36 -0700 (PDT) MIME-Version: 1.0 From: Ryan Stone Date: Thu, 12 Sep 2019 15:37:25 -0400 Message-ID: Subject: Deadlock involving truss -f, pdfork() and wait4() To: FreeBSD Current Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46TpvP53wmz4fn1 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=rDmmqF56; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rysto32@gmail.com designates 2607:f8b0:4864:20::72c as permitted sender) smtp.mailfrom=rysto32@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(0.00)[ip: (-9.32), ipnet: 2607:f8b0::/32(-2.71), asn: 15169(-2.25), country: US(-0.05)]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[c.2.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Sep 2019 19:37:38 -0000 I've hit an issue with a simple use of pdfork(). I have a process that calls pdfork() and the parent immediately does a wait4() on the child pid. This works fine under normal conditions, but if the parent is run under truss -f, the three processes deadlock. If I switch out pdfork() for fork(), the deadlock does not occur. This C file demonstrates the issue: https://people.freebsd.org/~rstone/pdfork.c If I run "truss -f ./pdfork", which uses fork(), it completes within a second. If I run "truss -f ./pdfork -p", which uses pdfork(), the processes deadlock. If I run "./pdfork -p" without truss, it completes normally. procstat reports the following kernel stacks: 27572 102043 truss - mi_switch+0xe2 sleepq_catch_signals+0x425 sleepq_wait_sig+0xf _sleep+0x1bf kern_wait6+0x695 sys_wait6+0x9f amd64_syscall+0x36e fast_syscall_common+0x101 27573 102469 pdfork - mi_switch+0xe2 sleepq_catch_signals+0x425 sleepq_wait_sig+0xf _sleep+0x1bf kern_wait6+0x695 sys_wait4+0x78 amd64_syscall+0x36e fast_syscall_common+0x101 27574 102053 pdfork - mi_switch+0xe2 thread_suspend_switch+0xd4 ptracestop+0x13b fork_return+0x14e fork_exit+0x83 fork_trampoline+0xe As near as I can tell, truss is blocked waiting for ptrace events, the parent process is blocked in wait4, and the child process is perhaps waiting for its parent to exit the kernel so it can send the ptrace event? I really don't see anything obvious in the pdfork() code path that would cause this to happen when fork() doesn't have the problem. It may be that pdfork() just changes the timing enough to expose a latent bug. I'm seeing this on a recentish current (r351363). From owner-freebsd-current@freebsd.org Thu Sep 12 23:00:19 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DEB06DCE84 for ; Thu, 12 Sep 2019 23:00:19 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46TvPH5brKz3NHS for ; Thu, 12 Sep 2019 23:00:19 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from mousie.catspoiler.org (unknown [76.212.85.177]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) (Authenticated sender: truckman) by smtp.freebsd.org (Postfix) with ESMTPSA id 53913627B for ; Thu, 12 Sep 2019 23:00:19 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Date: Thu, 12 Sep 2019 16:00:17 -0700 (PDT) From: Don Lewis Subject: spurious out of swap kills To: FreeBSD Current Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=us-ascii Content-Disposition: INLINE X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Sep 2019 23:00:19 -0000 My poudriere machine is running 13.0-CURRENT and gets updated to the latest version of -CURRENT periodically. At least in the last week or so, I've been seeing occasional port build failures when building my default set of ports, and I finally had some time to do some investigation. It's a 16-thread Ryzen machine, with 64 GB of RAM and 40 GB of swap. Poudriere is configured with USE_TMPFS="wrkdir data localbase" and I have .if ${.CURDIR:M*/www/chromium} MAKE_JOBS_NUMBER=16 .else MAKE_JOBS_NUMBER=7 .endif in /usr/local/etc/poudriere.d/make.conf, since this gives me the best overall build time for my set of ports. This hits memory pretty hard, especially when chromium, firefox, libreoffice, and both versions of openoffice are all building at the same time. During this time, the amount of space consumed by tmpfs for /wrkdir gets large when building these large ports. There is not enough RAM to hold it all, so some of the older data spills over to swap. Swap usage peaks at about 10 GB, leaving about 30 GB of free swap. Nevertheless, I see these errors, with rustc being the usual victim: Sep 11 23:21:43 zipper kernel: pid 16581 (rustc), jid 43, uid 65534, was killed: out of swap space Sep 12 02:48:23 zipper kernel: pid 1209 (rustc), jid 62, uid 65534, was killed: out of swap space Top shows the size of rustc being about 2 GB, so I doubt that it suddenly needs an additional 30 GB of swap. I'm wondering if there might be a transient kmem shortage that is causing a malloc(..., M_NOWAIT) failure in the swap allocation path that is the cause of the problem. From owner-freebsd-current@freebsd.org Fri Sep 13 00:06:40 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8F528DEDDA for ; Fri, 13 Sep 2019 00:06:40 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46Twsr3H0fz3R57; Fri, 13 Sep 2019 00:06:40 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-io1-xd35.google.com with SMTP id n197so59154951iod.9; Thu, 12 Sep 2019 17:06:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=zXSDpPl0zFUKkD0PMalBOt4ILr01VMLqlspB62yRWIk=; b=A+wDDJ1H9RwWBg1mX4BM23Yut5RYAmvPixKV2uW3jbSNUXgB7qO3g+TXFPSo9nSwJ2 C93FUBylNy9m1wlE+aY72gg938XYPS+oho+NeJNAp+kRk/8xKXeMtRYSybls6GewaLZM tEu3vRDqZ5T7b0S521K60Wre3O4K1VUDt4hhiVDAI452fEQcT+Ls0AUtS/8T/oqIHpX1 FwT+9M96OymiRm0bYnIL/1XRVUOKeQDkDfBQfdvEfdZjlqUnhEZ+JeTFNku437tGF1Gy agF1BzR9Q3jZYybzKF/8gEQ+VSvNZz9UGler3miy9jgKhaUNEgstL7OntF/elkHGgtr2 tMpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=zXSDpPl0zFUKkD0PMalBOt4ILr01VMLqlspB62yRWIk=; b=dlhZucB5cUwHcG2NsKjjDdQICjwUaZp53zbnxROlPalLbt4tU7KVUf69GGgF5/lhgh rYakLrnSm+fsce/F6EChZk01DIMD8GRYB5wu2V7vS1EMg4hsEzRCqABk+TQWqUCHPRK9 BMk1PBOBC9VcGAPCQrrbGiEN8+4F+dXSAaYiPnl95OaPRTHlGd1G1r9goN+epEbMTXzo nPdYPWj3ryfAttRJqG7bkmNE1G4GuBRJpj54QbgwVqIeMr8LIcY/j7cHQa/9tbi0BaZb Bwj+BEYW9aXrI9ZqY7tj+4+KnW4Xn/DjHiJbIyBTh1lGbFIv2TKH67yLs7GjA/cwYhzr nGLQ== X-Gm-Message-State: APjAAAX5OKK/IkpElwkhiKC4dvPP5Sw9JFydAH63iPusaWeEJkKhwVno 1pqXNzA0ATw2wOkF+QR0+kiarVMN X-Google-Smtp-Source: APXvYqzybUUCiuxCOywauynyBxalm0VYYIqxWmO5db61Hi7pezxb06eSUAF5ckSLU9HwydAv1FIyJA== X-Received: by 2002:a6b:db0e:: with SMTP id t14mr7719780ioc.93.1568333198655; Thu, 12 Sep 2019 17:06:38 -0700 (PDT) Received: from raichu (toroon0560w-lp140-01-69-159-39-167.dsl.bell.ca. [69.159.39.167]) by smtp.gmail.com with ESMTPSA id x26sm20937946ioa.37.2019.09.12.17.06.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Sep 2019 17:06:37 -0700 (PDT) Sender: Mark Johnston Date: Thu, 12 Sep 2019 20:06:35 -0400 From: Mark Johnston To: Don Lewis Cc: FreeBSD Current , kib@freebsd.org Subject: Re: spurious out of swap kills Message-ID: <20190913000635.GG8397@raichu> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 46Twsr3H0fz3R57 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Sep 2019 00:06:40 -0000 On Thu, Sep 12, 2019 at 04:00:17PM -0700, Don Lewis wrote: > My poudriere machine is running 13.0-CURRENT and gets updated to the > latest version of -CURRENT periodically. At least in the last week or > so, I've been seeing occasional port build failures when building my > default set of ports, and I finally had some time to do some > investigation. > > It's a 16-thread Ryzen machine, with 64 GB of RAM and 40 GB of swap. > Poudriere is configured with > USE_TMPFS="wrkdir data localbase" > and I have > .if ${.CURDIR:M*/www/chromium} > MAKE_JOBS_NUMBER=16 > .else > MAKE_JOBS_NUMBER=7 > .endif > in /usr/local/etc/poudriere.d/make.conf, since this gives me the best > overall build time for my set of ports. This hits memory pretty hard, > especially when chromium, firefox, libreoffice, and both versions of > openoffice are all building at the same time. During this time, the > amount of space consumed by tmpfs for /wrkdir gets large when building > these large ports. There is not enough RAM to hold it all, so some of > the older data spills over to swap. Swap usage peaks at about 10 GB, > leaving about 30 GB of free swap. Nevertheless, I see these errors, > with rustc being the usual victim: > > Sep 11 23:21:43 zipper kernel: pid 16581 (rustc), jid 43, uid 65534, was killed: out of swap space > Sep 12 02:48:23 zipper kernel: pid 1209 (rustc), jid 62, uid 65534, was killed: out of swap space > > Top shows the size of rustc being about 2 GB, so I doubt that it > suddenly needs an additional 30 GB of swap. > > I'm wondering if there might be a transient kmem shortage that is > causing a malloc(..., M_NOWAIT) failure in the swap allocation path > that is the cause of the problem. Perhaps this is a consequence of r351114? To confirm this, you might try increasing the value of vm.pfault_oom_wait to a larger value, like 20 or 30, and see if the OOM kills still occur. From owner-freebsd-current@freebsd.org Fri Sep 13 00:42:02 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CAF01E00A2 for ; Fri, 13 Sep 2019 00:42:02 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46Txff52GJz3xvd; Fri, 13 Sep 2019 00:42:02 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from mousie.catspoiler.org (unknown [76.212.85.177]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) (Authenticated sender: truckman) by smtp.freebsd.org (Postfix) with ESMTPSA id 156FF6F67; Fri, 13 Sep 2019 00:42:01 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Date: Thu, 12 Sep 2019 17:42:00 -0700 (PDT) From: Don Lewis Subject: Re: spurious out of swap kills To: Mark Johnston cc: FreeBSD Current , kib@freebsd.org In-Reply-To: <20190913000635.GG8397@raichu> Message-ID: References: <20190913000635.GG8397@raichu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=us-ascii Content-Disposition: INLINE X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Sep 2019 00:42:02 -0000 On 12 Sep, Mark Johnston wrote: > On Thu, Sep 12, 2019 at 04:00:17PM -0700, Don Lewis wrote: >> My poudriere machine is running 13.0-CURRENT and gets updated to the >> latest version of -CURRENT periodically. At least in the last week or >> so, I've been seeing occasional port build failures when building my >> default set of ports, and I finally had some time to do some >> investigation. >> >> It's a 16-thread Ryzen machine, with 64 GB of RAM and 40 GB of swap. >> Poudriere is configured with >> USE_TMPFS="wrkdir data localbase" >> and I have >> .if ${.CURDIR:M*/www/chromium} >> MAKE_JOBS_NUMBER=16 >> .else >> MAKE_JOBS_NUMBER=7 >> .endif >> in /usr/local/etc/poudriere.d/make.conf, since this gives me the best >> overall build time for my set of ports. This hits memory pretty hard, >> especially when chromium, firefox, libreoffice, and both versions of >> openoffice are all building at the same time. During this time, the >> amount of space consumed by tmpfs for /wrkdir gets large when building >> these large ports. There is not enough RAM to hold it all, so some of >> the older data spills over to swap. Swap usage peaks at about 10 GB, >> leaving about 30 GB of free swap. Nevertheless, I see these errors, >> with rustc being the usual victim: >> >> Sep 11 23:21:43 zipper kernel: pid 16581 (rustc), jid 43, uid 65534, was killed: out of swap space >> Sep 12 02:48:23 zipper kernel: pid 1209 (rustc), jid 62, uid 65534, was killed: out of swap space >> >> Top shows the size of rustc being about 2 GB, so I doubt that it >> suddenly needs an additional 30 GB of swap. >> >> I'm wondering if there might be a transient kmem shortage that is >> causing a malloc(..., M_NOWAIT) failure in the swap allocation path >> that is the cause of the problem. > > Perhaps this is a consequence of r351114? To confirm this, you might > try increasing the value of vm.pfault_oom_wait to a larger value, like > 20 or 30, and see if the OOM kills still occur. I wonder if increasing vm.pfault_oom_attempts might also be a good idea. From owner-freebsd-current@freebsd.org Fri Sep 13 05:53:40 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F1E48E639F for ; Fri, 13 Sep 2019 05:53:40 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46V4ZD57rjz4BNT; Fri, 13 Sep 2019 05:53:40 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x8D5rW4q048713 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 13 Sep 2019 08:53:35 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x8D5rW4q048713 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x8D5rWw9048712; Fri, 13 Sep 2019 08:53:32 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 13 Sep 2019 08:53:32 +0300 From: Konstantin Belousov To: Don Lewis Cc: Mark Johnston , FreeBSD Current , kib@freebsd.org Subject: Re: spurious out of swap kills Message-ID: <20190913055332.GN2559@kib.kiev.ua> References: <20190913000635.GG8397@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-Rspamd-Queue-Id: 46V4ZD57rjz4BNT X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-5.99 / 15.00]; NEURAL_HAM_MEDIUM(-0.99)[-0.995,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Sep 2019 05:53:41 -0000 On Thu, Sep 12, 2019 at 05:42:00PM -0700, Don Lewis wrote: > On 12 Sep, Mark Johnston wrote: > > On Thu, Sep 12, 2019 at 04:00:17PM -0700, Don Lewis wrote: > >> My poudriere machine is running 13.0-CURRENT and gets updated to the > >> latest version of -CURRENT periodically. At least in the last week or > >> so, I've been seeing occasional port build failures when building my > >> default set of ports, and I finally had some time to do some > >> investigation. > >> > >> It's a 16-thread Ryzen machine, with 64 GB of RAM and 40 GB of swap. > >> Poudriere is configured with > >> USE_TMPFS="wrkdir data localbase" > >> and I have > >> .if ${.CURDIR:M*/www/chromium} > >> MAKE_JOBS_NUMBER=16 > >> .else > >> MAKE_JOBS_NUMBER=7 > >> .endif > >> in /usr/local/etc/poudriere.d/make.conf, since this gives me the best > >> overall build time for my set of ports. This hits memory pretty hard, > >> especially when chromium, firefox, libreoffice, and both versions of > >> openoffice are all building at the same time. During this time, the > >> amount of space consumed by tmpfs for /wrkdir gets large when building > >> these large ports. There is not enough RAM to hold it all, so some of > >> the older data spills over to swap. Swap usage peaks at about 10 GB, > >> leaving about 30 GB of free swap. Nevertheless, I see these errors, > >> with rustc being the usual victim: > >> > >> Sep 11 23:21:43 zipper kernel: pid 16581 (rustc), jid 43, uid 65534, was killed: out of swap space > >> Sep 12 02:48:23 zipper kernel: pid 1209 (rustc), jid 62, uid 65534, was killed: out of swap space > >> > >> Top shows the size of rustc being about 2 GB, so I doubt that it > >> suddenly needs an additional 30 GB of swap. > >> > >> I'm wondering if there might be a transient kmem shortage that is > >> causing a malloc(..., M_NOWAIT) failure in the swap allocation path > >> that is the cause of the problem. > > > > Perhaps this is a consequence of r351114? To confirm this, you might > > try increasing the value of vm.pfault_oom_wait to a larger value, like > > 20 or 30, and see if the OOM kills still occur. > > I wonder if increasing vm.pfault_oom_attempts might also be a good idea. If you are sure that you cannot exhaust your swap space, set attempts to -1 to disable this mechanism. Basically, page fault handler waits for vm.pfault_oom_wait * vm.pfault_oom_attempts for a page allocation before killing the process. Default is 30 secs, and if you cannot get a page for 30 secs, there is something very wrong with the machine. From owner-freebsd-current@freebsd.org Fri Sep 13 14:05:37 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E5A4DF26A9 for ; Fri, 13 Sep 2019 14:05:37 +0000 (UTC) (envelope-from oshogbo.vx@gmail.com) Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46VHTq5MMlz3ByX for ; Fri, 13 Sep 2019 14:05:35 +0000 (UTC) (envelope-from oshogbo.vx@gmail.com) Received: by mail-pg1-f193.google.com with SMTP id u72so15303590pgb.10 for ; Fri, 13 Sep 2019 07:05:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=a0ctwTlwL0MXYKXjkQGU3EGj8KmPHjmsxjsU5sqXwAA=; b=JrDDnuT9C0cdHB31/jqFjBetXNNyi3r3xA/06a0BBLXKOQyrLcPlGGzqNxw+WqrOAN hdyTWxw+A0cQ6yDSon3UNsxSITVcGc5sDZacbIqOE0v9PP0yYq7V0YFmtbVlJV43JjnF LP5hhqwDzFl8hRZP5Ki/8jX3UtTt085FczkM5sDGv0Z7nplH8960WkSphxFYSX6ALhzb hWYVJ+zLC9smecs/fyaxmMvU/nwVVCWjVVTsp4zK+oBU0NaUEnQTBQow3lcmIzOtZogp X71HxMOOYnmot8HjQl/I/EUoQn/yOzSiW0SsUOFJ6Z7sA7tUgRxZOqlOo/LkaRFHB9Jj ipXg== X-Gm-Message-State: APjAAAWVMtiJGkSdAkSaqTHa9Rbp2mUDRGFMX9OjvQ8tnO3sOUy2s5Nh bmMrDnscgOtGVfc+/qHL96YCmaz18y0AzypO8Ec= X-Google-Smtp-Source: APXvYqwDSQU1EvYVtl7f4bMOB0WfyxP7QtP8h0P6wN2l7QYHE3Kj8CuD0VXedBPnGY5BCaoWH9v/zLiaN6kX5vhvhR4= X-Received: by 2002:a17:90a:340d:: with SMTP id o13mr5336079pjb.19.1568383533909; Fri, 13 Sep 2019 07:05:33 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Mariusz Zaborski Date: Fri, 13 Sep 2019 16:05:21 +0200 Message-ID: Subject: Re: Deadlock involving truss -f, pdfork() and wait4() To: Ryan Stone Cc: FreeBSD Current Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46VHTq5MMlz3ByX X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of oshogbovx@gmail.com designates 209.85.215.193 as permitted sender) smtp.mailfrom=oshogbovx@gmail.com X-Spamd-Result: default: False [-3.12 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[193.215.85.209.rep.mailspike.net : 127.0.0.18]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; MIME_TRACE(0.00)[0:+]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[193.215.85.209.list.dnswl.org : 127.0.5.0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(-1.12)[ipnet: 209.85.128.0/17(-3.32), asn: 15169(-2.24), country: US(-0.05)]; FORGED_SENDER(0.30)[oshogbo@freebsd.org,oshogbovx@gmail.com]; FREEMAIL_TO(0.00)[gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; TAGGED_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[oshogbo@freebsd.org,oshogbovx@gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Sep 2019 14:05:38 -0000 Hello Ryan, Can you verify is this patch fix your issue: https://reviews.freebsd.org/D20362 Thanks, Mariusz On Thu, 12 Sep 2019 at 21:37, Ryan Stone wrote: > > I've hit an issue with a simple use of pdfork(). I have a process > that calls pdfork() and the parent immediately does a wait4() on the > child pid. This works fine under normal conditions, but if the parent > is run under truss -f, the three processes deadlock. If I switch out > pdfork() for fork(), the deadlock does not occur. > > This C file demonstrates the issue: > > https://people.freebsd.org/~rstone/pdfork.c > > If I run "truss -f ./pdfork", which uses fork(), it completes within a > second. If I run "truss -f ./pdfork -p", which uses pdfork(), the > processes deadlock. If I run "./pdfork -p" without truss, it > completes normally. > > procstat reports the following kernel stacks: > > 27572 102043 truss - mi_switch+0xe2 > sleepq_catch_signals+0x425 sleepq_wait_sig+0xf _sleep+0x1bf > kern_wait6+0x695 sys_wait6+0x9f amd64_syscall+0x36e > fast_syscall_common+0x101 > 27573 102469 pdfork - mi_switch+0xe2 > sleepq_catch_signals+0x425 sleepq_wait_sig+0xf _sleep+0x1bf > kern_wait6+0x695 sys_wait4+0x78 amd64_syscall+0x36e > fast_syscall_common+0x101 > 27574 102053 pdfork - mi_switch+0xe2 > thread_suspend_switch+0xd4 ptracestop+0x13b fork_return+0x14e > fork_exit+0x83 fork_trampoline+0xe > > As near as I can tell, truss is blocked waiting for ptrace events, the > parent process is blocked in wait4, and the child process is perhaps > waiting for its parent to exit the kernel so it can send the ptrace > event? > > I really don't see anything obvious in the pdfork() code path that > would cause this to happen when fork() doesn't have the problem. It > may be that pdfork() just changes the timing enough to expose a latent > bug. > > I'm seeing this on a recentish current (r351363). > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@freebsd.org Fri Sep 13 16:24:57 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 36EE3F5827 for ; Fri, 13 Sep 2019 16:24:57 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (www.zefox.net [50.1.20.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "www.zefox.org", Issuer "www.zefox.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46VLZc153Cz3Mx7; Fri, 13 Sep 2019 16:24:55 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (localhost [127.0.0.1]) by www.zefox.net (8.15.2/8.15.2) with ESMTPS id x8DGOiNU029093 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 13 Sep 2019 09:24:45 -0700 (PDT) (envelope-from fbsd@www.zefox.net) Received: (from fbsd@localhost) by www.zefox.net (8.15.2/8.15.2/Submit) id x8DGOiUG029092; Fri, 13 Sep 2019 09:24:44 -0700 (PDT) (envelope-from fbsd) Date: Fri, 13 Sep 2019 09:24:44 -0700 From: bob prohaska To: Konstantin Belousov Cc: Don Lewis , Mark Johnston , FreeBSD Current , kib@freebsd.org, bob prohaska Subject: Re: spurious out of swap kills Message-ID: <20190913162444.GA28886@www.zefox.net> References: <20190913000635.GG8397@raichu> <20190913055332.GN2559@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190913055332.GN2559@kib.kiev.ua> User-Agent: Mutt/1.5.24 (2015-08-30) X-Rspamd-Queue-Id: 46VLZc153Cz3Mx7 X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of fbsd@www.zefox.net has no SPF policy when checking 50.1.20.27) smtp.mailfrom=fbsd@www.zefox.net X-Spamd-Result: default: False [2.79 / 15.00]; ARC_NA(0.00)[]; WWW_DOT_DOMAIN(0.50)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; IP_SCORE(0.08)[ip: (0.34), ipnet: 50.1.16.0/20(0.17), asn: 7065(-0.05), country: US(-0.05)]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[zefox.net]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.57)[0.573,0]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.24)[0.237,0]; R_SPF_NA(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7065, ipnet:50.1.16.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; MID_RHS_WWW(0.50)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Sep 2019 16:24:57 -0000 Not sure this is relevant, but in compiling chromium on a Pi3 with 6 GB of swap the job completed successfully some months ago, with peak swap use around 3.5 GB. The swap layout was sub-optimal, with a 2 GB partition combined with a 4 GB partition. A little over 4GB total seems usable. A few days ago the same attempt stopped with a series of OOMA kills, but in each case simply restarting allowed the compile to pick up where it left off and continue, eventually finishing with a runnable version of chromium. In this case swap use peaked a little over 4 GB. Might this suggest the machine isn't freeing swap in a timely manner? Thanks for reading, bob prohaska From owner-freebsd-current@freebsd.org Fri Sep 13 18:13:08 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6D8EBF7458 for ; Fri, 13 Sep 2019 18:13:08 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46VNzS2Mzgz3xqP; Fri, 13 Sep 2019 18:13:08 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-qt1-x843.google.com with SMTP id g16so1413199qto.9; Fri, 13 Sep 2019 11:13:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=R0jxhCKH/ykKWRzx1nsZPZVQKqlCqmbFf6nq9D07Rvc=; b=rIKvbvDmFxIpKY1c3bg8XRfFW7LHSFcr+COjLUbZoUd5/tXQMmOD0q3BudZD3vz1Pj GaS0t93yplaY9AD8V7pV7nq5gNx5DuYChYwryHHpAh1it0mx8bcbYWhWHG81WLyCB0nM P/xWG2+OoyfYUXFkMg7Wv4qAd/tnzF2wZ2grnj2igTntLDOHCyyl+GgXGqe9AcupSN73 qakFqWVHvPJLHW2BcTP1nv8fzM5AViB7FcLj/yslhy0ZDOE+vz6yngrXTMT7Qch4mx68 rxMvyDpDtpAez+ONkNFxtSSkcvqCZiHL5Zgq5nTe2Er3IF/oUw2IRAEAERupGt9JOJRX kNww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=R0jxhCKH/ykKWRzx1nsZPZVQKqlCqmbFf6nq9D07Rvc=; b=FaPGw8ip4LF9csKuN55RrpgAJRIwYrqBaXw9nevvrXwl9o2DkGQ4jGzGji6i3yKb8u Q1LOm696XVRguAm0f3ILy10TYBI5a9J9WUuh+83wAAdUvMUeIoplL3/Xn6K//1yegA5i DhqX6wWD3aeoVFW68moDfug87nA1NFuQqp/GrF0KEH2KZe33qWeGUGDN63vYF07XuXfM lKLqqcdS2ns4zOJR198O2WMKWKk4mq/nULigqKdANyJp6D0i5Hx8BcY4NcvXHN3XjXR4 0JUJ+FmHYRmNgYSiH3+tz3RSR8TcB+XJPdA+Entasg4HSRiAZezWHH6STwvdd1sPdvmy k+3A== X-Gm-Message-State: APjAAAVilOikH1XBiQJGRwDJbdgN+tfBeOwm46HLlgQjd3FqLN+XYTd6 tnFfGGb64SDAMS3c5oYX/5s9xujAs5hHwD0//6/mVwUnoWk= X-Google-Smtp-Source: APXvYqzL2HOnOw5TETrgwQE87xHs5U1IuVhM76OT4tgZ1ELif7X1E7TRuFBNbjyaEhibUs2koJIqtAMGa9pwLX/roQk= X-Received: by 2002:ac8:845:: with SMTP id x5mr4477819qth.42.1568398386792; Fri, 13 Sep 2019 11:13:06 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Ryan Stone Date: Fri, 13 Sep 2019 14:12:56 -0400 Message-ID: Subject: Re: Deadlock involving truss -f, pdfork() and wait4() To: Mariusz Zaborski Cc: FreeBSD Current Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46VNzS2Mzgz3xqP X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Sep 2019 18:13:08 -0000 This gets me a little further but now the wait4 call by the parent never reaps the child and instead blocks forever: # truss -f ./pdfork -p 706: mmap(0x0,131072,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 34361 970688 (0x800221000) 706: issetugid() = 0 (0x0) 706: open("/etc/libmap.conf",O_RDONLY|O_CLOEXEC,010440020) = 3 (0x3) 706: fstat(3,{ mode=-rw-r--r-- ,inode=241058,size=47,blksize=32768 }) = 0 (0x0 ) 706: read(3,"# $FreeBSD$\nincludedir /usr/loc"...,47) = 47 (0x2f) 706: close(3) = 0 (0x0) 706: open("/usr/local/etc/libmap.d",O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC, 0165) ERR#2 'No such file or directory' 706: open("/var/run/ld-elf.so.hints",O_RDONLY|O_CLOEXEC,010002355) = 3 (0x3) 706: read(3,"Ehnt\^A\0\0\0\M^@\0\0\0A\0\0\0\0"...,128) = 128 (0x80) 706: fstat(3,{ mode=-r--r--r-- ,inode=72,size=193,blksize=4096 }) = 0 (0x0) 706: pread(3,"/lib:/usr/lib:/usr/lib/compat:/u"...,65,0x80) = 65 (0x41) 706: close(3) = 0 (0x0) 706: open("/lib/libc.so.7",O_RDONLY|O_CLOEXEC|O_VERIFY,057400) = 3 (0x3) 706: fstat(3,{ mode=-r--r--r-- ,inode=402754,size=1915744,blksize=32768 }) = 0 (0x0) 706: mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 3436210176 0 (0x800241000) 706: mmap(0x0,4169728,PROT_NONE,MAP_GUARD,-1,0x0) = 34362105856 (0x800242000) 706: mmap(0x800242000,524288,PROT_READ,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PR EFAULT_READ,3,0x0) = 34362105856 (0x800242000) 706: mmap(0x8002c2000,1327104,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NO CORE|MAP_PREFAULT_READ,3,0x80000) = 34362630144 (0x8002c2000) 706: mmap(0x800406000,61440,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PRE FAULT_READ,3,0x1c4000) = 34363957248 (0x800406000) 706: mmap(0x800415000,2256896,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_A NON,-1,0x0) = 34364018688 (0x800415000) 706: munmap(0x800241000,4096) = 0 (0x0) 706: close(3) = 0 (0x0) 706: mprotect(0x80040c000,36864,PROT_READ) = 0 (0x0) 706: sysarch(AMD64_SET_FSBASE,0x7fffffffe110) = 0 (0x0) 706: sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIG TERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFS Z|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0) 706: mprotect(0x80040c000,36864,PROT_READ|PROT_WRITE) = 0 (0x0) 706: sigprocmask(SIG_SETMASK,{ },0x0) = 0 (0x0) 706: sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIG TERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFS Z|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0) 706: mprotect(0x80040c000,36864,PROT_READ) = 0 (0x0) 706: sigprocmask(SIG_SETMASK,{ },0x0) = 0 (0x0) 706: readlink("/etc/malloc.conf",0x7fffffffd830,1024) ERR#2 'No such file or d irectory' 706: issetugid() = 0 (0x0) 706: __sysctl(0x7fffffffd7e0,0x2,0x7fffffffd7dc,0x7fffffffd7d0,0x0,0x0) = 0 (0 x0) 706: mmap(0x0,2097152,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(21 ),-1,0x0) = 34368126976 (0x800800000) 706: cap_getmode({ 0 }) = 0 (0x0) 706: open("/dev/hpet0",O_RDONLY,00) = 3 (0x3) 706: mmap(0x0,4096,PROT_READ,MAP_SHARED,3,0x0) = 34362101760 (0x800241000) 706: close(3) = 0 (0x0) 706: mmap(0x0,32768,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12), -1,0x0) = 34366275584 (0x80063c000) 706: mmap(0x0,4194304,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(21 ),-1,0x0) = 34370224128 (0x800a00000) 706: mmap(0x0,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),- 1,0x0) = 34366308352 (0x800644000) 706: sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIG TERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFS Z|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0) 706: sigprocmask(SIG_SETMASK,{ },0x0) = 0 (0x0) 706: sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIG TERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFS Z|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0) 706: mprotect(0x203000,4096,PROT_READ) = 0 (0x0) 706: sigprocmask(SIG_SETMASK,{ },0x0) = 0 (0x0) 706: pdfork(0x7fffffffeba8,0x0) = 708 (0x2c4) 708: 708: nanosleep({ 1.000000000 }) = 0 (0x0) 708: sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIG TERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFS Z|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0) 708: sigprocmask(SIG_SETMASK,{ },0x0) = 0 (0x0) 708: sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIG TERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFS Z|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0) 708: sigprocmask(SIG_SETMASK,{ },0x0) = 0 (0x0) 708: sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIG TERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFS Z|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0) 708: sigprocmask(SIG_SETMASK,{ },0x0) = 0 (0x0) load: 0.27 cmd: pdfork 706 [wait] 18.20r 0.00u 0.00s 0% 2072k # ps PID TT STAT TIME COMMAND 698 u0 Is 0:00.01 login [pam] (login) 700 u0 I 0:00.04 -sh (sh) 705 u0 I+ 0:00.10 truss -f ./pdfork -p 706 u0 IX+ 0:00.01 ./pdfork -p 708 u0 Z+ 0:00.00 714 0 S 0:00.01 su 715 0 S 0:00.01 su (sh) 716 0 R+ 0:00.00 ps # procstat -kk 708 PID TID COMM TDNAME KSTACK # procstat -kk 706 PID TID COMM TDNAME KSTACK 706 100095 pdfork - mi_switch+0x174 sleepq_switch+0x110 sleepq_catch_signals+0x417 slee pq_wait_sig+0xf _sleep+0x2d0 kern_wait6+0x48f sys_wait4+0x78 amd64_syscall+0x337 fast_syscall_common+0x101 # procstat -kk 705 PID TID COMM TDNAME KSTACK 705 100077 truss - mi_switch+0x174 sleepq_switch+0x110 sleepq_catch_signals+0x417 slee pq_wait_sig+0xf _sleep+0x2d0 kern_wait6+0x48f sys_wait6+0x9f amd64_syscall+0x337 fast_syscall_common+0x101 On Fri, Sep 13, 2019 at 10:05 AM Mariusz Zaborski wrote: > > Hello Ryan, > > Can you verify is this patch fix your issue: > https://reviews.freebsd.org/D20362 > > Thanks, > Mariusz > > On Thu, 12 Sep 2019 at 21:37, Ryan Stone wrote: > > > > I've hit an issue with a simple use of pdfork(). I have a process > > that calls pdfork() and the parent immediately does a wait4() on the > > child pid. This works fine under normal conditions, but if the parent > > is run under truss -f, the three processes deadlock. If I switch out > > pdfork() for fork(), the deadlock does not occur. > > > > This C file demonstrates the issue: > > > > https://people.freebsd.org/~rstone/pdfork.c > > > > If I run "truss -f ./pdfork", which uses fork(), it completes within a > > second. If I run "truss -f ./pdfork -p", which uses pdfork(), the > > processes deadlock. If I run "./pdfork -p" without truss, it > > completes normally. > > > > procstat reports the following kernel stacks: > > > > 27572 102043 truss - mi_switch+0xe2 > > sleepq_catch_signals+0x425 sleepq_wait_sig+0xf _sleep+0x1bf > > kern_wait6+0x695 sys_wait6+0x9f amd64_syscall+0x36e > > fast_syscall_common+0x101 > > 27573 102469 pdfork - mi_switch+0xe2 > > sleepq_catch_signals+0x425 sleepq_wait_sig+0xf _sleep+0x1bf > > kern_wait6+0x695 sys_wait4+0x78 amd64_syscall+0x36e > > fast_syscall_common+0x101 > > 27574 102053 pdfork - mi_switch+0xe2 > > thread_suspend_switch+0xd4 ptracestop+0x13b fork_return+0x14e > > fork_exit+0x83 fork_trampoline+0xe > > > > As near as I can tell, truss is blocked waiting for ptrace events, the > > parent process is blocked in wait4, and the child process is perhaps > > waiting for its parent to exit the kernel so it can send the ptrace > > event? > > > > I really don't see anything obvious in the pdfork() code path that > > would cause this to happen when fork() doesn't have the problem. It > > may be that pdfork() just changes the timing enough to expose a latent > > bug. > > > > I'm seeing this on a recentish current (r351363). > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@freebsd.org Fri Sep 13 18:37:23 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 88012F7A96 for ; Fri, 13 Sep 2019 18:37:23 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46VPWQ6T1jz3yl3; Fri, 13 Sep 2019 18:37:22 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-io1-xd30.google.com with SMTP id d17so42791449ios.13; Fri, 13 Sep 2019 11:37:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=5Qx0GXouBf82DYf/SnRPVZZBk6ja8twCzszm05JDo9M=; b=bSzODw+d0MlFt74H7nV4JPlYdLIwAcYSqcSqrLw+Cxh17Qz9gRR0MF/fPWEwmIOEcR SI/5jWDxv7rH7GgIpc4wDVMxOKPCW1LWRpuPAzEn5ga4PaGoxU2iPzIFzsudZFBt2Lgy I3mAxf1rTJYKPSu80TPEhJF08NfeGSRQqW7E4NSpH35rRf7VLzSg/XgBwS7wt2HjiGBj 2McIAC9Qp8p+RPXBpqLKM5E4ORIbNJph3F8Y1UIQzjCha+SMC5ULneigoLs2Aas4W4lu 7QcTWNo1Nd4ZS/fXCJYejFr+jUMt2+pLaViNOVyibxpB4wc+O+6d0+HFjf74/6BPzMaB iy9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=5Qx0GXouBf82DYf/SnRPVZZBk6ja8twCzszm05JDo9M=; b=tlSYLcrFQCZUsaBtjqTSGzhLPipAwsustcSATU8G4n13fZ967k4O+mPJ+ZO2QbEWIY ItfvV24TDuOGJhlsAaTbhAEY/be/Y8rUMD+uiqUq1/0/Jn5wj17GWWt4dsgTKu+CcvY7 QXpS/Xe87bdH1pGmcKRggcr48HeIKGGQ793Jf7WFJzzXRdGpjblIYtY29/jSiefVUFBz PcXwIta1ZtFOAAAvpQVLQbMINFvYCXDifmA/5RVwfZAenPW+CiGgwgWWC7N4XQBofzI+ 5EsrFzjzK43lINLNx17kawoQTApJJd5J+RYo+NTaVb3UJK1H5dhhYTQzGmzlmAAm7l6P BblA== X-Gm-Message-State: APjAAAUXKWmOvHwEwGOqUQNtcKnlbx0OEewOn0tK/M6op8VrxK+zW8vW IlqekhWeJDGX6V+cRWmwHQ0s2dFQ X-Google-Smtp-Source: APXvYqzV9mkyzjw9IRU0NGPnEVRlY/EZPXTy4hTk4ZcC+tGKK9Zi5y/rknuAra33YuBiQF8iCdvJXw== X-Received: by 2002:a02:712b:: with SMTP id n43mr24250607jac.2.1568399841986; Fri, 13 Sep 2019 11:37:21 -0700 (PDT) Received: from raichu (toroon0560w-lp140-01-69-159-39-167.dsl.bell.ca. [69.159.39.167]) by smtp.gmail.com with ESMTPSA id k22sm20409152iom.45.2019.09.13.11.37.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Sep 2019 11:37:20 -0700 (PDT) Sender: Mark Johnston Date: Fri, 13 Sep 2019 14:37:18 -0400 From: Mark Johnston To: Ryan Stone Cc: Mariusz Zaborski , FreeBSD Current Subject: Re: Deadlock involving truss -f, pdfork() and wait4() Message-ID: <20190913183718.GC84156@raichu> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 46VPWQ6T1jz3yl3 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=bSzODw+d; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::d30 as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-3.73 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCVD_IN_DNSWL_NONE(0.00)[0.3.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.03)[ip: (-5.17), ipnet: 2607:f8b0::/32(-2.71), asn: 15169(-2.24), country: US(-0.05)]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; FREEMAIL_TO(0.00)[gmail.com]; MID_RHS_NOT_FQDN(0.50)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Sep 2019 18:37:23 -0000 On Fri, Sep 13, 2019 at 02:12:56PM -0400, Ryan Stone wrote: > This gets me a little further but now the wait4 call by the parent > never reaps the child and instead blocks forever: Does it perform a wildcarded wait(), or does it explicitly specify the PID of the child? By design, the former will not return children created by pdfork(). From owner-freebsd-current@freebsd.org Fri Sep 13 18:56:02 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AC452F7FE0 for ; Fri, 13 Sep 2019 18:56:02 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-io1-f47.google.com (mail-io1-f47.google.com [209.85.166.47]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46VPwy46Gqz40kD; Fri, 13 Sep 2019 18:56:02 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-io1-f47.google.com with SMTP id n197so64870990iod.9; Fri, 13 Sep 2019 11:56:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=1kro4TsNc2u5gD9ePloPjURn3UaksDqqER6TId0HiZk=; b=LQTBFD5qGrRr5sOkJakqQqzpwW0aMuyDEMXz9Nb+QpWkMwnAGgj4JoAAXQUbhNJigi Px/IinK0dYjFoDP5Ui0r3w6F1NC7UjJoNlwIA+s3Po4g/8p3YELwE4imKzxYzTCRb4kg ZqzytIi8VHJ2bbpqt+PWWCCTrAX5vHNsX5iD6/efHkiXKpOmBif5k9ajSMpQGCWTGXfW vFL8BKKM+xq2lPubF5tZzkKYYU3frX3IsU+hWBt49UvAkqI1VQZ/dGvpTWYwIhPyCVVG B44bDK8o1rHSMws7NaZoN2vouN5a3EkfKgcEVmIVp3BQABKEnUlg0H4/qpWYAABEgegd at1g== X-Gm-Message-State: APjAAAXwMOr7eMdDypAUpQWmv0HIFIoiMvY/ilbKQJlxuZJ/9NDDOUEC 0Go2aioE8OT1SskmvtStsxpiDoog X-Google-Smtp-Source: APXvYqxfmURrSVSZO00sjwMGAA7tSdg3li/81csPuPTYvsqTjPlJD5qA6EIIMgApxpzcq+FkKeA5yQ== X-Received: by 2002:a02:7009:: with SMTP id f9mr24052jac.81.1568400961147; Fri, 13 Sep 2019 11:56:01 -0700 (PDT) Received: from mail-io1-f48.google.com (mail-io1-f48.google.com. [209.85.166.48]) by smtp.gmail.com with ESMTPSA id h4sm27073904iok.1.2019.09.13.11.56.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 13 Sep 2019 11:56:00 -0700 (PDT) Received: by mail-io1-f48.google.com with SMTP id r26so64691440ioh.8; Fri, 13 Sep 2019 11:56:00 -0700 (PDT) X-Received: by 2002:a5e:c74d:: with SMTP id g13mr1408215iop.65.1568400960791; Fri, 13 Sep 2019 11:56:00 -0700 (PDT) MIME-Version: 1.0 References: <20190913183718.GC84156@raichu> In-Reply-To: <20190913183718.GC84156@raichu> Reply-To: cem@freebsd.org From: Conrad Meyer Date: Fri, 13 Sep 2019 11:55:49 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Deadlock involving truss -f, pdfork() and wait4() To: Mark Johnston Cc: Ryan Stone , FreeBSD Current Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46VPwy46Gqz40kD X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Sep 2019 18:56:02 -0000 On Fri, Sep 13, 2019 at 11:37 AM Mark Johnston wrote: > > On Fri, Sep 13, 2019 at 02:12:56PM -0400, Ryan Stone wrote: > > This gets me a little further but now the wait4 call by the parent > > never reaps the child and instead blocks forever: > > Does it perform a wildcarded wait(), or does it explicitly specify the > PID of the child? By design, the former will not return children > created by pdfork(). Explicit PID: https://people.freebsd.org/~rstone/pdfork.c Best, Conrad From owner-freebsd-current@freebsd.org Fri Sep 13 19:04:09 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4A7D3D03B7 for ; Fri, 13 Sep 2019 19:04:09 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46VQ6K18Shz41DP; Fri, 13 Sep 2019 19:04:08 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-qk1-x732.google.com with SMTP id u184so26334035qkd.4; Fri, 13 Sep 2019 12:04:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=vzlFZh13uypuw6jx1qKWMeDx8zged9i5dMQzrKz/t/M=; b=rmm8Mzv2uwd870Y2zYQ/0178OKr/zaA+33/X2Ca/y/HXpiGGy7CNHtuGwChisxqxUv bkBHgWNTjnvS9HKgS2WqPkXY+IgUg1tSVSL4cZTdPEGGXUsgmNIu/y9zLMaDL11xuof0 o619WwaSfDkrbX2B3uDc0tNXW7DGgkJ5oNM2j7M68VYeZPXOdmvH7sbzf/pkfe+X4Zyo wGIxodboUxPYEiF1zRgvJVX84WhSKiocq10ZPwa9qsYdwODUCV+EZpGa8+7xta7MG11p b9wYsXvPIPnAXvMNlxaA6oudHfxCoFl30o68YK1ujgQh8tpGYEp36dyzWG+9Xvk+jlC5 0G7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=vzlFZh13uypuw6jx1qKWMeDx8zged9i5dMQzrKz/t/M=; b=KQMOGB1i1xOjOBQEiIBb0OJ4hCzqDdf5eU8lXXlrN23gYhCBrBe+5RgkYCXJojvbGU WV2i8Dz6GjsHpyypdeKd5fi+pbLoKsC/IC1NKSO64bPHS3Ha10EqLlTpYJ0aapkB79D8 z9brk70xufe+4wvSSSoenkttrzN8ESU15MgSOLs3ADg+7n4WEHcWKlSiqs5TV85xkcg/ ISkBM9fdKaWVYzYrdKfMW2BhRU3PK4/YlKNmlcUWD5zLWa3R6ld5Nge8TKeqMr6W8/E8 AseyItSyIYIsYy1qsYOZJkI0cl2v6+Rrx2PFZ5qzJthooIU39m0+yJa+zAk0Tzpn2Sht hNMw== X-Gm-Message-State: APjAAAXDsvu4I0ph02QJwEzyqJbXSZxhKvzqnVRKZSxjfMztTVa/SHXr UIMFv3ssWojOMUjsGZcUOQyuufYW2q06GRZpMSp6m+YX X-Google-Smtp-Source: APXvYqwCMWiFMpzXYoW9uvCi3JYhfgcHlqMILGJdcYy8oBsypg0SST/PapIWOWU2uFPzTvZxZu/2CuMbvSpo2eC/8JY= X-Received: by 2002:a37:a843:: with SMTP id r64mr45063413qke.363.1568401447743; Fri, 13 Sep 2019 12:04:07 -0700 (PDT) MIME-Version: 1.0 References: <20190913183718.GC84156@raichu> In-Reply-To: <20190913183718.GC84156@raichu> From: Ryan Stone Date: Fri, 13 Sep 2019 15:03:56 -0400 Message-ID: Subject: Re: Deadlock involving truss -f, pdfork() and wait4() To: Mark Johnston Cc: Mariusz Zaborski , FreeBSD Current Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46VQ6K18Shz41DP X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Sep 2019 19:04:09 -0000 As Conrad has pointed out, it's an explicit PID. The test completes successfully when not run under truss -f. On Fri, Sep 13, 2019 at 2:37 PM Mark Johnston wrote: > > On Fri, Sep 13, 2019 at 02:12:56PM -0400, Ryan Stone wrote: > > This gets me a little further but now the wait4 call by the parent > > never reaps the child and instead blocks forever: > > Does it perform a wildcarded wait(), or does it explicitly specify the > PID of the child? By design, the former will not return children > created by pdfork(). From owner-freebsd-current@freebsd.org Fri Sep 13 23:39:14 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3DE07D64FB for ; Fri, 13 Sep 2019 23:39:14 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46VXCk0hZJz4FnS; Fri, 13 Sep 2019 23:39:14 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from mousie.catspoiler.org (unknown [76.212.85.177]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) (Authenticated sender: truckman) by smtp.freebsd.org (Postfix) with ESMTPSA id 7448911144; Fri, 13 Sep 2019 23:39:13 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Date: Fri, 13 Sep 2019 16:39:10 -0700 (PDT) From: Don Lewis Subject: Re: spurious out of swap kills To: Mark Johnston cc: FreeBSD Current , kib@freebsd.org In-Reply-To: Message-ID: References: <20190913000635.GG8397@raichu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=us-ascii Content-Disposition: INLINE X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Sep 2019 23:39:14 -0000 On 12 Sep, Don Lewis wrote: > On 12 Sep, Mark Johnston wrote: >> On Thu, Sep 12, 2019 at 04:00:17PM -0700, Don Lewis wrote: >>> My poudriere machine is running 13.0-CURRENT and gets updated to the >>> latest version of -CURRENT periodically. At least in the last week or >>> so, I've been seeing occasional port build failures when building my >>> default set of ports, and I finally had some time to do some >>> investigation. >>> >>> It's a 16-thread Ryzen machine, with 64 GB of RAM and 40 GB of swap. >>> Poudriere is configured with >>> USE_TMPFS="wrkdir data localbase" >>> and I have >>> .if ${.CURDIR:M*/www/chromium} >>> MAKE_JOBS_NUMBER=16 >>> .else >>> MAKE_JOBS_NUMBER=7 >>> .endif >>> in /usr/local/etc/poudriere.d/make.conf, since this gives me the best >>> overall build time for my set of ports. This hits memory pretty hard, >>> especially when chromium, firefox, libreoffice, and both versions of >>> openoffice are all building at the same time. During this time, the >>> amount of space consumed by tmpfs for /wrkdir gets large when building >>> these large ports. There is not enough RAM to hold it all, so some of >>> the older data spills over to swap. Swap usage peaks at about 10 GB, >>> leaving about 30 GB of free swap. Nevertheless, I see these errors, >>> with rustc being the usual victim: >>> >>> Sep 11 23:21:43 zipper kernel: pid 16581 (rustc), jid 43, uid 65534, was killed: out of swap space >>> Sep 12 02:48:23 zipper kernel: pid 1209 (rustc), jid 62, uid 65534, was killed: out of swap space >>> >>> Top shows the size of rustc being about 2 GB, so I doubt that it >>> suddenly needs an additional 30 GB of swap. >>> >>> I'm wondering if there might be a transient kmem shortage that is >>> causing a malloc(..., M_NOWAIT) failure in the swap allocation path >>> that is the cause of the problem. >> >> Perhaps this is a consequence of r351114? To confirm this, you might >> try increasing the value of vm.pfault_oom_wait to a larger value, like >> 20 or 30, and see if the OOM kills still occur. > > I wonder if increasing vm.pfault_oom_attempts might also be a good idea. sysctl vm.pfault_oom_attempts=10 by itself seemed to work well. From owner-freebsd-current@freebsd.org Sat Sep 14 06:00:07 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C4835E8700 for ; Sat, 14 Sep 2019 06:00:07 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-15.consmr.mail.bf2.yahoo.com (sonic309-15.consmr.mail.bf2.yahoo.com [74.6.129.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46VhgB5FRBz4Y9l for ; Sat, 14 Sep 2019 06:00:06 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: LzuxDPwVM1nZysMYI6qdNL2WhSIxhlL6.iUUJjOg2q50oQkptBuQaaahKUmBGlW JLE2ErGM.knpr88moy6hcVfpqQafBjkfCVk3bOaV.CrWYoAeGJQbmd6F70vgnRQNx6Dlizh.jt0Q cYJbUGvVmoGxuGTcCeYpqpgd5CROPUI8jyakTLP1jRsia9XmrWmmY0GecnHEZzj89y4m5nim07tj QJLX64nffurNcRMRDoOzzwev_udCthDxACHr9f7BW_e2IA2aGihifob0JC.5Z036UuDNuZQeQwhI F_KzDEHyQSgTPMfT8zluSQCQQmAex9KPjEzau4VzJY3mxjLjvR4DglTyxbFBYu5H8kkKf5HSZPvG o881Ro.3mLKohXSdrGL.RUjA2B5nTAMbrWu1MRxwdtMvPDFsEWTox.8PvqIz9UWdSKLckszrtTGf 4vqOUerIvzB7lfcuomC7lQpb4xBTTY49ehI0RDwGlTfWWnRXSsKsGP31f5ue_WyiH0NG1i.8kf2B DsTM75XNawRd7riAt7jcjLo0N58szHjb.9EYPOVQpZQnJeon98deQBPIsds9FstgMXDr2WaVivZM Q0ZDRT44NZh4CGYcrP1MrwlgdR4elCf_JZ8WiG0l7G9ngmrdSpXUuQGwfg1pYViRPEfKEd1pXeHs 6UimJBt0ePFo0hu_Q26TvmoNuaDuIs0A7X09d_pIXzMuz19D.hKg6LbWLL2iPpjOtEwyefC0Ghxm 6VJ25pM0585Smv4MrSPDsj9G0x9OLkqRvGgo9BPgCT1NHS7Jpou7psffsuOliQrRdvTLxqc6.DOL zqfNlt2AJFaIZz0N0q0F30z3TL_7lw0QYUcP4UMqcYSunzz_fymioJjg3ZgzaZPls00QPC6K8yIq kuqWz9vGbH5aIR5guN_CGkOGizRnoH0nUh4UBO.Nij9iS5QhK53LayCgNZ_3Q_VbFtQVSvKBfPAo aNh3khxvjbQpbsX3oRgI.cSGsUSsseaKL0JT6C8z41XpDbwvyve0V6FNuse6RfZR9D68ZPT_IffY iyZfPWP1b9XfxrW9LkxVNFiRoo1tHlrC8G1lwSWvuCWn.5V.mAVQRolpuC8wm6_3fKKzcCO5tJFa Z_SOhKnqas4pyccK6l3nu99v2rHaZXje5GOVWCE.2HlLF2AAYJHdRycwzHT.8LfeL7u7Jjvo5rCv 2qelU7IpPOKEnEB8GJG45ahh6sXPSB2hJjeAG8h5IJtg_TxPZwJP9OfNg0TSibyccn.XCksyCHJc garohi2RkN3uBxhit8XITTbFHdLQZff02FZeMfD7PIfMxpDUJ1Z2GSW4j8A-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.bf2.yahoo.com with HTTP; Sat, 14 Sep 2019 06:00:05 +0000 Received: by smtp407.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 6ba0ac3b58b1275bd760445b71d0f5e0; Sat, 14 Sep 2019 06:00:00 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: spurious out of swap kills Message-Id: <28BF21DA-8B16-4CD8-8E5E-C1B596FE3684@yahoo.com> Date: Fri, 13 Sep 2019 22:59:58 -0700 To: bob prohaska , FreeBSD Current X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 46VhgB5FRBz4Y9l X-Spamd-Bar: / X-Spamd-Result: default: False [0.72 / 15.00]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; FAKE_REPLY(1.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-0.04)[-0.039,0]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(0.00)[ip: (3.36), ipnet: 74.6.128.0/21(1.45), asn: 26101(1.16), country: US(-0.05)]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.26)[0.256,0]; RCVD_IN_DNSWL_NONE(0.00)[125.129.6.74.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Sep 2019 06:00:07 -0000 bob prohaska fbsd at www.zefox.net wrote on Fri Sep 13 16:24:57 UTC 2019 : > Not sure this is relevant, but in compiling chromium on a Pi3 with 6 GB > of swap the job completed successfully some months ago, with peak swap > use around 3.5 GB. The swap layout was sub-optimal, with a 2 GB partition > combined with a 4 GB partition. A little over 4GB total seems usable. > > A few days ago the same attempt stopped with a series of OOMA kills, > but in each case simply restarting allowed the compile to pick up > where it left off and continue, eventually finishing with a runnable > version of chromium. In this case swap use peaked a little over 4 GB. > > Might this suggest the machine isn't freeing swap in a timely manner? Are you saying that your increases to: vm.pageout_oom_seq no longer prove sufficient? What value for vm.pageout_oom_seq were you using that got the recent failures? (Mark Johnston's slow_swap.diff patch [and related] for investigations of some OOM killing contributions never became official and has not been updated to apply to updated code. It has been over a year since those patches were used for the arm small board computer investigations that lead to my learning about vm.pageout_oom_seq .) If more or different configuration/tuning is required, I'm going to eventually want to learn about it as well. === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-current@freebsd.org Sat Sep 14 06:14:59 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 80C75E8FB4 for ; Sat, 14 Sep 2019 06:14:59 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-1.consmr.mail.bf2.yahoo.com (sonic303-1.consmr.mail.bf2.yahoo.com [74.6.131.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46Vj0L2qYlz4ZJ2 for ; Sat, 14 Sep 2019 06:14:58 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: 2CDQbBUVM1kB8cZRdmfDXalU5zBJAVMcClER97ud17rlTarm9uOuRKQD6KXGrOI svjQzNla8SdUD12RjmNOmGMtyfTlzC3wu3NpawZiCg3OT.s_JJwM5S69_V6iu4r6J7ROjfKTpyEG CzK07I7Hg3f.NW2gtNycgh_cwogv45UyRqs17EuLXXK6xISr_NgpuAkD1ZjjRQcaaFJFpyZsnYyd bJbcciPIx4PnhGvAyv34y_YDt1ny0FAQWPJBuKU0WghoIT5L37kZ4w1ywLye1Cf97KcqH6pFJajT X7XdFWZ1r2rE_pyCE_ttxecEBgFt2scAt1F8qnwHNIr2d6KJuT7.Okp1scMw5B54zGOLibXjdFc3 VDyvqz5D9wIrAqzc_HkdNOp115cq_830suQ2Lu01_Bz3bdbO6GPAawSoR.uT9H.BZwBpaaYYlmmG oeV1qyEImtnVfJ0rpwM.qmDoCKVnXyZTo1XDbzxhuLXT82VFEqa5TUZPQT2GuhMZNsZEIGiG.gy9 9oUxmjVzmPABBFFL9.BcDNkh0_.8F4W06VOtZzO0cScaVymsViBW4r0wTTedAKHc28wp9MLIbdlJ GArj3VOrJTOPSB1cidDmPbavCUlSkEqN1epsdcw37mXxMyTdRUte9FKa3BQ.dy_86xe6PCJ2kpDp DXfIuEZ_MAHu9IuhorbYLUUrwMxFmlV1_0SfgvmrX6xVKamnX2pslULj4NahCJvpnm6XIC.Yej2N C1vaZD8t1tc.B6WMgL_6hEtfRxhLoGZ83.EAAQ8ujXI1ghV8.LQTcUSBUB7ubQ4o9dHmZpNWGlSB qICcHqXRaoAx0HqmDbyGafDImdCIKFffApe5YAoEMaIyPwED9o2hUeDdH0jolAwxGaQPM1lGjjLf N_nmdP.q4uWtYWkv9vw9YFyJgJ6O3ynWpBwXhZLEvRcKkg4Vj9mOohiXgLjn3wYTazW6GwElPeif Py1zsWl5NNKCb2iG2H_LUJERaAJjpKZ4PyBQHJFj.GMmsYw7dir8lsXdtiAjszrGVNlNaP9cVqxO nfj.xLtZp1dfAs1apnAgJMdZqt7Ng4_bgNI.tsTQUQ4ptUbVzEPFldOY1Uv0JaOLC6_PO2WI5_eS taHBejvi7D9yCaztZksT2UxOCzn1VyM6h.uHNB3HrUjCjIijKKEn2JPVthifnZsYN7AQ2ZlPxQqK _f5EYy48FfxE.x3ii_uue6CsPzXBk2jGoVnDxLnn6sX5SwdZf7.e0WgFM.TQ_Ph36N_ZbUju5q2O GzYEKOTqQ0pfeUrUib3e7pmopL5P0GMe3bWHfkwh6kwrFevHPINA8_O_nFLM- Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.bf2.yahoo.com with HTTP; Sat, 14 Sep 2019 06:14:57 +0000 Received: by smtp426.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 73493a2f06f187fec621f15ed484febd; Sat, 14 Sep 2019 06:14:53 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: spurious out of swap kills Message-Id: <9A2342FC-D168-485A-9918-0109F9CD1646@yahoo.com> Date: Fri, 13 Sep 2019 23:14:50 -0700 To: " truckman@freebsd.org " , FreeBSD Current X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 46Vj0L2qYlz4ZJ2 X-Spamd-Bar: / X-Spamd-Result: default: False [0.65 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; FAKE_REPLY(1.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-0.08)[-0.075,0]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(0.00)[ip: (3.46), ipnet: 74.6.128.0/21(1.45), asn: 26101(1.16), country: US(-0.05)]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.23)[0.230,0]; RCVD_IN_DNSWL_NONE(0.00)[40.131.6.74.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Sep 2019 06:14:59 -0000 Don Lewis truckman at FreeBSD.org wrote on Thu Sep 12 23:00:19 UTC 2019 : . . . > Nevertheless, I see these errors, > with rustc being the usual victim: >=20 > Sep 11 23:21:43 zipper kernel: pid 16581 (rustc), jid 43, uid 65534, = was killed: out of swap space > Sep 12 02:48:23 zipper kernel: pid 1209 (rustc), jid 62, uid 65534, = was killed: out of swap space . . . Unfortunately, the wording of this type of message is a misnomer for what drives the kills: it is actually driven by being unable to gain more free memory but FreeBSD will not swap-out processes that stay runnable (or are running), only ones that are waiting. Even a single process that stays runnable and keeps lots of RAM in the active category can lead to kills when swap is unused or little used. So the kill-behavior is very workload dependent. Real "out of swap" conditions tend to also have messages similar to: Aug 5 17:54:01 sentinel kernel: swap_pager_getswapspace(32): failed If you are not seeing such messages, then it is likely that the mount of swap space still free is not the actual thing driving the kills. Are you seeing "swap_pager_getswapspace(32): failed" messages? (It used to be that the system simply leaves the dirty pages in memory when a swap_pager_getswapspace failed message is produced. Of itself, it did not cause a kill. I do not know about now.) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-current@freebsd.org Sat Sep 14 16:10:19 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E906BF8A48; Sat, 14 Sep 2019 16:10:19 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (www.zefox.net [50.1.20.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "www.zefox.org", Issuer "www.zefox.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46VyCG4jnSz48ck; Sat, 14 Sep 2019 16:10:18 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (localhost [127.0.0.1]) by www.zefox.net (8.15.2/8.15.2) with ESMTPS id x8EGAFWq033684 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 14 Sep 2019 09:10:16 -0700 (PDT) (envelope-from fbsd@www.zefox.net) Received: (from fbsd@localhost) by www.zefox.net (8.15.2/8.15.2/Submit) id x8EGAFoo033683; Sat, 14 Sep 2019 09:10:15 -0700 (PDT) (envelope-from fbsd) Date: Sat, 14 Sep 2019 09:10:14 -0700 From: bob prohaska To: Mark Millard Cc: FreeBSD Current , freebsd-arm@freebsd.org Subject: Re: spurious out of swap kills Message-ID: <20190914161014.GA33442@www.zefox.net> References: <28BF21DA-8B16-4CD8-8E5E-C1B596FE3684@yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <28BF21DA-8B16-4CD8-8E5E-C1B596FE3684@yahoo.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Rspamd-Queue-Id: 46VyCG4jnSz48ck X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of fbsd@www.zefox.net has no SPF policy when checking 50.1.20.27) smtp.mailfrom=fbsd@www.zefox.net X-Spamd-Result: default: False [2.01 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.15)[-0.150,0]; WWW_DOT_DOMAIN(0.50)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; IP_SCORE(0.08)[ip: (0.34), ipnet: 50.1.16.0/20(0.17), asn: 7065(-0.05), country: US(-0.05)]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[zefox.net]; AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.18)[0.178,0]; R_SPF_NA(0.00)[]; FREEMAIL_TO(0.00)[yahoo.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7065, ipnet:50.1.16.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; MID_RHS_WWW(0.50)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Sep 2019 16:10:20 -0000 On Fri, Sep 13, 2019 at 10:59:58PM -0700, Mark Millard wrote: > bob prohaska fbsd at www.zefox.net wrote on > Fri Sep 13 16:24:57 UTC 2019 : > > > Not sure this is relevant, but in compiling chromium on a Pi3 with 6 GB > > of swap the job completed successfully some months ago, with peak swap > > use around 3.5 GB. The swap layout was sub-optimal, with a 2 GB partition > > combined with a 4 GB partition. A little over 4GB total seems usable. > > > > A few days ago the same attempt stopped with a series of OOMA kills, > > but in each case simply restarting allowed the compile to pick up > > where it left off and continue, eventually finishing with a runnable > > version of chromium. In this case swap use peaked a little over 4 GB. > > > > Might this suggest the machine isn't freeing swap in a timely manner? > > Are you saying that your increases to: > > vm.pageout_oom_seq > > no longer prove sufficient? What value for vm.pageout_oom_seq were > you using that got the recent failures? > Correct. Initial value was 2048, later raised to 4096. Far as I could tell the change didn't help. No explict j value was set for make, but no more than four jobs were observed in top A log of storage activity along with swap total and the last two console messages is at http://www.zefox.net/~fbsd/rpi3/swaptests/r351586/swapscript.log along with a sorted list of total swap use, which can be used as a sort of index to the log file. The initial "out of swap space" at the very beginning is a relic from before logging started. Da0 is a Sandisk SDCZ80 usb 3.0 device, mmcsd0 is a Samsung Evo + 128 GB device. The two points of curiosity to me are: 1. Why did swap use increase from 3.5 GB months ago to 4.2 GB now? 2. Why does stopping and restarting make (which would seem to free un-needed swap) allow the job to finish? > If more or different configuration/tuning is required, I'm going to > eventually want to learn about it as well. > You will have some company. Thanks for reading, bob prohaska From owner-freebsd-current@freebsd.org Sat Sep 14 17:38:17 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 49DC3FA65C for ; Sat, 14 Sep 2019 17:38:17 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46W08m3jzNz4DrN for ; Sat, 14 Sep 2019 17:38:16 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from pi by home.opsec.eu with local (Exim 4.92.2 (FreeBSD)) (envelope-from ) id 1i9Bzp-000MoT-ID for freebsd-current@freebsd.org; Sat, 14 Sep 2019 19:38:05 +0200 Date: Sat, 14 Sep 2019 19:38:05 +0200 From: Kurt Jaeger To: FreeBSD Current Subject: poudriere, swap full and top says memory is free ? Message-ID: <20190914173805.GC2863@home.opsec.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Rspamd-Queue-Id: 46W08m3jzNz4DrN X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of lists@opsec.eu designates 2001:14f8:200::1 as permitted sender) smtp.mailfrom=lists@opsec.eu X-Spamd-Result: default: False [-4.94 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[opsec.eu]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(-3.64)[ip: (-9.62), ipnet: 2001:14f8::/32(-4.77), asn: 12502(-3.82), country: DE(-0.01)]; TO_DN_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:12502, ipnet:2001:14f8::/32, country:DE]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Sep 2019 17:38:17 -0000 Hi! I'm running - a poudriere build - of a list of ports - on 12.0-RELEASE-p10 - on a 4 core+4 hyperthreads CPU, an Intel(R) Xeon(R) CPU E3-1230 v6 @ 3.50GHz - with 32 GB RAM - zpool with 2x 500 GB SSDs as a mirror and right now, this can be seen: last pid: 90922; load averages: 5.02, 5.14, 5.73 up 0+03:53:08 19:31:05 82 processes: 6 running, 76 sleeping CPU: 60.6% user, 0.0% nice, 2.1% system, 0.0% interrupt, 37.3% idle Mem: 4598M Active, 2854M Inact, 11G Laundry, 6409M Wired, 6375M Free ARC: 3850M Total, 1721M MFU, 2090M MRU, 665K Anon, 19M Header, 19M Other 3406M Compressed, 3942M Uncompressed, 1.16:1 Ratio Swap: 18G Total, 18G Used, 396K Free, 99% Inuse, 68K In So: Swap is full, approx. 6 GB memory is reported as free. This is surprising. Can I somehow tune this in any way, so that the memory available is used for the build ? Or is the problem somewhere else ? Running similar builds on 12.0 without patches reported swap_pager_getswapspace(24): failed messages. -- pi@opsec.eu +49 171 3101372 One year to go ! From owner-freebsd-current@freebsd.org Sat Sep 14 18:05:13 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CDB8BFAECA for ; Sat, 14 Sep 2019 18:05:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic307-9.consmr.mail.ne1.yahoo.com (sonic307-9.consmr.mail.ne1.yahoo.com [66.163.190.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46W0lr3q1Bz4G8H for ; Sat, 14 Sep 2019 18:05:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: U7vAjOkVM1nitrfSD_zwoorP53o4qBG7qg2dCG3c0SY5xRf_GW66FMnL0QgKPJ2 pP4OPj2QlpnuZrFfE_2aETCw22J_ekkSfgqF8dY6Y2SY_DZD1eq708adQERxawvMdATryDYL6aUQ ZDM8K21WCYzffPjdqRUdwEd9OvYo0f5gT0pWEIiF7T7wBzvTUYfzJpSnGWyYvYypa3QGwNe_ax3j P3cZnhpFc3k2p3.ip0xAbXqksMw6sBdsxWYDw5Cp4QMBUj.iNYFtdQbiF4cxumhzsx7YvYSce3N8 s.Bw4pWBdTtduIAqBhFYD9YyWSwjI9ctXLT0WlmtcmXN8KP_UTUl1wwsWT9iuX.0vpi_8CU7ceK8 78haZdNOfQFyVx83Rod7YVShCYHfZ0SU5eOVIaUbviOVDABNHnBAajXmKZhim70N3moW4ASMp998 RTcy6nad2dtG1fK4V2dTEkBqXIGIvMEw.C5B6f2Zfql1eLAb5OPqKImtQ5yP6.sXliJShf232tBy dI2mqgwEatnHtqQfy3nNdlBpMNstdsZNPEddXx7Jw3jXoi0eEXK1aD3H7lvvq00udlPyNEN15Zdg 4hadZu7qog5haqFNeiG7RrL7sqk_dlcsSvoi5MXbSZVNbbkQwm33d0Yv5eGi6Hd3bRdC5XVw.aja ZokYlc2EtoNBR8WaEdW0Oan6tH4nu1jb0Lr7ixkEw8fKugZjaD9RmyfEemZfFi8Yf0j8Ni5rtYoI 9KeKpcWlV2xcq3hAeEqM5gbut9nkVsbupNlR8lJLofEQmpCLw6jCz5UkoSSQuCvao5730b.RYtGx 9MBcliC.qTe8TKwL8OI4UAUi5SqwyrdodnECPE6xgW8LdZIazfj.4zYvJfboCKhhDHGPAergKo5Y tkmhhb90IkEuNFWsOJtko1cxEl1x9t_On5vFfucOjWhjjzhTDiSdYZPlxpvm8xCVVAsUww68yGMm WFmyyqBJVafmubYz.Ir_HtCCCUeJXTQk5ConK9TCp1DFgVN2kFltCUo1lZDq53l.fZFmX44R5.PT L_TAbrrrvFGwuQ69nl_4dfPngGTDZ69RLH_JS2tGTyDx7_NMaKagPh1AMhetkUC.EQyWrZ_8bi1i l._yVjVRRdLbxYqsVPP99RNOKth4eSk2PbJGtw4Nvxx0m_Ilan0ZPbBNs7Ig2mTfAk0IqrBDfIh7 HiJq.4NNiJXKrN_hQBF59oMvSG8A_suDdsY5uh9E.IXyV8y5lwrluNWLqIWrdgsBH15vhU..Iwms Wq4mR_nCRw397AvTNppkZPVDbO9Cmr6DGPcRIenN4NUu5B07I8L8PWaaV0g-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.ne1.yahoo.com with HTTP; Sat, 14 Sep 2019 18:05:10 +0000 Received: by smtp420.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 13b11ce7938a56cd8c0fc711236cd56c; Sat, 14 Sep 2019 18:05:10 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: head -r352274 buildkernel targetting armv7 failure: am335x/am335x_dmtpps.c:304:3: error: implicit declaration of function 'spinlock_enter' is invalid in C99 [-Werror,-Wimplicit-function-declaration] Message-Id: <0E884723-2826-4EDF-A16F-841E01E9D4EC@yahoo.com> Date: Sat, 14 Sep 2019 11:05:08 -0700 To: FreeBSD Current , freebsd-arm@freebsd.org X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 46W0lr3q1Bz4G8H X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.29 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.88)[-0.879,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_GOOD(-0.10)[text/plain]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-0.91)[-0.914,0]; IP_SCORE_FREEMAIL(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[32.190.163.66.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.00)[ip: (4.06), ipnet: 66.163.184.0/21(1.30), asn: 36646(1.04), country: US(-0.05)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Sep 2019 18:05:13 -0000 After updating my amd64 context to head -r352274, attempting an amd64->armv7 cross buildworld buildkernel ended up failing with: --- am335x_dmtpps.o --- /usr/src/sys/arm/ti/am335x/am335x_dmtpps.c:304:3: error: implicit = declaration of function 'spinlock_enter' is invalid in C99 = [-Werror,-Wimplicit-function-declaration] mtx_lock_spin(&sc->pps_mtx); ^ /usr/src/sys/sys/mutex.h:383:26: note: expanded from macro = 'mtx_lock_spin' #define mtx_lock_spin(m) mtx_lock_spin_flags((m), 0) ^ /usr/src/sys/sys/mutex.h:452:2: note: expanded from macro = 'mtx_lock_spin_flags' mtx_lock_spin_flags_((m), (opts), LOCK_FILE, LOCK_LINE) ^ /usr/src/sys/sys/mutex.h:429:2: note: expanded from macro = 'mtx_lock_spin_flags_' __mtx_lock_spin((m), curthread, (opts), (file), (line)) ^ /usr/src/sys/sys/mutex.h:258:2: note: expanded from macro = '__mtx_lock_spin' spinlock_enter(); = \ ^ /usr/src/sys/arm/ti/am335x/am335x_dmtpps.c:304:3: error: this function = declaration is not a prototype [-Werror,-Wstrict-prototypes] /usr/src/sys/sys/mutex.h:383:26: note: expanded from macro = 'mtx_lock_spin' #define mtx_lock_spin(m) mtx_lock_spin_flags((m), 0) ^ /usr/src/sys/sys/mutex.h:452:2: note: expanded from macro = 'mtx_lock_spin_flags' mtx_lock_spin_flags_((m), (opts), LOCK_FILE, LOCK_LINE) ^ /usr/src/sys/sys/mutex.h:429:2: note: expanded from macro = 'mtx_lock_spin_flags_' __mtx_lock_spin((m), curthread, (opts), (file), (line)) ^ /usr/src/sys/sys/mutex.h:258:2: note: expanded from macro = '__mtx_lock_spin' spinlock_enter(); = \ ^ . . . (spinlock_enter was not the only example.) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-current@freebsd.org Sat Sep 14 18:21:24 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E7A18FB6F2 for ; Sat, 14 Sep 2019 18:21:24 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound3d.ore.mailhop.org (outbound3d.ore.mailhop.org [54.186.57.195]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46W16X4Ljkz4H5p for ; Sat, 14 Sep 2019 18:21:24 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1568485283; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=klFW3yvkPpzOyHUmvbCfgUzkP8BW6GdjNZLdF+Cnmlw6aSj0ZMBID9EjKLmIgPBzbyzaYYTCBUDSS tbdISjf0O3PkpmF++jEUxJpNKcbGeyIVa7t03ZS+j9MzSuWznU1zrKr1eY1N+cFkJBWMb5ZIjY9YxN wZ944662r4FF+tGUHQmLOZ6VikI+4TJUsUJfN4tvm2k7q4uuypEQScMiGWKgg8zQjcUr8D9wCiUTV/ 9JEpNnTmlkfmNxBuHZgJyJ9pP60xcFU6aa3LsjoL8U/EH0lhMicOfR5uQAEBpOZYNh+erqUbiFwQWv 16sUqAI7VczLf5nOR1RrroR6NEVqBig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:dkim-signature:from; bh=WrGudQQQp/k6tQy/rIiXJ/OP84sFSGnd9aCvHMicsV0=; b=IV26p3+t6NoU1EGtppKkQhASR9yigUwqrQckijUDHCOR4rThfml62Tp9EtAXrrKG+eEJ4jcOsMvmr fnSF+i6RrspVSw/juAqVrosb/B445YIf1TAz3yrysQrWCMTmCRRc85+W6ybKrhGPWUhxttC6LuIpzV smPawJP8jQyKK+2VBzAuRis+ce5JNSnOlnAJGbd9XpnQ0DZFO+OXzkBgwLL/pu2EwcQiwHlVVsDCxK b91xovBajrC5LgbLtPB338pJ119BYYIbjnPvhhkUjfCc8lSdYTk0Ueb0jCWhMdqrpDbVjimu4BjjVF ZVqdRkI9Pe478pB+saV8VGSpatYZN5g== ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:from; bh=WrGudQQQp/k6tQy/rIiXJ/OP84sFSGnd9aCvHMicsV0=; b=UAenpQBp5VLn9MiUk04UsW9iUc3A1O+/f6Sbx88h7l9nob9BTFg+uA+ZiqEfimCRkg9rrEi6ang9I qmqG09c1eRnWhQRCx7cvKXg8n0julWCm1+T1Sd5BVrkR4w6xcrW81tSWZfS6IF4if1gT5wccUU09Cq T2SKD68xeNJEiQR3SrKh61mEWrQIYmNdct6/XJHOypcQKrOMCxmtrxEM+7Y3DetAjEhUFbX789LTqm BlMa1af8Bj7EqzMNYbmquuCXLvrplplcolA/LZV/yZladOc0AAawarV5S8T1i2Ja6ZZR0by79XxceE mcTDQPQVH6QbNBg8TywD9EmTwrWo4Pg== X-MHO-RoutePath: aGlwcGll X-MHO-User: 734be60b-d71c-11e9-b67d-cdd75d6ce7a8 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound3.ore.mailhop.org (Halon) with ESMTPSA id 734be60b-d71c-11e9-b67d-cdd75d6ce7a8; Sat, 14 Sep 2019 18:21:21 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x8EILK7m087044; Sat, 14 Sep 2019 12:21:20 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: Subject: Re: head -r352274 buildkernel targetting armv7 failure: am335x/am335x_dmtpps.c:304:3: error: implicit declaration of function 'spinlock_enter' is invalid in C99 [-Werror,-Wimplicit-function-declaration] From: Ian Lepore To: Mark Millard , FreeBSD Current , freebsd-arm@freebsd.org Date: Sat, 14 Sep 2019 12:21:20 -0600 In-Reply-To: <0E884723-2826-4EDF-A16F-841E01E9D4EC@yahoo.com> References: <0E884723-2826-4EDF-A16F-841E01E9D4EC@yahoo.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46W16X4Ljkz4H5p X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-2.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; ASN(0.00)[asn:16509, ipnet:54.186.0.0/15, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Sep 2019 18:21:25 -0000 On Sat, 2019-09-14 at 11:05 -0700, Mark Millard via freebsd-arm wrote: > After updating my amd64 context to head -r352274, > attempting an amd64->armv7 cross buildworld buildkernel > ended up failing with: > > > --- am335x_dmtpps.o --- > /usr/src/sys/arm/ti/am335x/am335x_dmtpps.c:304:3: error: implicit > declaration of function 'spinlock_enter' is invalid in C99 [-Werror,- > Wimplicit-function-declaration] > mtx_lock_spin(&sc->pps_mtx); > ^ > /usr/src/sys/sys/mutex.h:383:26: note: expanded from macro > 'mtx_lock_spin' > #define mtx_lock_spin(m) mtx_lock_spin_flags((m), 0) > ^ > /usr/src/sys/sys/mutex.h:452:2: note: expanded from macro > 'mtx_lock_spin_flags' > mtx_lock_spin_flags_((m), (opts), LOCK_FILE, LOCK_LINE) > ^ > /usr/src/sys/sys/mutex.h:429:2: note: expanded from macro > 'mtx_lock_spin_flags_' > __mtx_lock_spin((m), curthread, (opts), (file), (line)) > ^ > /usr/src/sys/sys/mutex.h:258:2: note: expanded from macro > '__mtx_lock_spin' > spinlock_enter(); > \ > ^ > /usr/src/sys/arm/ti/am335x/am335x_dmtpps.c:304:3: error: this > function declaration is not a prototype [-Werror,-Wstrict-prototypes] > /usr/src/sys/sys/mutex.h:383:26: note: expanded from macro > 'mtx_lock_spin' > #define mtx_lock_spin(m) mtx_lock_spin_flags((m), 0) > ^ > /usr/src/sys/sys/mutex.h:452:2: note: expanded from macro > 'mtx_lock_spin_flags' > mtx_lock_spin_flags_((m), (opts), LOCK_FILE, LOCK_LINE) > ^ > /usr/src/sys/sys/mutex.h:429:2: note: expanded from macro > 'mtx_lock_spin_flags_' > __mtx_lock_spin((m), curthread, (opts), (file), (line)) > ^ > /usr/src/sys/sys/mutex.h:258:2: note: expanded from macro > '__mtx_lock_spin' > spinlock_enter(); > \ > ^ > . . . > > (spinlock_enter was not the only example.) > > My bad, I forgot to include when I switched the code to spinlocks. Should be fixed by r352333. -- Ian From owner-freebsd-current@freebsd.org Sat Sep 14 18:29:33 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E6A92FBA77 for ; Sat, 14 Sep 2019 18:29:33 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46W1Hv5R0Hz4HTT for ; Sat, 14 Sep 2019 18:29:31 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id x8EISvjM089596 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 14 Sep 2019 11:28:57 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id x8EISvu9089595; Sat, 14 Sep 2019 11:28:57 -0700 (PDT) (envelope-from jmg) Date: Sat, 14 Sep 2019 11:28:57 -0700 From: John-Mark Gurney To: Kurt Jaeger Cc: FreeBSD Current Subject: Re: poudriere, swap full and top says memory is free ? Message-ID: <20190914182857.GM96402@funkthat.com> Mail-Followup-To: Kurt Jaeger , FreeBSD Current References: <20190914173805.GC2863@home.opsec.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190914173805.GC2863@home.opsec.eu> X-Operating-System: FreeBSD 11.0-RELEASE-p7 amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Sat, 14 Sep 2019 11:28:57 -0700 (PDT) X-Rspamd-Queue-Id: 46W1Hv5R0Hz4HTT X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [-0.42 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.95)[-0.954,0]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_TLS_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-0.93)[-0.927,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; IP_SCORE(-0.74)[ip: (-1.91), ipnet: 208.87.216.0/21(-0.95), asn: 32354(-0.76), country: US(-0.05)]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Sep 2019 18:29:34 -0000 Kurt Jaeger wrote this message on Sat, Sep 14, 2019 at 19:38 +0200: > - a poudriere build > - of a list of ports > - on 12.0-RELEASE-p10 > - on a 4 core+4 hyperthreads CPU, an Intel(R) Xeon(R) CPU E3-1230 v6 > @ 3.50GHz > - with 32 GB RAM > - zpool with 2x 500 GB SSDs as a mirror > > and right now, this can be seen: > > last pid: 90922; load averages: 5.02, 5.14, 5.73 up 0+03:53:08 19:31:05 > 82 processes: 6 running, 76 sleeping > CPU: 60.6% user, 0.0% nice, 2.1% system, 0.0% interrupt, 37.3% idle > Mem: 4598M Active, 2854M Inact, 11G Laundry, 6409M Wired, 6375M Free > ARC: 3850M Total, 1721M MFU, 2090M MRU, 665K Anon, 19M Header, 19M Other > 3406M Compressed, 3942M Uncompressed, 1.16:1 Ratio > Swap: 18G Total, 18G Used, 396K Free, 99% Inuse, 68K In > > So: Swap is full, approx. 6 GB memory is reported as free. > > This is surprising. Can I somehow tune this in any way, so that > the memory available is used for the build ? Or is the problem somewhere > else ? Are you sure that this hasn't just recently completed a large link of something like Chromium? There are known to be compiles that can take many GB's of memory and if they recently exited, there hasn't been time to swap stuff back in... or is this the steady state over the entire compile? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-current@freebsd.org Sat Sep 14 18:49:39 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 56BA3FC18C for ; Sat, 14 Sep 2019 18:49:39 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-2.consmr.mail.bf2.yahoo.com (sonic305-2.consmr.mail.bf2.yahoo.com [74.6.133.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46W1l61yVHz4JRp for ; Sat, 14 Sep 2019 18:49:37 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: 5BsV46gVM1mzhZrBvKAkAQtJBUEFf7moNfs4uQSplvOEX2qjHmekKGLqJoEhh7X _CGj0.rznU4KFChxrVfulN_EszgqH7auJfdxZ483udQUu8Efma9VRV0PWInDaXz_tGVFAKjG.LmA eXZG3mhABgopWocV8t_iDOc_S46a2.o0v2vsVJvD1UUh69qKSHj_Rke07VRqPth5BQpjGRUjSHO2 6se1AfFWyn9Js1LGtO1YMELiDzRL96GNV8ii3SQt7U.ZFIGIJZgz_7Y2ow8pqWjhkfEayps.0k3. 5P8.3CLQ3D44L6tmh1aTq.5R_Pshpam.oXCktQ9PVA1KAx5sAymuxFVlubivhn_HCcGHozP6IWI. oumuHX6sxp7bAHfGTNnzDYXlzBGM2IJAZ3DiUPPP8_82lW7j.CCD8mV8cFTp0TR2fbslKZKOeA.J 1BtRNuXQims1pvVRaLnw2DzH4M9oRFHltZ5sG7v0UgbKFckVO58S7AigJd1bs50HK4VbnFMvrBgq IcGug_aQtH6V8joiRi9nmVGHcy2LPDsRCgCKzVQLlGgrcW0h8otmyO11Un6RHfWBMvpKCBsiECes L2AkLlRWeROQSU5sim6Y3pluDbxVJhZFDssf83lPcH5olwNqcyxsvTcFMbLfnQ6O362jJEyS.zI. uS.B3z3VQeQaXdQWsoWuB9Mi1GJDPZ1NkNAWwbMFXbl.Nz3XXTeA7Mw6qBfJztIcfkYDUfnsaj2X O.S5Mlrt8g4W_ZxTNVv5LT.RLzAGUEh.3gXjvWiqfTvAvyyyrh1k2str69j7AVV9wJtYPyQ.bcZd 50d2Li1zC1cxvTKoEIb6u25tcLt3L6rWEkyb7UtFI6O0MHhLc_DsdnNGgJlUEGvmhDojdtbUA.ot _noXTjk6mHgPpoSPcg.2ujj3CkogApCXKdFpNbtjl0WvSFs.bJ8PCqlxf_39Civ1b0RwDnr990L1 21OBT5mL3_rxP06rt832hSu.jHek4ifhAI3ih2QF4eFXlOuRCAGPRWg60Cw0okxpUWSNwgR8mwDu Q.qdfTcJJBI4WCSsunX.lNYmIYIcd9nVeH0esMlXwfA1Q05WHXtokTkNNbSqNpX_gkFeZJ_S9reI .APNnmiF753XTdcU2IuFOVJx_JrJkpRvpaJb5YmfGuByE1bcOKzniQEyV2JZvKCL84nLfys8yHjD uKZ0DHzqyLv3tpkX8ntVH6H.WNDn7aA7Op83JjYgGRD4Nxc9_ZeSGApOZscYGsUNH6.GeVw2QFzz gpeUFtnRy8emVisrsTbV9bs1KtpZxTaZ5sR4SIPXnpQcZpm74T07tj42QLIXVo4rc Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.bf2.yahoo.com with HTTP; Sat, 14 Sep 2019 18:49:35 +0000 Received: by smtp410.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 4844f306538fab6cdf33a9de820ec6e0; Sat, 14 Sep 2019 18:49:33 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: head -r352274 buildkernel targetting armv7 failure: am335x/am335x_dmtpps.c:304:3: error: implicit declaration of function 'spinlock_enter' is invalid in C99 [-Werror,-Wimplicit-function-declaration] From: Mark Millard In-Reply-To: Date: Sat, 14 Sep 2019 11:49:31 -0700 Cc: FreeBSD Current , freebsd-arm@freebsd.org Content-Transfer-Encoding: 7bit Message-Id: <2F803AD5-900F-44F3-99E9-89082D604CA8@yahoo.com> References: <0E884723-2826-4EDF-A16F-841E01E9D4EC@yahoo.com> To: Ian Lepore X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 46W1l61yVHz4JRp X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.31 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.86)[-0.864,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_GOOD(-0.10)[text/plain]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-0.95)[-0.947,0]; TO_DN_SOME(0.00)[]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; RCVD_IN_DNSWL_NONE(0.00)[41.133.6.74.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.00)[ip: (3.76), ipnet: 74.6.128.0/21(1.45), asn: 26101(1.16), country: US(-0.05)]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Sep 2019 18:49:39 -0000 On 2019-Sep-14, at 11:21, Ian Lepore wrote: > On Sat, 2019-09-14 at 11:05 -0700, Mark Millard via freebsd-arm wrote: >> After updating my amd64 context to head -r352274, >> attempting an amd64->armv7 cross buildworld buildkernel >> ended up failing with: >> >> >> --- am335x_dmtpps.o --- >> /usr/src/sys/arm/ti/am335x/am335x_dmtpps.c:304:3: error: implicit >> declaration of function 'spinlock_enter' is invalid in C99 [-Werror,- >> Wimplicit-function-declaration] >> mtx_lock_spin(&sc->pps_mtx); >> ^ >> (...shortened...) >> . . . >> >> (spinlock_enter was not the only example.) >> >> > > My bad, I forgot to include when I switched the code to > spinlocks. Should be fixed by r352333. Thanks. It is interesting that: https://ci.freebsd.org/job/FreeBSD-head-armv7-build/6042/ shows a successful build of -r352274 (the last before -r352275 broke both arm and aarch64). Prior builds also were successful. I'm manually applied your update to -r352274 and am rebuilding from scratch. === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From owner-freebsd-current@freebsd.org Sat Sep 14 19:41:22 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C1370FD5E4 for ; Sat, 14 Sep 2019 19:41:22 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46W2tn5Lpfz4MjZ for ; Sat, 14 Sep 2019 19:41:21 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from pi by home.opsec.eu with local (Exim 4.92.2 (FreeBSD)) (envelope-from ) id 1i9Dv3-000NDk-Mp for freebsd-current@freebsd.org; Sat, 14 Sep 2019 21:41:17 +0200 Date: Sat, 14 Sep 2019 21:41:17 +0200 From: Kurt Jaeger To: FreeBSD Current Subject: Re: poudriere, swap full and top says memory is free ? Message-ID: <20190914194117.GD2863@home.opsec.eu> References: <20190914173805.GC2863@home.opsec.eu> <20190914182857.GM96402@funkthat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190914182857.GM96402@funkthat.com> X-Rspamd-Queue-Id: 46W2tn5Lpfz4MjZ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of lists@opsec.eu designates 2001:14f8:200::1 as permitted sender) smtp.mailfrom=lists@opsec.eu X-Spamd-Result: default: False [-4.90 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[opsec.eu]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; SUBJECT_ENDS_QUESTION(1.00)[]; TO_DN_ALL(0.00)[]; IP_SCORE(-3.60)[ip: (-9.48), ipnet: 2001:14f8::/32(-4.73), asn: 12502(-3.79), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:12502, ipnet:2001:14f8::/32, country:DE]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Sep 2019 19:41:22 -0000 Hi! > > Mem: 4598M Active, 2854M Inact, 11G Laundry, 6409M Wired, 6375M Free > > ARC: 3850M Total, 1721M MFU, 2090M MRU, 665K Anon, 19M Header, 19M Other > > 3406M Compressed, 3942M Uncompressed, 1.16:1 Ratio > > Swap: 18G Total, 18G Used, 396K Free, 99% Inuse, 68K In > > > > So: Swap is full, approx. 6 GB memory is reported as free. > > This is surprising. Can I somehow tune this in any way, so that > > the memory available is used for the build ? Or is the problem somewhere > > else ? > > Are you sure that this hasn't just recently completed a large link of > something like Chromium? Yes, because I plot memory/swap/etc using nagios. It's not only a spike. > There are known to be compiles that can take > many GB's of memory and if they recently exited, there hasn't been time > to swap stuff back in... or is this the steady state over the entire > compile? Building a few ports (firefox, libreoffice etc) takes some time, so it has been stable during that phase. -- pi@opsec.eu +49 171 3101372 One year to go ! From owner-freebsd-current@freebsd.org Sat Sep 14 23:56:35 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D0484D2F9C for ; Sat, 14 Sep 2019 23:56:35 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-22.consmr.mail.gq1.yahoo.com (sonic310-22.consmr.mail.gq1.yahoo.com [98.137.69.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46W8YH0FL4z4d08 for ; Sat, 14 Sep 2019 23:56:34 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: Pk7Lg44VM1mhl5nqPWTt24AQrCpWJbiabHpqarYh2tP6kwDS0Kbl7NPQ3F2g_83 buQL674.ZLsE3aVxIkVpf3Hbh8thPwlB0crzaKK6Acq0FW9OKZku4PASJAzY5D7b9rW82rSfdp6Z r003hYSvJf0yTJvR3KtPZog5WvhvNDoZ8lPl3I1PiUNCNkUNbVhzSCEvpZxRAArDAc7GcdvyrE04 HUzLpC6McaZ7RUxnR52XdUgbW4kpoMKZv2cipE7iigXXbA2PhmIx0LpAqNJGIusFLqc8SibRwaOj UKhXlTpa9fvE0vfEVf9EkTEgXgum_i3b_Ffne_tbNoyIyd0XL7dfxNPhHwFzKdHcf9HUsT3.znax bZB3f1ZpIg18nJ2PUuMwQP57OYVOwQUXiMF6VgTN8dRNEGONvaLfUxRSb6mKK9_rplvoicnYs.y0 HEqJmQodpSQ2jaUmh5iG6RKQGBJtKZV7vnaSXe1YhTrkrhOW8MUNCILuF30g0Y2C0vaiDRervXm. j_co3UStpvRz7FCeI.VpagI5FGIEu13Qvc5A539C6Vz9JtWkHjR9QK5PsiqXJcTudG.6sRXS6fvf aeaLl5pFPQRPUQ7yuEl7f2SCrCpn2kkPcbUfMEjbB7Wz42YPkXc6dMd8jVDUxHeOlPWosLCvuwEZ 9BaP9Hef_ZytbG64ZUWUfJUoX_SwGFaOcgoXATjAL37LwDiI13To3HT9cEvzeae4jlUZ3NSzVhId sBxffEIyJKH59B9Y.3EzfQd8wpjOqRgNQFGRWocSyGWeIVMeszXGvSE9d.14TFv0B0.gYRiDd2sI YNw5j3v8.YIZNL_M2cFNpjk33u5IfRXQsk6_Xx8S4Gskt4tFKkj_5kCYFHqcQg4Jc4U.3WaufBE4 MoZFBjLwhkjpX0dfCCU8Vs93Cndqb_B0qQurQl5PE.ZziL7wKoy_TjgXvambwjhxSdjJwsYWoWF2 _LZEqmbiNSRDZQl0cUetJReGXTTPT4yNLpawurAA571Xyq80sEfWEhlOs_SfBAoJxg8cRdWe6wZM UETvUqcwbGtKDgbLPcCAWpUXyJCMvpEaSkCHsz4tAduq1nehT0AL6oJHJLLGmr91V8_Gn2IqaHhW nOUr46onZQciFTvRfKSghDacJ0ivQ1OVsOnkupY0xbco0kcXAniFjpvv8Zz63.g7pALAIhk_4ewj ZZjk_PU3o_qoY0oD2dg3.tNV_q2j0fq0uEwgIizaiZj_yg.f_SysMKYRWUhtPt90kM5aYrRBd9VF TygXb70O0e.MlZjXBZ9tDHu8JSEGn5zbJQhto5UJHJjV_LgCk25CoD84XplnuRLEmU08myf4t._P viWtftmDZeRQA.w8XoYZuAfIwiP2AKBZM.9A5EQ-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Sat, 14 Sep 2019 23:56:32 +0000 Received: by smtp401.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID b0136dac9959d9f52e5dda39ac632019; Sat, 14 Sep 2019 23:56:28 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: spurious out of swap kills Message-Id: <8269B1A1-48DE-4997-B1DD-0E1F69EC9A52@yahoo.com> Date: Sat, 14 Sep 2019 16:56:28 -0700 To: "truckman@freebsd.org" , FreeBSD Current X-Mailer: Apple Mail (2.3445.104.11) References: <8269B1A1-48DE-4997-B1DD-0E1F69EC9A52.ref@yahoo.com> X-Rspamd-Queue-Id: 46W8YH0FL4z4d08 X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.47 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.979,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.996,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(0.00)[ip: (1.22), ipnet: 98.137.64.0/21(0.93), asn: 36647(0.74), country: US(-0.05)]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[148.69.137.98.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Sep 2019 23:56:35 -0000 Konstantin Belousov kostikbel at gmail.com wrote on Fri Sep 13 05:53:41 UTC 2019 : > Basically, page fault handler waits for vm.pfault_oom_wait * > vm.pfault_oom_attempts for a page allocation before killing the = process. > Default is 30 secs, and if you cannot get a page for 30 secs, there is > something very wrong with the machine. The following was not for something like a Ryzen, but for an armv7 board using a USB device for the file system and swap/paging partition. Still it may be a suggestive example of writing out a large amount of laundry. There was an exchange I had with Warner L. that implied easily having long waits in the queue when trying to write out the laundry (or other such) in low end contexts. I extract some of it below. dT: 1.006s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps = ms/d %busy Name 56 312 0 0 0.0 312 19985 142.6 0 0 = 0.0 99.6| da0 Note: L(q) could be a lot bigger than 56 but I work with the example figures that I used at the time and that Warner commented on. The 142.6 ms/w includes time waiting in the queue and was vastly more stable than the L(q) figures. Warner wrote, in part: QUOTE 142.6ms/write is the average of the time that the operations that = completed during the polling interval took to complete. There's no = estimating here. So, at 6 or 7 per second for the operation to complete, coupled with a = parallel factor of 1 (typical for low end junk flash), we wind up with = 56 operations in the queue taking 8-10s to complete. END QUOTE Things went on from there but part of it was based on a reporting patch that Mark Johnston had provided. Me: It appears to me that, compared to a observed capacity of roughly around 20 MiBytes/sec for writes, large amounts of bytes are being queued up to be written in a short time, for which it just takes a while for the backlog to be finished. Warner: Yes. That matches my expectation as well. In other devices, I've found = that I needed to rate-limit things to more like 50-75% of the max value = to keep variance in performance low. It's the whole reason I wrote the = CAM I/O scheduler. Me: The following is from multiple such runs, several manually stopped but some killed because of sustained low free memory. I had left vm.pageout_oom_seq=3D12 in place for this, making the kills easier to get than the 120 figure would. It does not take very long generally for some sort of message to show up. (Added Note: 65s and 39s were at the large end of what I reported at the time.) . . . swap_pager: indefinite wait buffer: bufobj: 0, blkno: 164064, size: = 12288 waited 65s for async swap write waited 65s for swap buffer waited 65s for async swap write waited 65s for async swap write waited 65s for async swap write v_free_count: 955, v_inactive_count: 1 Aug 20 06:11:49 pine64 kernel: pid 1047 (stress), uid 0, was killed: out = of swap space waited 5s for async swap write waited 5s for swap buffer waited 5s for async swap write waited 5s for async swap write waited 5s for async swap write swap_pager: indefinite wait buffer: bufobj: 0, blkno: 314021, size: = 12288 swap_pager: indefinite wait buffer: bufobj: 0, blkno: 314084, size: = 32768 swap_pager: indefinite wait buffer: bufobj: 0, blkno: 314856, size: = 32768 swap_pager: indefinite wait buffer: bufobj: 0, blkno: 314638, size: = 131072 swap_pager: indefinite wait buffer: bufobj: 0, blkno: 312518, size: 4096 swap_pager: indefinite wait buffer: bufobj: 0, blkno: 312416, size: = 16384 waited 39s for async swap write waited 39s for swap buffer waited 39s for async swap write waited 39s for async swap write waited 39s for async swap write swap_pager: indefinite wait buffer: bufobj: 0, blkno: 314802, size: = 24576 . . . Warner: These numbers are consistent with the theory that the swap device = becomes overwhelmed, spiking latency and causing crappy down-stream = effects. You can use the I/O scheduler to limit the write rates at the = low end. You might also be able to schedule a lower write queue depth at = the top end as well, but I've not seen good ways to do that. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)