From owner-freebsd-stable@freebsd.org Mon Nov 18 12:04:08 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6810C1BDD26 for ; Mon, 18 Nov 2019 12:04:08 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47GngB5X74z3Fkw; Mon, 18 Nov 2019 12:04:06 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id xAIC3rOH025802 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Nov 2019 12:03:56 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: delphij@gmail.com Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id xAIC3nFP051983 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 18 Nov 2019 19:03:49 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: bhyve memory leak in stable/11 To: Xin LI References: <7fddcea5-2188-afe1-3ea9-a53dffdbec32@grosbein.net> Cc: FreeBSD stable , Alexander Motin From: Eugene Grosbein Message-ID: Date: Mon, 18 Nov 2019 19:03:44 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 47GngB5X74z3Fkw X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.72 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; IP_SCORE(-1.62)[ip: (-4.14), ipnet: 2a01:4f8::/29(-2.29), asn: 24940(-1.64), country: DE(-0.01)]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Nov 2019 12:04:08 -0000 After 2 days of uptime there is a guest that should have only 2GB of RAM but uses nearly 22G "extra": # ps -o vsz,rss,command | grep sfile01 29234040 24851428 bhyve: sfile01 (bhyve) # bhyvectl --vm=sfile01 --get-all | grep -B1 -A1 mem ID Length Name 0 2048MB sysmem 1 2048KB bootrom Address Length Segment Offset Prot Flags 0 2048MB sysmem 0 RWX FFE00000 2048KB bootrom 0 R-X -- number of ExtINTs delivered to vcpu 0 Resident memory 2085765120 Wired memory 0 vcpu total runtime 5701638719326 Also, there is another guest vm using 23696M despite of limit 2048M, and another one using 25238M despite of limit 4096M, and another one using 52867M despite of limit 40960M. Please point me to right direction for debugging this. From owner-freebsd-stable@freebsd.org Mon Nov 18 13:07:09 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 912281BF599 for ; Mon, 18 Nov 2019 13:07:09 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Gq3w3235z4MhD; Mon, 18 Nov 2019 13:07:08 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id xAID6vRx026284 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Nov 2019 13:07:00 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: delphij@gmail.com Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id xAID6nLn052686 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 18 Nov 2019 20:06:49 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: bhyve memory leak in stable/11 To: Xin LI References: <7fddcea5-2188-afe1-3ea9-a53dffdbec32@grosbein.net> Cc: Alexander Motin , FreeBSD stable From: Eugene Grosbein Message-ID: <0cb84655-bdd2-1881-cfa2-09875c0aa7ff@grosbein.net> Date: Mon, 18 Nov 2019 20:06:44 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 47Gq3w3235z4MhD X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.72 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; IP_SCORE(-1.62)[ip: (-4.17), ipnet: 2a01:4f8::/29(-2.30), asn: 24940(-1.64), country: DE(-0.01)]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Nov 2019 13:07:09 -0000 18.11.2019 19:03, Eugene Grosbein wrote: > Please point me to right direction for debugging this. Is it normal that over 1/3rd of 360G total physical RAM is in "Laundry" category in addition to 173G Wired? last pid: 20372; load averages: 8.04, 7.73, 7.84 up 2+05:55:29 16:04:02 130 processes: 3 running, 126 sleeping, 1 zombie CPU: 1.1% user, 0.0% nice, 13.2% system, 0.1% interrupt, 85.7% idle Mem: 42G Active, 8325M Inact, 112G Laundry, 173G Wired, 7809M Free ARC: 131G Total, 28G MFU, 90G MRU, 11M Anon, 2442M Header, 10G Other 107G Compressed, 363G Uncompressed, 3.41:1 Ratio Swap: 64G Total, 16G Used, 48G Free, 24% Inuse PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 78042 root 34 20 0 54328M 52867M kqread 7 81.4H 210.63% bhyve: sappdev (bhyve) 59085 root 20 20 0 31512M 25256M kqread 6 490:16 16.25% bhyve: sdc01 (bhyve) 59568 root 28 20 0 28549M 24270M kqread 6 143:32 1.22% bhyve: sfile01 (bhyve) 60011 root 20 20 0 30262M 23697M kqread 27 121:22 1.08% bhyve: skms01 (bhyve) 63676 root 34 20 0 16418M 12799M kqread 3 113:06 19.92% bhyve: solap (bhyve) 26819 root 26 20 0 12321M 10472M kqread 28 151:43 10.12% bhyve: srdapp01 (bhyve) 63662 root 34 20 0 8226M 6969M kqread 4 114:52 20.36% bhyve: ssql01 (bhyve) From owner-freebsd-stable@freebsd.org Mon Nov 18 14:29:17 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0ACF61C1278 for ; Mon, 18 Nov 2019 14:29:17 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "gromit.dlib.vt.edu", Issuer "Chumby Certificate Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Grtg3yLmz3GFl; Mon, 18 Nov 2019 14:29:15 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from pmather-dld-1.lib.vt.edu (pmather-dld-1.lib.vt.edu [128.173.51.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gromit.dlib.vt.edu (Postfix) with ESMTPSA id 603DE35; Mon, 18 Nov 2019 09:29:05 -0500 (EST) Content-Type: text/plain; charset=us-ascii; delsp=yes; format=flowed Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: bhyve memory leak in stable/11 From: Paul Mather In-Reply-To: <0cb84655-bdd2-1881-cfa2-09875c0aa7ff@grosbein.net> Date: Mon, 18 Nov 2019 09:29:04 -0500 Cc: Xin LI , Alexander Motin , FreeBSD stable Content-Transfer-Encoding: 7bit Message-Id: References: <7fddcea5-2188-afe1-3ea9-a53dffdbec32@grosbein.net> <0cb84655-bdd2-1881-cfa2-09875c0aa7ff@grosbein.net> To: Eugene Grosbein X-Mailer: Apple Mail (2.3445.9.1) X-Rspamd-Queue-Id: 47Grtg3yLmz3GFl X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=vt.edu (policy=none); spf=none (mx1.freebsd.org: domain of paul@gromit.dlib.vt.edu has no SPF policy when checking 128.173.49.70) smtp.mailfrom=paul@gromit.dlib.vt.edu X-Spamd-Result: default: False [-2.09 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[vt.edu : No valid SPF, No valid DKIM,none]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-0.99)[-0.988,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-0.61)[ip: (-1.53), ipnet: 128.173.0.0/16(-0.76), asn: 1312(-0.70), country: US(-0.05)]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:1312, ipnet:128.173.0.0/16, country:US]; FREEMAIL_CC(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Nov 2019 14:29:17 -0000 On Nov 18, 2019, at 8:06 AM, Eugene Grosbein wrote: > 18.11.2019 19:03, Eugene Grosbein wrote: > >> Please point me to right direction for debugging this. > > Is it normal that over 1/3rd of 360G total physical RAM is in "Laundry" > category in addition to 173G Wired? > > last pid: 20372; load averages: 8.04, 7.73, > 7.84 up 2+05:55:29 16:04:02 > 130 processes: 3 running, 126 sleeping, 1 zombie > CPU: 1.1% user, 0.0% nice, 13.2% system, 0.1% interrupt, 85.7% idle > Mem: 42G Active, 8325M Inact, 112G Laundry, 173G Wired, 7809M Free > ARC: 131G Total, 28G MFU, 90G MRU, 11M Anon, 2442M Header, 10G Other > 107G Compressed, 363G Uncompressed, 3.41:1 Ratio > Swap: 64G Total, 16G Used, 48G Free, 24% Inuse > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > 78042 root 34 20 0 54328M 52867M kqread 7 81.4H 210.63% > bhyve: sappdev (bhyve) > 59085 root 20 20 0 31512M 25256M kqread 6 490:16 16.25% > bhyve: sdc01 (bhyve) > 59568 root 28 20 0 28549M 24270M kqread 6 143:32 1.22% > bhyve: sfile01 (bhyve) > 60011 root 20 20 0 30262M 23697M kqread 27 121:22 1.08% > bhyve: skms01 (bhyve) > 63676 root 34 20 0 16418M 12799M kqread 3 113:06 19.92% > bhyve: solap (bhyve) > 26819 root 26 20 0 12321M 10472M kqread 28 151:43 10.12% > bhyve: srdapp01 (bhyve) > 63662 root 34 20 0 8226M 6969M kqread 4 114:52 20.36% > bhyve: ssql01 (bhyve) I wondered the same back in late March this year: https://www.mail-archive.com/freebsd-stable@freebsd.org/msg137556.html I have a 12-STABLE system that has 16 GB RAM yet regularly shows hundred of megabytes of "Laundry." To be fair, it's also showing a good chunk of free memory, so maybe the philosophy is "why bother to do ANYTHING unless you absolutely have to?" (There's also a lower amount of "Inactive" memory, but still amounting to a couple of hundred megabytes.) My concern is that when I do need to grab a lot of free memory in a hurry (like when I do a Poudriere bulk run, or when I use the GitLab instance that runs in a jail on the machine), then there is a mad scramble to obtain memory. It seems increasingly that "idle" processes get pushed out to swap at these times. Oftentimes, when doing a Poudriere run, this means the GitLab processes get swapped out, which means when I next access GitLab there's a long latency whilst it gets paged back in to memory. Given there's normally a lot of idle CPU time on this system, why doesn't the laundry ever seem to get done? Is it just a matter that it is being done, but, also, more laundry is being created at an equally fast rate by something else running on the system? (Is there a way of finding out what is generating laundry?) Or, does laundry processing (and other memory reclamation) stop when the system believes there is "enough" free memory to warrant not doing any more reclamation work? (If so, how much is "enough", and is it possible to alter what the system considers to be "enough?") Cheers, Paul. From owner-freebsd-stable@freebsd.org Tue Nov 19 11:42:10 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 52B7F1AF039 for ; Tue, 19 Nov 2019 11:42:10 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47HP7P1NPRz4Mk7 for ; Tue, 19 Nov 2019 11:42:08 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Date: Tue, 19 Nov 2019 12:42:06 +0100 (CET) From: Ronald Klop To: freebsd-stable@freebsd.org Message-ID: <1237616943.9.1574163726832@localhost> Subject: jexec as user? MIME-Version: 1.0 X-Mailer: Realworks (484.1597.7baef01ebb0) Importance: Normal X-Priority: 3 (Normal) X-Rspamd-Queue-Id: 47HP7P1NPRz4Mk7 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of ronald-lists@klop.ws designates 194.109.157.24 as permitted sender) smtp.mailfrom=ronald-lists@klop.ws X-Spamd-Result: default: False [0.62 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:194.109.157.0/24]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_DN_NONE(0.00)[]; NEURAL_SPAM_MEDIUM(0.34)[0.336,0]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.89)[-0.890,0]; DMARC_NA(0.00)[klop.ws]; RCVD_IN_DNSWL_NONE(0.00)[24.157.109.194.list.dnswl.org : 127.0.15.0]; HAS_X_PRIO_THREE(0.00)[3]; IP_SCORE(-0.03)[ipnet: 194.109.0.0/16(-0.15), asn: 3265(-0.00), country: NL(0.02)]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL]; MID_RHS_NOT_FQDN(0.50)[] Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Nov 2019 11:42:10 -0000 Hi, Is it possible to jexec into a jail as a regular user. Or to enable that somewhere? Or is the way to do such a thing to set up ssh in the jail? Regards, Ronald. From owner-freebsd-stable@freebsd.org Tue Nov 19 12:42:22 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5B4511B16FD for ; Tue, 19 Nov 2019 12:42:22 +0000 (UTC) (envelope-from chris@cretaforce.gr) Received: from relay2.cretaforce.gr (relay2.cretaforce.gr [195.201.253.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.cretaforce.gr", Issuer "RapidSSL RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47HQSr6kNpz4R5N for ; Tue, 19 Nov 2019 12:42:20 +0000 (UTC) (envelope-from chris@cretaforce.gr) Received: from server1.cretaforce.gr (server1.cretaforce.gr [138.201.248.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.cretaforce.gr", Issuer "RapidSSL RSA CA 2018" (verified OK)) by smtp2.cretaforce.gr (Postfix) with ESMTPS id 3AAA31F50D for ; Tue, 19 Nov 2019 14:42:16 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cretaforce.gr; s=cretaforce; t=1574167336; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Orq8UhSJnCMq0a+Tt3FpCfte2oVopJbp9XxegI3j+2o=; b=ZkqJ6sXgI0F6PtigNlt/GSo7KLDZDtHr0sCwvPVFdhfkLVbcje2tuxwJjNybDXgkLTK2BD 4WWZpHND6UQ29CMuBaKVvS06sk7ZncsI/5KoGLq7VSavHPRDM33ER4VzaKfGdZyZthkHKT CdO9DLDR3nznH9GzS1nwCWfMqrgZIBY= Received: from christoss-air.fritz.box (ppp-94-64-150-254.home.otenet.gr [94.64.150.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: chris@cretaforce.gr) by server1.cretaforce.gr (Postfix) with ESMTPSA id 076C927335 for ; Tue, 19 Nov 2019 14:42:15 +0200 (EET) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3601.0.10\)) Subject: Re: jexec as user? From: Christos Chatzaras X-Priority: 3 (Normal) In-Reply-To: <1237616943.9.1574163726832@localhost> Date: Tue, 19 Nov 2019 14:42:14 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1237616943.9.1574163726832@localhost> To: freebsd-stable@freebsd.org X-Mailer: Apple Mail (2.3601.0.10) X-Rspamd-Queue-Id: 47HQSr6kNpz4R5N X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=cretaforce.gr header.s=cretaforce header.b=ZkqJ6sXg; dmarc=none; spf=pass (mx1.freebsd.org: domain of chris@cretaforce.gr designates 195.201.253.149 as permitted sender) smtp.mailfrom=chris@cretaforce.gr X-Spamd-Result: default: False [-3.14 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.201.253.149]; MV_CASE(0.50)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[cretaforce.gr:+]; HAS_X_PRIO_THREE(0.00)[3]; RCVD_IN_DNSWL_LOW(-0.10)[149.253.201.195.list.dnswl.org : 127.0.5.1]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-1.05)[ipnet: 195.201.0.0/16(-3.57), asn: 24940(-1.65), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:24940, ipnet:195.201.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[254.150.64.94.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[cretaforce.gr:s=cretaforce]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[cretaforce.gr]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[cretaforce.gr.dwl.dnswl.org : 127.0.5.1]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Nov 2019 12:42:22 -0000 > Is it possible to jexec into a jail as a regular user. Or to enable = that somewhere? > Or is the way to do such a thing to set up ssh in the jail? I was searching few months ago the same and I think it's not possible. Maybe you can do it using sudo.= From owner-freebsd-stable@freebsd.org Tue Nov 19 13:02:54 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 110441B21D9 for ; Tue, 19 Nov 2019 13:02:54 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [IPv6:2607:f3e0:0:3::19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pyroxene2.sentex.ca", Issuer "pyroxene2.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47HQwX6JDvz4SM7 for ; Tue, 19 Nov 2019 13:02:52 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [192.168.43.29] ([192.168.43.29]) by pyroxene2a.sentex.ca (8.15.2/8.15.2) with ESMTPS id xAJD2o5R022523 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Tue, 19 Nov 2019 08:02:50 -0500 (EST) (envelope-from mike@sentex.net) Subject: Re: jexec as user? To: Ronald Klop , freebsd-stable@freebsd.org References: <1237616943.9.1574163726832@localhost> From: mike tancsa Openpgp: preference=signencrypt Autocrypt: addr=mike@sentex.net; keydata= mQENBFywzOMBCACoNFpwi5MeyEREiCeHtbm6pZJI/HnO+wXdCAWtZkS49weOoVyUj5BEXRZP xflV2ib2hflX4nXqhenaNiia4iaZ9ft3I1ebd7GEbGnsWCvAnob5MvDZyStDAuRxPJK1ya/s +6rOvr+eQiXYNVvfBhrCfrtR/esSkitBGxhUkBjOti8QwzD71JVF5YaOjBAs7jZUKyLGj0kW yDg4jUndudWU7G2yc9GwpHJ9aRSUN8e/mWdIogK0v+QBHfv/dsI6zVB7YuxCC9Fx8WPwfhDH VZC4kdYCQWKXrm7yb4TiVdBh5kgvlO9q3js1yYdfR1x8mjK2bH2RSv4bV3zkNmsDCIxjABEB AAG0HW1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5uZXQ+iQFUBBMBCAA+FiEEmuvCXT0aY6hs 4SbWeVOEFl5WrMgFAlywzOYCGwMFCQHhM4AFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQ eVOEFl5WrMhnPAf7Bf+ola0V9t4i8rwCMGvzkssGaxY/5zNSZO9BgSgfN0WzgmBEOy/3R4km Yn5KH94NltJYAAE5hqkFmAwK6psOqAR9cxHrRfU+gV2KO8pCDc6K/htkQcd/mclJYpCHp6Eq EVJOiAxcNaYuHZkeMdXDuvvI5Rk82VHk84BGgxIqIrhLlkguoPbXOOa+8c/Mpb1sRAGZEOuX EzKNC49+GS9gKW6ISbanyPsGEcFyP7GKMzcHBPf3cPrewZQZ6gBoNscasL6IJeAQDqzQAxbU GjO0qBSMRgnLXK7+DJlxrYdHGXqNbV6AYsmHJ6c2WWWiuRviFBqXinlgJ2FnYebZPAfWibkB DQRcsMzkAQgA1Dpo/xWS66MaOJLwA28sKNMwkEk1Yjs+okOXDOu1F+0qvgE8sVmrOOPvvWr4 axtKRSG1t2QUiZ/ZkW/x/+t0nrM39EANV1VncuQZ1ceIiwTJFqGZQ8kb0+BNkwuNVFHRgXm1 qzAJweEtRdsCMohB+H7BL5LGCVG5JaU0lqFU9pFP40HxEbyzxjsZgSE8LwkI6wcu0BLv6K6c Lm0EiHPOl5G8kgRi38PS7/6s3R8QDsEtbGsYy6O82k3zSLIjuDBwA9GRaeigGppTxzAHVjf5 o9KKu4O7gC2KKVHPegbXS+GK7DU0fjzX57H5bZ6komE5eY4p3oWT/CwVPSGfPs8jOwARAQAB iQE8BBgBCAAmFiEEmuvCXT0aY6hs4SbWeVOEFl5WrMgFAlywzOQCGwwFCQHhM4AACgkQeVOE Fl5WrMhmjQf/dBCjAVn1J0GzSsHiLvSAQz1cchbdy8LD0Tnpzjgp5KLU7sNojbI8vqt4yKAi cayI88j8+xxNXPMWM4pHELuUuVHS5XTpHa/wwulUtI5w/zyKlUDsIvqTPZLUEwH7DfNBueVM WyNaIjV2kxSmM8rNMC+RkgyfbjGLCkmWsMRVuLIUYpl5D9WHmenUbiErlKU2KvEEXEg/aLKq 3m/AdM9RAYsP9O4l+sAZEfyYoNJzDhTZMzn/9Q0uFPLK9smDQh4WBTFaApveVJPHRKmHPoNF Xxj+yScYdQ4SKH34WnhNSELvnZQ3ulH5tpASmm0w+GxfZqSc8+QCwoKtBRDUxoE56A== Message-ID: Date: Tue, 19 Nov 2019 08:02:50 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.1 MIME-Version: 1.0 In-Reply-To: <1237616943.9.1574163726832@localhost> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 47HQwX6JDvz4SM7 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:3::19 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-1.71 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; HFILTER_HELO_IP_A(1.00)[pyroxene2a.sentex.ca]; DMARC_NA(0.00)[sentex.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; HFILTER_HELO_NORES_A_OR_MX(0.30)[pyroxene2a.sentex.ca]; IP_SCORE(-1.71)[ipnet: 2607:f3e0::/32(-4.93), asn: 11647(-3.55), country: CA(-0.09)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Nov 2019 13:02:54 -0000 On 11/19/2019 6:42 AM, Ronald Klop wrote: > Hi, > > Is it possible to jexec into a jail as a regular user. Or to enable > that somewhere? > Or is the way to do such a thing to set up ssh in the jail? > On 11.3 at least, does not the built in functionality of jexec do what you need ? jexec [-l] [-u username | -U username] jail [command ...] # jexec -U testuser 3 csh testuser@cacticonsole:/ % id uid=1005(testuser) gid=1005(testuser) groups=1005(testuser) testuser@cacticonsole:/ %     ---Mike From owner-freebsd-stable@freebsd.org Tue Nov 19 13:09:53 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 004E61B23EC for ; Tue, 19 Nov 2019 13:09:53 +0000 (UTC) (envelope-from chris@cretaforce.gr) Received: from relay1.cretaforce.gr (relay1.cretaforce.gr [195.201.253.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.cretaforce.gr", Issuer "RapidSSL RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47HR4b3MPXz4SY5 for ; Tue, 19 Nov 2019 13:09:50 +0000 (UTC) (envelope-from chris@cretaforce.gr) Received: from server1.cretaforce.gr (server1.cretaforce.gr [138.201.248.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.cretaforce.gr", Issuer "RapidSSL RSA CA 2018" (verified OK)) by smtp1.cretaforce.gr (Postfix) with ESMTPS id CBBB51F4D9 for ; Tue, 19 Nov 2019 15:09:46 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cretaforce.gr; s=cretaforce; t=1574168986; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ggdpNlf9Wyt6yTEr7LRKXpWI981CPc7duaxXtxYQ1Lg=; b=ghaCULGyg3+9RK/EUxQyWj1R2+Hh14YwcbyX5FA71uM6CoXpy7vy5MF1KNgxkdM6hyH2BY GsG+qZGaxiCgc/EJG17koBAvEOk7ZpEY314olQkPnNFQ51qqmqnv/VJRDJD4ae+MBkfel1 8zDv+LyqO3F67LjPpIEVJCt9InvUGGA= Received: from christoss-air.fritz.box (ppp-94-64-150-254.home.otenet.gr [94.64.150.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: chris@cretaforce.gr) by server1.cretaforce.gr (Postfix) with ESMTPSA id 914B427335 for ; Tue, 19 Nov 2019 15:09:46 +0200 (EET) From: Christos Chatzaras Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3601.0.10\)) Subject: Re: jexec as user? Date: Tue, 19 Nov 2019 15:09:45 +0200 References: <1237616943.9.1574163726832@localhost> To: freebsd-stable In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3601.0.10) X-Rspamd-Queue-Id: 47HR4b3MPXz4SY5 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=cretaforce.gr header.s=cretaforce header.b=ghaCULGy; dmarc=none; spf=pass (mx1.freebsd.org: domain of chris@cretaforce.gr designates 195.201.253.145 as permitted sender) smtp.mailfrom=chris@cretaforce.gr X-Spamd-Result: default: False [-3.14 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.201.253.145]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[cretaforce.gr:+]; RCVD_IN_DNSWL_LOW(-0.10)[145.253.201.195.list.dnswl.org : 127.0.5.1]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-1.04)[ipnet: 195.201.0.0/16(-3.56), asn: 24940(-1.65), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:24940, ipnet:195.201.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[254.150.64.94.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[cretaforce.gr:s=cretaforce]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[cretaforce.gr]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[cretaforce.gr.dwl.dnswl.org : 127.0.5.1]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Nov 2019 13:09:53 -0000 > On 19 Nov 2019, at 15:02, mike tancsa wrote: >=20 > On 11/19/2019 6:42 AM, Ronald Klop wrote: >> Hi, >>=20 >> Is it possible to jexec into a jail as a regular user. Or to enable >> that somewhere? >> Or is the way to do such a thing to set up ssh in the jail? >>=20 > On 11.3 at least, does not the built in functionality of jexec do what > you need ? >=20 > jexec [-l] [-u username | -U username] jail [command ...] >=20 > # jexec -U testuser 3 csh > testuser@cacticonsole:/ % id > uid=3D1005(testuser) gid=3D1005(testuser) groups=3D1005(testuser) > testuser@cacticonsole:/ % >=20 I think he wants to use jexec as a normal user from the main OS. If he wants to run jexec as root and login to jail as user then your = command works.= From owner-freebsd-stable@freebsd.org Tue Nov 19 13:16:34 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 615F61B275B for ; Tue, 19 Nov 2019 13:16:34 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [IPv6:2607:f3e0:0:3::19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pyroxene2.sentex.ca", Issuer "pyroxene2.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47HRDK69bMz4SyK for ; Tue, 19 Nov 2019 13:16:33 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [192.168.43.29] ([192.168.43.29]) by pyroxene2a.sentex.ca (8.15.2/8.15.2) with ESMTPS id xAJDGUOa023628 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Tue, 19 Nov 2019 08:16:30 -0500 (EST) (envelope-from mike@sentex.net) Subject: Re: jexec as user? To: Christos Chatzaras , freebsd-stable References: <1237616943.9.1574163726832@localhost> From: mike tancsa Openpgp: preference=signencrypt Autocrypt: addr=mike@sentex.net; keydata= mQENBFywzOMBCACoNFpwi5MeyEREiCeHtbm6pZJI/HnO+wXdCAWtZkS49weOoVyUj5BEXRZP xflV2ib2hflX4nXqhenaNiia4iaZ9ft3I1ebd7GEbGnsWCvAnob5MvDZyStDAuRxPJK1ya/s +6rOvr+eQiXYNVvfBhrCfrtR/esSkitBGxhUkBjOti8QwzD71JVF5YaOjBAs7jZUKyLGj0kW yDg4jUndudWU7G2yc9GwpHJ9aRSUN8e/mWdIogK0v+QBHfv/dsI6zVB7YuxCC9Fx8WPwfhDH VZC4kdYCQWKXrm7yb4TiVdBh5kgvlO9q3js1yYdfR1x8mjK2bH2RSv4bV3zkNmsDCIxjABEB AAG0HW1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5uZXQ+iQFUBBMBCAA+FiEEmuvCXT0aY6hs 4SbWeVOEFl5WrMgFAlywzOYCGwMFCQHhM4AFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQ eVOEFl5WrMhnPAf7Bf+ola0V9t4i8rwCMGvzkssGaxY/5zNSZO9BgSgfN0WzgmBEOy/3R4km Yn5KH94NltJYAAE5hqkFmAwK6psOqAR9cxHrRfU+gV2KO8pCDc6K/htkQcd/mclJYpCHp6Eq EVJOiAxcNaYuHZkeMdXDuvvI5Rk82VHk84BGgxIqIrhLlkguoPbXOOa+8c/Mpb1sRAGZEOuX EzKNC49+GS9gKW6ISbanyPsGEcFyP7GKMzcHBPf3cPrewZQZ6gBoNscasL6IJeAQDqzQAxbU GjO0qBSMRgnLXK7+DJlxrYdHGXqNbV6AYsmHJ6c2WWWiuRviFBqXinlgJ2FnYebZPAfWibkB DQRcsMzkAQgA1Dpo/xWS66MaOJLwA28sKNMwkEk1Yjs+okOXDOu1F+0qvgE8sVmrOOPvvWr4 axtKRSG1t2QUiZ/ZkW/x/+t0nrM39EANV1VncuQZ1ceIiwTJFqGZQ8kb0+BNkwuNVFHRgXm1 qzAJweEtRdsCMohB+H7BL5LGCVG5JaU0lqFU9pFP40HxEbyzxjsZgSE8LwkI6wcu0BLv6K6c Lm0EiHPOl5G8kgRi38PS7/6s3R8QDsEtbGsYy6O82k3zSLIjuDBwA9GRaeigGppTxzAHVjf5 o9KKu4O7gC2KKVHPegbXS+GK7DU0fjzX57H5bZ6komE5eY4p3oWT/CwVPSGfPs8jOwARAQAB iQE8BBgBCAAmFiEEmuvCXT0aY6hs4SbWeVOEFl5WrMgFAlywzOQCGwwFCQHhM4AACgkQeVOE Fl5WrMhmjQf/dBCjAVn1J0GzSsHiLvSAQz1cchbdy8LD0Tnpzjgp5KLU7sNojbI8vqt4yKAi cayI88j8+xxNXPMWM4pHELuUuVHS5XTpHa/wwulUtI5w/zyKlUDsIvqTPZLUEwH7DfNBueVM WyNaIjV2kxSmM8rNMC+RkgyfbjGLCkmWsMRVuLIUYpl5D9WHmenUbiErlKU2KvEEXEg/aLKq 3m/AdM9RAYsP9O4l+sAZEfyYoNJzDhTZMzn/9Q0uFPLK9smDQh4WBTFaApveVJPHRKmHPoNF Xxj+yScYdQ4SKH34WnhNSELvnZQ3ulH5tpASmm0w+GxfZqSc8+QCwoKtBRDUxoE56A== Message-ID: Date: Tue, 19 Nov 2019 08:16:30 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 47HRDK69bMz4SyK X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:3::19 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-1.71 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; HFILTER_HELO_IP_A(1.00)[pyroxene2a.sentex.ca]; DMARC_NA(0.00)[sentex.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; HFILTER_HELO_NORES_A_OR_MX(0.30)[pyroxene2a.sentex.ca]; IP_SCORE(-1.71)[ipnet: 2607:f3e0::/32(-4.93), asn: 11647(-3.55), country: CA(-0.09)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Nov 2019 13:16:34 -0000 On 11/19/2019 8:09 AM, Christos Chatzaras wrote: > On 19 Nov 2019, at 15:02, mike tancsa wrote: >> On 11/19/2019 6:42 AM, Ronald Klop wrote: >>> Hi, >>> >>> Is it possible to jexec into a jail as a regular user. Or to enable >>> that somewhere? >>> Or is the way to do such a thing to set up ssh in the jail? >>> >> On 11.3 at least, does not the built in functionality of jexec do what >> you need ? >> >> jexec [-l] [-u username | -U username] jail [command ...] >> >> # jexec -U testuser 3 csh >> testuser@cacticonsole:/ % id >> uid=1005(testuser) gid=1005(testuser) groups=1005(testuser) >> testuser@cacticonsole:/ % >> > I think he wants to use jexec as a normal user from the main OS. > > If he wants to run jexec as root and login to jail as user then your command works. Ahhh, my mistake.    A sudo entry then ?     ---Mike From owner-freebsd-stable@freebsd.org Tue Nov 19 17:25:36 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D6D391B8C39 for ; Tue, 19 Nov 2019 17:25:36 +0000 (UTC) (envelope-from wfc@mintsol.com) Received: from scully.mintsol.com (scully.mintsol.com [199.182.77.206]) by mx1.freebsd.org (Postfix) with ESMTP id 47HXlg5YBYz3F6M for ; Tue, 19 Nov 2019 17:25:35 +0000 (UTC) (envelope-from wfc@mintsol.com) Received: from mintsol.com (officecc.mintsol.com [96.85.114.33]) by scully.mintsol.com with esmtp; Tue, 19 Nov 2019 12:25:29 -0500 id 00A498C1.000000005DD42589.0001500A Received: from localhost (localhost [127.0.0.1]) (IDENT: uid 1002) by mintsol.com with esmtp; Tue, 19 Nov 2019 12:25:29 -0500 id 000008C6.5DD42589.00010A2D Date: Tue, 19 Nov 2019 12:25:29 -0500 (EST) From: Walter Cramer To: mike tancsa cc: Christos Chatzaras , freebsd-stable Subject: Re: jexec as user? In-Reply-To: Message-ID: <20191119120818.M60603@mulder.mintsol.com> References: <1237616943.9.1574163726832@localhost> MIME-Version: 1.0 X-Rspamd-Queue-Id: 47HXlg5YBYz3F6M X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of wfc@mintsol.com designates 199.182.77.206 as permitted sender) smtp.mailfrom=wfc@mintsol.com X-Spamd-Result: default: False [-2.50 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a:scully.mintsol.com]; NEURAL_HAM_LONG(-0.99)[-0.991,0]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; DMARC_NA(0.00)[mintsol.com]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; CTYPE_MIXED_BOGUS(1.00)[]; IP_SCORE(-2.31)[ip: (-6.05), ipnet: 199.182.77.0/24(-3.03), asn: 22768(-2.42), country: US(-0.05)]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:22768, ipnet:199.182.77.0/24, country:US]; MIME_TRACE(0.00)[0:+,1:+] Content-Type: TEXT/PLAIN; charset=utf-8; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Nov 2019 17:25:36 -0000 On Tue, 19 Nov 2019, mike tancsa wrote: > On 11/19/2019 8:09 AM, Christos Chatzaras wrote: > On 19 Nov 2019, at 15:02, mike tancsa wrote: >> On 11/19/2019 6:42 AM, Ronald Klop wrote: >>> Hi, >>> >>> Is it possible to jexec into a jail as a regular user. Or to enable >>> that somewhere? >>> Or is the way to do such a thing to set up ssh in the jail? >>> >> On 11.3 at least, does not the built in functionality of jexec do what >> you need ? >> >> jexec [-l] [-u username | -U username] jail [command ...] >> >> # jexec -U testuser 3 csh >> testuser@cacticonsole:/ % id >> uid=3D1005(testuser) gid=3D1005(testuser) groups=3D1005(testuser) >> testuser@cacticonsole:/ % >> > I think he wants to use jexec as a normal user from the main OS. > > If he wants to run jexec as root and login to jail as user then your comm= and works. Ahhh, my mistake.=C2=A0 =C2=A0 A sudo entry then ? =C2=A0=C2=A0=C2=A0 ---Mike At least on older FreeBSD versions, it's easy to wrap `jexec` in a few=20 lines (literally a half-dozen) of C code, suid after compiling, and have=20 users in the host environment jump into jails with it. (I haven't set=20 this up in a while, to know if there are issues with 11.X or 12.X.) OTOH, there is a bitter-regret-filled gap between knowing enough to do=20 that, and knowing enough to securely write and deploy suid-root programs. -Walter _______________________________________________ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Tue Nov 19 19:31:27 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D97C41BBE20 for ; Tue, 19 Nov 2019 19:31:27 +0000 (UTC) (envelope-from SRS0=ydOD=ZL=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47HbXt4HZvz3M3f for ; Tue, 19 Nov 2019 19:31:26 +0000 (UTC) (envelope-from SRS0=ydOD=ZL=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 149BC28411; Tue, 19 Nov 2019 20:31:22 +0100 (CET) Received: from illbsd.quip.test (ip-62-24-92-232.net.upcbroadband.cz [62.24.92.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id A71F32840C; Tue, 19 Nov 2019 20:31:20 +0100 (CET) Subject: Re: jexec as user? To: Christos Chatzaras , freebsd-stable References: <1237616943.9.1574163726832@localhost> Cc: Ronald Klop From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <06464ab7-abc4-9ee4-a27e-9e4591eebc83@quip.cz> Date: Tue, 19 Nov 2019 20:31:20 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.3 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 47HbXt4HZvz3M3f X-Spamd-Bar: +++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of SRS0=ydOD=ZL=quip.cz=000.fbsd@elsa.codelab.cz has no SPF policy when checking 94.124.105.4) smtp.mailfrom=SRS0=ydOD=ZL=quip.cz=000.fbsd@elsa.codelab.cz X-Spamd-Result: default: False [5.07 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; IP_SCORE(0.89)[ip: (0.41), ipnet: 94.124.104.0/21(0.20), asn: 42000(3.76), country: CZ(0.09)]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[quip.cz]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.98)[0.984,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_SPAM_LONG(1.00)[0.996,0]; RCVD_IN_DNSWL_NONE(0.00)[4.105.124.94.list.dnswl.org : 127.0.10.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=ydOD=ZL=quip.cz=000.fbsd@elsa.codelab.cz]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; SUBJECT_ENDS_QUESTION(1.00)[]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz, SRS0=ydOD=ZL=quip.cz=000.fbsd@elsa.codelab.cz] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Nov 2019 19:31:27 -0000 Christos Chatzaras wrote on 2019/11/19 14:09: > > >> On 19 Nov 2019, at 15:02, mike tancsa wrote: >> >> On 11/19/2019 6:42 AM, Ronald Klop wrote: >>> Hi, >>> >>> Is it possible to jexec into a jail as a regular user. Or to enable >>> that somewhere? >>> Or is the way to do such a thing to set up ssh in the jail? >>> >> On 11.3 at least, does not the built in functionality of jexec do what >> you need ? >> >> jexec [-l] [-u username | -U username] jail [command ...] >> >> # jexec -U testuser 3 csh >> testuser@cacticonsole:/ % id >> uid=1005(testuser) gid=1005(testuser) groups=1005(testuser) >> testuser@cacticonsole:/ % >> > > I think he wants to use jexec as a normal user from the main OS. > > If he wants to run jexec as root and login to jail as user then your command works. If you want to use jexec as normal user in host, look at sysutils/jailme from ports: https://www.freshports.org/sysutils/jailme/ This version is installed setuid and does some sanity checking to ensure the username and UID match between the jail and the host system. WWW: https://github.com/Intermedix/jailme Miroslav Lachman PS: I never used jailme personally From owner-freebsd-stable@freebsd.org Tue Nov 19 21:04:24 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B5AB51BDE6A for ; Tue, 19 Nov 2019 21:04:24 +0000 (UTC) (envelope-from dewaynegeraghty@gmail.com) Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Hdc76clmz3RFN for ; Tue, 19 Nov 2019 21:04:23 +0000 (UTC) (envelope-from dewaynegeraghty@gmail.com) Received: by mail-wr1-x431.google.com with SMTP id i12so25603214wro.5 for ; Tue, 19 Nov 2019 13:04:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=8CMwH0s1RFku6hM9DF/bhzCuN56AcrZZK8cneXqD+/c=; b=WzX5uPbVLCRw2fDPqQGV6blFnOMKmQ7mrcczVeptrZtvGUFVDHFuO+rkGxIk9uKJH8 f6bvmMejW16TNovvlLawzQLdZGKHi8+scrPZbLdQdXzMsN+sBo2lectuChF8Z1cHiRco 9rmXKxckdpoY1Hsur0KIYoVtt0cknrSRUuCJC//hvrCN5931PavlQ3pMwuMxu6/8WQQl R2mA7/2ARq/R8g/cqrsZT0i0SFkpcrNuAFbYqTlq67KuSzRdPZnjoXZlMf4XDxeGLlkB XWqF1S8u75sdaoQuVSIRg3RgZqPn8Dn148bm6ga2oKbNUFHu8Ny9V+CS5l0q9GolKdT7 Pkaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=8CMwH0s1RFku6hM9DF/bhzCuN56AcrZZK8cneXqD+/c=; b=ZkrK3kbuvngvaZCZB9gkEvUQcp3QSh7F866xyAqaZyVdbmFjNSqKiVRaPLF/Q5dSaD kOYxqyVWur0bDvb2t+dDYCInsOXDUS52+yTEOnNoVMO/1nG3RtyVBSYP9q2DijLHX3mx 43Tjq613hQJKEPsVipE8dFtvHqSKTfISd5fUM2SVRna3pzBGgfGwum6f4ayTjs4AsxBS YWajWnR16QnplIgcIq6c5HogeNtQffN0HiXr15Hfr1A7pofWVjoJgqsp8G2glUEQOsae 9zjtbiHKi6wx7kyEogLjS9uVUz2scptvnf+b6eqdOoL4Xnabh7ODejKdVt5EmpY+50e5 aHCQ== X-Gm-Message-State: APjAAAX7tGk8I2AGFeiD0bRZ4l1EAyG7HfhJREnmH3NVrstH//oNyEPw EOQGRM78Z+OoUY+W2k+3jKyy4zamqpD6RJRnfEo= X-Google-Smtp-Source: APXvYqzuv1tsnDMFvKeFyfYseVk0UeveA/IXx7KC3OCwl7Cip64i/efVQp7vt9v69oGUt415Pop2DKiiPcayLTewnMU= X-Received: by 2002:adf:eecc:: with SMTP id a12mr37858266wrp.363.1574197461561; Tue, 19 Nov 2019 13:04:21 -0800 (PST) MIME-Version: 1.0 References: <1237616943.9.1574163726832@localhost> <06464ab7-abc4-9ee4-a27e-9e4591eebc83@quip.cz> In-Reply-To: <06464ab7-abc4-9ee4-a27e-9e4591eebc83@quip.cz> From: Dewayne Geraghty Date: Wed, 20 Nov 2019 08:03:54 +1100 Message-ID: Subject: Re: jexec as user? To: Ronald Klop Cc: freebsd-stable X-Rspamd-Queue-Id: 47Hdc76clmz3RFN X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=WzX5uPbV; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of dewaynegeraghty@gmail.com designates 2a00:1450:4864:20::431 as permitted sender) smtp.mailfrom=dewaynegeraghty@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; IP_SCORE(0.00)[ip: (-9.02), ipnet: 2a00:1450::/32(-2.72), asn: 15169(-1.97), country: US(-0.05)]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[1.3.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; SUBJECT_ENDS_QUESTION(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Nov 2019 21:04:24 -0000 Good question Ronald. A test - I can login to jail (b3) where I run apache as www user, so # jexec -U www b3 /bin/tcsh > whoami; id www uid=80(www) gid=80(www) groups=80(www) Expected - good! and I can, in the host # su -m www -c "whoami; id" www uid=80(www) gid=80(www) groups=80(www) Good - so my user exists in both host and jail. Though for your purposes the host user could be anyone. So we've demonstrated that I have an unpriv'ed user in both the host and jailed context. But.... # /usr/bin/su -m www -c "jexec -U www b3 /usr/bin/whoami" jexec: initgroups: www: Operation not permitted So unless I/we can identify the cause of this, you're stuck Which surprised me, as I typically run stuff in my jails using commands from the host, like: /usr/sbin/jexec -U www b3 /usr/local/sbin/httpd -f /usr/local/etc/apache24/httpd.conf Now to part 2 of your question. I do run sshd quite happily in the jails, so that may be an option for you. (actually I use dropbear in situations where I don't required the proper audit logs and its approx 50% of the sshd resources ;)) From owner-freebsd-stable@freebsd.org Tue Nov 19 21:33:29 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7B7891BEC99 for ; Tue, 19 Nov 2019 21:33:29 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47HfFh3qjNz3ycW for ; Tue, 19 Nov 2019 21:33:28 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1iXB7f-000Bua-Ck for freebsd-stable@freebsd.org; Wed, 20 Nov 2019 00:33:19 +0300 Date: Wed, 20 Nov 2019 00:33:19 +0300 From: Slawa Olhovchenkov To: freebsd-stable@freebsd.org Subject: Access to NETMAP from c++ program Message-ID: <20191119213319.GD38096@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-Rspamd-Queue-Id: 47HfFh3qjNz3ycW X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of slw@zxy.spb.ru has no SPF policy when checking 195.70.199.98) smtp.mailfrom=slw@zxy.spb.ru X-Spamd-Result: default: False [1.01 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.10)[-0.096,0]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[zxy.spb.ru]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; NEURAL_SPAM_LONG(0.21)[0.208,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:5495, ipnet:195.70.192.0/19, country:RU]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.00)[country: RU(0.01)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Nov 2019 21:33:29 -0000 Is this posible (now) for access to NETAMP from C++? I am see headers conflict: In file included from /usr/include/net/netmap_user.h:104: In file included from /usr/include/net/netmap.h:812: /usr/include/stdatomic.h:141:21: error: reference to 'memory_order' is ambiguous atomic_thread_fence(memory_order __order __unused) ^ /usr/include/stdatomic.h:134:3: note: candidate found by name lookup is 'memory_order' } memory_order; ^ /usr/include/c++/v1/atomic:585:3: note: candidate found by name lookup is 'std::__1::memory_order' } memory_order; ^ Yes, I am need in C++ program. Include before also don't work, w/ different error. From owner-freebsd-stable@freebsd.org Wed Nov 20 09:47:59 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6478A1B0711 for ; Wed, 20 Nov 2019 09:47:59 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47HyY96prpz4ZQF for ; Wed, 20 Nov 2019 09:47:57 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Date: Wed, 20 Nov 2019 10:47:55 +0100 (CET) From: Ronald Klop To: Miroslav Lachman <000.fbsd@quip.cz> Cc: freebsd-stable , Christos Chatzaras Message-ID: <1244063778.4.1574243275499@localhost> In-Reply-To: <06464ab7-abc4-9ee4-a27e-9e4591eebc83@quip.cz> References: <1237616943.9.1574163726832@localhost> <06464ab7-abc4-9ee4-a27e-9e4591eebc83@quip.cz> Subject: Re: jexec as user? MIME-Version: 1.0 X-Mailer: Realworks (484.1609.fa1c05e5f4f) Importance: Normal X-Priority: 3 (Normal) X-Rspamd-Queue-Id: 47HyY96prpz4ZQF X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of ronald-lists@klop.ws designates 194.109.157.24 as permitted sender) smtp.mailfrom=ronald-lists@klop.ws X-Spamd-Result: default: False [-0.49 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.71)[-0.715,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:194.109.157.0/24]; NEURAL_HAM_LONG(-0.95)[-0.953,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MIME_TRACE(0.00)[0:+,1:+,2:~]; DMARC_NA(0.00)[klop.ws]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[24.157.109.194.list.dnswl.org : 127.0.15.0]; HAS_X_PRIO_THREE(0.00)[3]; IP_SCORE(-0.03)[ipnet: 194.109.0.0/16(-0.15), asn: 3265(-0.00), country: NL(0.02)]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL]; MID_RHS_NOT_FQDN(0.50)[] Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2019 09:47:59 -0000 Thanks for all the advice. I am indeed looking for using jail from the non-root user in the host. Jailme sounds like a good solution. My use case is providing a relatively save way of giving a user the possibility to experiment with root rights (like creating and installing ports) without wracking the host system. The users are trusted so it is not so much about security. More about keeping the host system clean. Regards, Ronald. Van: Miroslav Lachman <000.fbsd@quip.cz> Datum: dinsdag, 19 november 2019 20:31 Aan: Christos Chatzaras , freebsd-stable CC: Ronald Klop Onderwerp: Re: jexec as user? > > Christos Chatzaras wrote on 2019/11/19 14:09: > > > > > >> On 19 Nov 2019, at 15:02, mike tancsa wrote: > >> > >> On 11/19/2019 6:42 AM, Ronald Klop wrote: > >>> Hi, > >>> > >>> Is it possible to jexec into a jail as a regular user. Or to enable > >>> that somewhere? > >>> Or is the way to do such a thing to set up ssh in the jail? > >>> > >> On 11.3 at least, does not the built in functionality of jexec do what > >> you need ? > >> > >> jexec [-l] [-u username | -U username] jail [command ...] > >> > >> # jexec -U testuser 3 csh > >> testuser@cacticonsole:/ % id > >> uid=1005(testuser) gid=1005(testuser) groups=1005(testuser) > >> testuser@cacticonsole:/ % > >> > > > > I think he wants to use jexec as a normal user from the main OS. > > > > If he wants to run jexec as root and login to jail as user then your command works. > > If you want to use jexec as normal user in host, look at sysutils/jailme from ports: > > https://www.freshports.org/sysutils/jailme/ > This version is installed setuid and does some sanity checking to ensure the username and UID match between the jail and the host system. > > WWW: https://github.com/Intermedix/jailme > > Miroslav Lachman > > PS: I never used jailme personally > > > From owner-freebsd-stable@freebsd.org Wed Nov 20 10:44:38 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 685B91B2448 for ; Wed, 20 Nov 2019 10:44:38 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47HzpX5x2Qz4dRN for ; Wed, 20 Nov 2019 10:44:36 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id xAKAiTw8044345 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 Nov 2019 10:44:30 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: ronald-lists@klop.ws Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id xAKAiQkD033819 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 20 Nov 2019 17:44:26 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: jexec as user? To: Ronald Klop , Miroslav Lachman <000.fbsd@quip.cz> References: <1237616943.9.1574163726832@localhost> <06464ab7-abc4-9ee4-a27e-9e4591eebc83@quip.cz> <1244063778.4.1574243275499@localhost> Cc: Christos Chatzaras , freebsd-stable From: Eugene Grosbein Message-ID: Date: Wed, 20 Nov 2019 17:44:20 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <1244063778.4.1574243275499@localhost> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 47HzpX5x2Qz4dRN X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-2.77 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; IP_SCORE(-1.67)[ip: (-4.38), ipnet: 2a01:4f8::/29(-2.31), asn: 24940(-1.64), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2019 10:44:38 -0000 20.11.2019 16:47, Ronald Klop wrote: > Thanks for all the advice. I am indeed looking for using jail from the non-root user in the host. Jailme sounds like a good solution. > > My use case is providing a relatively save way of giving a user the possibility to experiment with root rights (like creating and installing ports) without wracking the host system. > The users are trusted so it is not so much about security. More about keeping the host system clean. You also could run ssh service inside the jail and give users opportunity to experiment with ssh and keys :-) From owner-freebsd-stable@freebsd.org Wed Nov 20 12:26:57 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4343A1B5ABA for ; Wed, 20 Nov 2019 12:26:57 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47J24b0Bz4z3Gf1 for ; Wed, 20 Nov 2019 12:26:54 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Date: Wed, 20 Nov 2019 13:26:52 +0100 (CET) From: Ronald Klop To: Eugene Grosbein Cc: freebsd-stable , Miroslav Lachman <000.fbsd@quip.cz>, Christos Chatzaras Message-ID: <1232388449.5.1574252812064@localhost> In-Reply-To: References: <1237616943.9.1574163726832@localhost> <06464ab7-abc4-9ee4-a27e-9e4591eebc83@quip.cz> <1244063778.4.1574243275499@localhost> Subject: Re: jexec as user? MIME-Version: 1.0 X-Mailer: Realworks (485.1613.5095e1c6f16) Importance: Normal X-Priority: 3 (Normal) X-Rspamd-Queue-Id: 47J24b0Bz4z3Gf1 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of ronald-lists@klop.ws designates 194.109.157.24 as permitted sender) smtp.mailfrom=ronald-lists@klop.ws X-Spamd-Result: default: False [-0.41 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.63)[-0.629,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip4:194.109.157.0/24]; NEURAL_HAM_LONG(-0.95)[-0.952,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MIME_TRACE(0.00)[0:+,1:+,2:~]; DMARC_NA(0.00)[klop.ws]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[24.157.109.194.list.dnswl.org : 127.0.15.0]; HAS_X_PRIO_THREE(0.00)[3]; IP_SCORE(-0.03)[ipnet: 194.109.0.0/16(-0.15), asn: 3265(-0.00), country: NL(0.02)]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL]; MID_RHS_NOT_FQDN(0.50)[] Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2019 12:26:57 -0000 Yeah, ssh is also possible. See my original mail :-) https://lists.freebsd.org/pipermail/freebsd-stable/2019-November/091742.html I run my jails with "ip4 = inherit;". So I would need to do some port-forwarding trickery with ssh on different ports. The users already login on the host to do various actions. Jailme gives the easiest access without to much maintenance for now. Regards, Ronald. Van: Eugene Grosbein Datum: woensdag, 20 november 2019 11:44 Aan: Ronald Klop , Miroslav Lachman <000.fbsd@quip.cz> CC: Christos Chatzaras , freebsd-stable Onderwerp: Re: jexec as user? > > 20.11.2019 16:47, Ronald Klop wrote: > > > Thanks for all the advice. I am indeed looking for using jail from the non-root user in the host. Jailme sounds like a good solution. > > > > My use case is providing a relatively save way of giving a user the possibility to experiment with root rights (like creating and installing ports) without wracking the host system. > > The users are trusted so it is not so much about security. More about keeping the host system clean. > > You also could run ssh service inside the jail and give users opportunity to experiment with ssh and keys :-) > > > > From owner-freebsd-stable@freebsd.org Wed Nov 20 13:37:40 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CA34E1B72A3 for ; Wed, 20 Nov 2019 13:37:40 +0000 (UTC) (envelope-from meths@btinternet.com) Received: from re-prd-fep-040.btinternet.com (mailomta28-re.btinternet.com [213.120.69.121]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47J3fC56XLz3KRf for ; Wed, 20 Nov 2019 13:37:38 +0000 (UTC) (envelope-from meths@btinternet.com) Received: from re-prd-rgout-004.btmx-prd.synchronoss.net ([10.2.54.7]) by re-prd-fep-040.btinternet.com with ESMTP id <20191120133737.WAXG11338.re-prd-fep-040.btinternet.com@re-prd-rgout-004.btmx-prd.synchronoss.net> for ; Wed, 20 Nov 2019 13:37:37 +0000 X-Originating-IP: [95.150.228.36] X-OWM-Source-IP: 95.150.228.36 (GB) X-OWM-Env-Sender: meths@btinternet.com X-VadeSecure-score: verdict=clean score=0/300, class=clean X-RazorGate-Vade: gggruggvucftvghtrhhoucdtuddrgedufedrudehtddgheefucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuueftkffvkffujffvgffngfevqffopdfqfgfvnecuuegrihhlohhuthemuceftddtnecunecujfgurhepvffhuffkffgfgggtgfesthejredttdefjeenucfhrhhomheplfhonhcuvfhisggslhgvuceomhgvthhhshessghtihhnthgvrhhnvghtrdgtohhmqeenucffohhmrghinhepfhhrvggvsghsugdrohhrghenucfkphepleehrdduhedtrddvvdekrdefieenucfrrghrrghmpehhvghloheplgduledvrdduieekrddtrddugeekngdpihhnvghtpeelhedrudehtddrvddvkedrfeeipdhmrghilhhfrhhomhepoehmvghthhhssegsthhinhhtvghrnhgvthdrtghomhequceuqfffjgepkeeukffvoffkoffgpdhrtghpthhtohepoehfrhgvvggsshguqdhsthgrsghlvgesfhhrvggvsghsugdrohhrgheqnecuvehluhhsthgvrhfuihiivgeptd X-RazorGate-Vade-Verdict: clean 0 X-RazorGate-Vade-Classification: clean Received: from [192.168.0.148] (95.150.228.36) by re-prd-rgout-004.btmx-prd.synchronoss.net (5.8.337) (authenticated as meths@btinternet.com) id 5DB07BDD051567C6 for freebsd-stable@freebsd.org; Wed, 20 Nov 2019 13:37:37 +0000 To: freebsd-stable@freebsd.org From: Jon Tibble Subject: 12.0->12.1 and beadm/bectl issues Message-ID: Date: Wed, 20 Nov 2019 13:37:14 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 47J3fC56XLz3KRf X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.05 / 15.00]; HAS_XOIP(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:213.120.69.0/24]; FREEMAIL_FROM(0.00)[btinternet.com]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[btinternet.com:+]; DMARC_POLICY_ALLOW(-0.50)[btinternet.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[121.69.120.213.list.dnswl.org : 127.0.5.1]; IP_SCORE(0.00)[asn: 2856(0.08), country: GB(-0.08)]; RECEIVED_SPAMHAUS_PBL(0.00)[36.228.150.95.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; ASN(0.00)[asn:2856, ipnet:213.120.0.0/14, country:GB]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[btinternet.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[btinternet.com]; R_DKIM_ALLOW(-0.20)[btinternet.com:s=btmx201904]; NEURAL_HAM_MEDIUM(-0.96)[-0.959,0]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.986,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Nov 2019 13:37:40 -0000 Hi, After upgrading from 12.0-RELEASE-p11 to 12.1-RELEASE I was having some issues with kld_load and linux support which, after searching [1], seemed due to a missing /boot folder after the upgrade. This was fixed with 'ln -s /bootpool/boot /boot'. Then yesterday when I was trying to switch from quarterly packages to latest I wanted to use a new boot environment and so went through the beadm create and beadm activate but it wouldn't activate with a zpool.cache cp message and it left the new BE mounted under /tmp. After umounting and destroying I repeated the process with bectl and it worked fine, however, upon reboot I was not in the new BE but the same BE and the new one was still marked as activated for use next boot. So firstly: are the be* issues related to the earlier upgrade fix? Secondly: shouldn't beadm and bectl behave the same? Thirdly: how can I properly activate and boot to a new BE? Below is the command output of the beadm/bectl process described above. If there's any more information I can provide please let me know. Thanks, Jon [1] https://forums.freebsd.org/threads/cannot-identify-running-kernel-after-upgrading-to-freebsd-12.68772/ This is a two disk mirrored zpool on GELI with encrypted swap as configured out of the box by the 12.0 installer. root@prometheus:~ # uname -a FreeBSD prometheus 12.1-RELEASE-p1 FreeBSD 12.1-RELEASE-p1 GENERIC amd64 root@prometheus:~ # beadm list BE Active Mountpoint Space Created 12_0-RELEASE-p11 - - 1.1G 2019-10-29 21:33 12_1-RELEASE-p1-quarterly NR / 32.7G 2019-11-05 22:24 root@prometheus:~ # beadm create test Created successfully root@prometheus:~ # beadm list BE Active Mountpoint Space Created 12_0-RELEASE-p11 - - 1.1G 2019-10-29 21:33 12_1-RELEASE-p1-quarterly NR / 32.7G 2019-11-05 22:24 test - - 8.0K 2019-11-20 13:24 root@prometheus:~ # beadm activate test cp: /tmp/BE-test.pJtR9Rs6/boot/zfs/zpool.cache and /boot/zfs/zpool.cache are identical (not copied). root@prometheus:~ # beadm list BE Active Mountpoint Space Created 12_0-RELEASE-p11 - - 1.1G 2019-10-29 21:33 12_1-RELEASE-p1-quarterly NR / 32.7G 2019-11-05 22:24 test - /tmp/BE-test.pJtR9Rs6 136.0K 2019-11-20 13:24 root@prometheus:~ # beadm umount test Unmounted successfully root@prometheus:~ # beadm list BE Active Mountpoint Space Created 12_0-RELEASE-p11 - - 1.1G 2019-10-29 21:33 12_1-RELEASE-p1-quarterly NR / 32.7G 2019-11-05 22:24 test - - 136.0K 2019-11-20 13:24 root@prometheus:~ # beadm destroy test Are you sure you want to destroy 'test'? This action cannot be undone (y/[n]): y Destroyed successfully root@prometheus:~ # beadm list BE Active Mountpoint Space Created 12_0-RELEASE-p11 - - 1.1G 2019-10-29 21:33 12_1-RELEASE-p1-quarterly NR / 32.7G 2019-11-05 22:24 root@prometheus:~ # bectl list BE Active Mountpoint Space Created 12_0-RELEASE-p11 - - 1.14G 2019-10-29 21:33 12_1-RELEASE-p1-quarterly NR / 32.7G 2019-11-05 22:24 root@prometheus:~ # bectl create test root@prometheus:~ # bectl list BE Active Mountpoint Space Created 12_0-RELEASE-p11 - - 1.14G 2019-10-29 21:33 12_1-RELEASE-p1-quarterly NR / 32.7G 2019-11-05 22:24 test - - 8K 2019-11-20 13:25 root@prometheus:~ # bectl activate test successfully activated boot environment test root@prometheus:~ # bectl list BE Active Mountpoint Space Created 12_0-RELEASE-p11 - - 1.14G 2019-10-29 21:33 12_1-RELEASE-p1-quarterly N / 8K 2019-11-05 22:24 test R - 32.7G 2019-11-20 13:25 root@prometheus:~ # beadm list BE Active Mountpoint Space Created 12_0-RELEASE-p11 - - 1.1G 2019-10-29 21:33 12_1-RELEASE-p1-quarterly N / 8.0K 2019-11-05 22:24 test R - 32.7G 2019-11-20 13:25 root@prometheus:~ # Following a reboot I'll still be running in 12_1-RELEASE-p1-quarterly and test will still be marked R. From owner-freebsd-stable@freebsd.org Thu Nov 21 01:55:40 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 296271A9277 for ; Thu, 21 Nov 2019 01:55:40 +0000 (UTC) (envelope-from darius@dons.net.au) Received: from ipmail01.adl6.internode.on.net (ipmail01.adl6.internode.on.net [150.101.137.136]) by mx1.freebsd.org (Postfix) with ESMTP id 47JN1h2TwRz3ByD for ; Thu, 21 Nov 2019 01:55:35 +0000 (UTC) (envelope-from darius@dons.net.au) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A2DeEACO3/JbZGhwAg5iHQEBBQEHBQGBZ?= =?us-ascii?q?YFWLmZwEieMb4sdATUCBgwCAQEBAQEBBoExfogbjjWBZgsBAR8MAYQ8AoNwIzg?= =?us-ascii?q?SAQMBAQIBAQIFUzEMhW8ODw2BPSSDNAGBdAyYYZAWHoUihFwTjAl4gQeBEScfg?= =?us-ascii?q?heEeINwgiYCiRKHQo8bCQKRJRiJXYcglRWCWgIKBxSBXSBCgRRsGTsqAYJBCTW?= =?us-ascii?q?BaReIXoVRLAEygQUBAY1VAQE?= Received: from ppp14-2-112-104.adl-apt-pir-bras32.tpg.internode.on.net (HELO midget.dons.net.au) ([14.2.112.104]) by ipmail01.adl6.internode.on.net with ESMTP; 21 Nov 2019 12:24:05 +1030 Received: from midget.dons.net.au (localhost [127.0.0.1]) by midget.dons.net.au (8.15.2/8.15.2) with ESMTPS id xAL1rthL082503 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Thu, 21 Nov 2019 12:23:57 +1030 (ACDT) (envelope-from darius@dons.net.au) Received: (from mailnull@localhost) by midget.dons.net.au (8.15.2/8.15.2/Submit) id xAL1crsO071204 for ; Thu, 21 Nov 2019 12:08:53 +1030 (ACDT) (envelope-from darius@dons.net.au) X-Authentication-Warning: midget.dons.net.au: mailnull set sender to using -f X-MIMEDefang-Relay-be813b1f1da6d6b27d681222cb70cc4f5b642383: 203.31.81.177 Received: from [203.31.81.177] ([203.31.81.177]) by ppp14-2-112-104.adl-apt-pir-bras32.tpg.internode.on.net (envelope-sender ) (MIMEDefang) with ESMTP id xAL1cmPu071197; Thu, 21 Nov 2019 12:08:53 +1030 From: "O'Connor, Daniel" Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Date: Thu, 21 Nov 2019 12:08:48 +1030 Subject: smartd, mfi, SAS and SATA Message-Id: <72D950A5-4DFE-4CF6-A2C1-77A37D51B89E@dons.net.au> To: freebsd-stable X-Mailer: Apple Mail (2.3445.104.11) X-Spam-Score: 1.3 (*) No, score=1.3 required=5.0 tests=HELO_MISC_IP, RDNS_NONE, SPF_NONE autolearn=no autolearn_force=no version=3.4.2 X-Scanned-By: MIMEDefang 2.83 on 10.0.2.1 X-Rspamd-Queue-Id: 47JN1h2TwRz3ByD X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of darius@dons.net.au has no SPF policy when checking 150.101.137.136) smtp.mailfrom=darius@dons.net.au X-Spamd-Result: default: False [4.90 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; HAS_XAW(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; DMARC_NA(0.00)[dons.net.au]; TO_DN_ALL(0.00)[]; NEURAL_SPAM_MEDIUM(1.00)[0.999,0]; NEURAL_SPAM_LONG(1.00)[1.000,0]; R_SPF_NA(0.00)[]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:4739, ipnet:150.101.0.0/16, country:AU]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(1.50)[ip: (3.76), ipnet: 150.101.0.0/16(2.49), asn: 4739(1.25), country: AU(0.00)]; RCVD_IN_DNSWL_LOW(-0.10)[136.137.101.150.list.dnswl.org : 127.0.5.1] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2019 01:55:40 -0000 Hi everyone, I recently took delivery of a Supermicro X11SRM-F with a Broadcom = MegaRAID 9361-8i SAS 8 port card which has 4 Intel D3-S4610 960 GB SSDs = and 4 Hitachi/WD Ultrastar HC300 4TB drives each in a RAID5. I have /usr/local/etc/smartd.conf with just 'DEVICESCAN' and when smartd = starts I see.. Nov 21 01:17:49 maarsy-acq3 smartd[2103]: Opened configuration file = /usr/local/etc/smartd.conf Nov 21 01:17:49 maarsy-acq3 smartd[2103]: Drive: DEVICESCAN, implied = '-a' Directive on line 23 of file /usr/local/etc/smartd.conf Nov 21 01:17:49 maarsy-acq3 smartd[2103]: Configuration file = /usr/local/etc/smartd.conf was parsed, found DEVICESCAN, scanning = devices Nov 21 01:17:49 maarsy-acq3 smartd[2103]: Device: /dev/pass0, opened Nov 21 01:17:49 maarsy-acq3 smartd[2103]: Device: /dev/pass0, [HGST = HUS726T4TAL5204 C40H], lu id: 0x5000cca097502308, S/N: V6HE27MR, 4.00 = TB Nov 21 01:17:50 maarsy-acq3 smartd[2103]: Device: /dev/pass0, is SMART = capable. Adding to "monitor" list. Nov 21 01:17:50 maarsy-acq3 smartd[2103]: Device: /dev/pass1, opened Nov 21 01:17:50 maarsy-acq3 smartd[2103]: Device: /dev/pass1, [HGST = HUS726T4TAL5204 C40H], lu id: 0x5000cca09751d2c8, S/N: V6HEZZZR, 4.00 = TB Nov 21 01:17:50 maarsy-acq3 smartd[2103]: Device: /dev/pass1, is SMART = capable. Adding to "monitor" list. Nov 21 01:17:50 maarsy-acq3 smartd[2103]: Device: /dev/pass2, opened Nov 21 01:17:50 maarsy-acq3 smartd[2103]: Device: /dev/pass2, [HGST = HUS726T4TAL5204 C40H], lu id: 0x5000cca097509ad8, S/N: V6HEA6ZR, 4.00 = TB Nov 21 01:17:51 maarsy-acq3 smartd[2103]: Device: /dev/pass2, is SMART = capable. Adding to "monitor" list. Nov 21 01:17:51 maarsy-acq3 smartd[2103]: Device: /dev/pass3, opened Nov 21 01:17:51 maarsy-acq3 smartd[2103]: Device: /dev/pass3, [HGST = HUS726T4TAL5204 C40H], lu id: 0x5000cca0974f1630, S/N: V6HDHALR, 4.00 = TB Nov 21 01:17:51 maarsy-acq3 smartd[2103]: Device: /dev/pass3, is SMART = capable. Adding to "monitor" list. Nov 21 01:17:51 maarsy-acq3 smartd[2103]: Device: /dev/pass4, type = changed from 'scsi' to 'sat' Nov 21 01:17:51 maarsy-acq3 smartd[2103]: Device: /dev/pass5, type = changed from 'scsi' to 'sat' Nov 21 01:17:51 maarsy-acq3 smartd[2103]: Device: /dev/pass6, type = changed from 'scsi' to 'sat' Nov 21 01:17:52 maarsy-acq3 smartd[2103]: Device: /dev/pass7, type = changed from 'scsi' to 'sat' Nov 21 01:17:52 maarsy-acq3 smartd[2103]: Monitoring 0 ATA/SATA, 4 = SCSI/SAS and 0 NVMe devices Nov 21 01:17:53 maarsy-acq3 smartd[2105]: smartd has fork()ed into = background mode. New PID=3D2105. Nov 21 01:17:53 maarsy-acq3 smartd[2105]: file /var/run/smartd.pid = written containing PID 2105 So it is monitoring the SAS disks but has ignored the SATA SSDs :( [maarsy-acq3 1:33] ~> camcontrol devlist at scbus8 target 8 lun 0 (pass0) at scbus8 target 9 lun 0 (pass1) at scbus8 target 10 lun 0 (pass2) at scbus8 target 11 lun 0 (pass3) at scbus8 target 12 lun 0 (pass4) at scbus8 target 13 lun 0 (pass5) at scbus8 target 14 lun 0 (pass6) at scbus8 target 15 lun 0 (pass7) at scbus9 target 0 lun 0 (da0,pass8) If I run smartctl on an SSD I get.. [maarsy-acq3 1:33] ~> sudo smartctl -a /dev/pass4|less smartctl 7.0 2018-12-30 r4883 [FreeBSD 12.0-RELEASE amd64] (local build) Copyright (C) 2002-18, Bruce Allen, Christian Franke, = www.smartmontools.org Smartctl open device: /dev/pass4 [SAT] failed: SATA device detected, MegaRAID SAT layer is reportedly buggy, use '-d sat' to try anyhow And using what it suggests seems to work - does anyone know a way to = force it to work with DEVICESCAN? For now I've just hard coded it like so.. DEFAULT -m root /dev/pass0 /dev/pass1 /dev/pass2 /dev/pass3 /dev/pass4 -d sat /dev/pass5 -d sat /dev/pass6 -d sat /dev/pass7 -d sat but it seems clunky.. also I see these slightly puzzling messages for = each SSD.. Nov 21 01:37:08 maarsy-acq3 smartd[3656]: Device: /dev/pass4 [SAT], = opened Nov 21 01:37:08 maarsy-acq3 smartd[3656]: Device: /dev/pass4 [SAT], = INTEL SSDSC2KB960G8, S/N:PHYF92630636960CGN, WWN:5-5cd2e4-150f430c3, = FW:XCV10120, 960 GB Nov 21 01:37:08 maarsy-acq3 smartd[3656]: Device: /dev/pass4 [SAT], = found in smartd database: Intel S4510/S4610/S4500/S4600 Series SSDs Nov 21 01:37:08 maarsy-acq3 smartd[3656]: Device: /dev/pass4 [SAT], not = capable of SMART Health Status check Nov 21 01:37:08 maarsy-acq3 smartd[3656]: Device: /dev/pass4 [SAT], = can't monitor Offline_Uncorrectable count - no Attribute 198 Nov 21 01:37:08 maarsy-acq3 smartd[3656]: Device: /dev/pass4 [SAT], is = SMART capable. Adding to "monitor" list. Which I am hoping aren't anything to worry about.. -- Daniel O'Connor "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum From owner-freebsd-stable@freebsd.org Thu Nov 21 09:51:47 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0F35E1B79CA for ; Thu, 21 Nov 2019 09:51:47 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47JZb60wh2z496h for ; Thu, 21 Nov 2019 09:51:45 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id xAL9pHZE053785 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 21 Nov 2019 09:51:19 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: Received: from [10.58.0.4] (dadv@[10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id xAL9pEsX045639 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Thu, 21 Nov 2019 16:51:14 +0700 (+07) (envelope-from eugen@grosbein.net) To: FreeBSD stable From: Eugene Grosbein Subject: Logging boot messages from the loader Message-ID: Date: Thu, 21 Nov 2019 16:51:13 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 47JZb60wh2z496h X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.78 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[grosbein.net]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.68)[ip: (-4.41), ipnet: 2a01:4f8::/29(-2.32), asn: 24940(-1.64), country: DE(-0.01)]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2019 09:51:47 -0000 Hi! How do I get boot messages generated by loader itself written to some log after successfull boot? This is generic BIOS-based 11.3-STABLE/amd64 system with syscons console driver. /var/run/dmesg.boot starts with kernel messages without previous lines from the loader. Specifically, I need messages about loading kernel modules with full paths. From owner-freebsd-stable@freebsd.org Thu Nov 21 10:22:15 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C15251B8929 for ; Thu, 21 Nov 2019 10:22:15 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (cross.sbone.de [195.201.62.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47JbGD030mz4BhQ for ; Thu, 21 Nov 2019 10:22:11 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id D3F1A8D4A13E; Thu, 21 Nov 2019 10:22:03 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 5FD2BE707C0; Thu, 21 Nov 2019 10:22:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id YzdZJh6yzdYv; Thu, 21 Nov 2019 10:22:01 +0000 (UTC) Received: from [192.168.2.110] (unknown [IPv6:fde9:577b:c1a9:31:a48e:25c1:a165:8cd7]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 2833EE707B8; Thu, 21 Nov 2019 10:22:00 +0000 (UTC) From: "Bjoern A. Zeeb" To: "Eugene Grosbein" Cc: "FreeBSD stable" Subject: Re: Logging boot messages from the loader Date: Thu, 21 Nov 2019 10:22:00 +0000 X-Mailer: MailMate (2.0BETAr6142) Message-ID: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 47JbGD030mz4BhQ X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of bzeeb-lists@lists.zabbadoz.net designates 195.201.62.131 as permitted sender) smtp.mailfrom=bzeeb-lists@lists.zabbadoz.net X-Spamd-Result: default: False [-5.12 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.201.62.131]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[zabbadoz.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-2.82)[ip: (-8.87), ipnet: 195.201.0.0/16(-3.57), asn: 24940(-1.64), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:195.201.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2019 10:22:15 -0000 On 21 Nov 2019, at 9:51, Eugene Grosbein wrote: > Hi! > > How do I get boot messages generated by loader itself written to some > log after successfull boot? > This is generic BIOS-based 11.3-STABLE/amd64 system with syscons > console driver. > > /var/run/dmesg.boot starts with kernel messages without previous lines > from the loader. > Specifically, I need messages about loading kernel modules with full > paths. loader cannot write to the kernel message buffer as there is no kernel yet. There might be possible ways to preserve that but I doubt loader is fit for that task. A boot -v (boot_verbose=“YES”) might spit the information out from the kernel as well. Not sure if that helps you. However on modern server machines a boot -v often overflows the message buffer.. so you might have to tune that as well. Extracted from an older console log of a random lab machine of someone else: .. WARNING: WITNESS option enabled, expect reduced performance.^M^@ WARNING: DIAGNOSTIC option enabled, expect reduced performance.^M^@ Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff80e66000.^M^@ Preloaded elf obj module "/boot/kernel/if_igb.ko" at 0xffffffff80e66968.^M^@ Preloaded elf obj module "/boot/kernel/nfscl.ko" at 0xffffffff80e66fd0.^M^@ Preloaded elf obj module "/boot/kernel/nfslock.ko" at 0xffffffff80e67638.^M^@ Preloaded elf obj module "/boot/kernel/nfssvc.ko" at 0xffffffff80e67ba0.^M^@ Preloaded elf obj module "/boot/kernel/krpc.ko" at 0xffffffff80e68088.^M^@ Preloaded elf obj module "/boot/kernel/nfscommon.ko" at 0xffffffff80e686b0.^M^@ Preloaded elf obj module "/boot/kernel/ahci.ko" at 0xffffffff80e68d20.^M^@ Preloaded elf obj module "/boot/kernel/cam.ko" at 0xffffffff80e69308.^M^@ Preloaded elf obj module "/boot/kernel/mfi.ko" at 0xffffffff80e699f0.^M^@ Preloaded elf obj module "/boot/kernel/geom_part_gpt.ko" at 0xffffffff80e6a058.^M^@ Preloaded elf obj module "/boot/kernel/sysvsem.ko" at 0xffffffff80e6a648.^M^@ Preloaded elf obj module "/boot/kernel/sysvshm.ko" at 0xffffffff80e6acb0.^M^@ Calibrating TSC clock ... TSC clock: 3400185886 Hz^M^@ CPU: Intel(R) Xeon(R) CPU X5272 @ 3.40GHz (3400.19-MHz K8-class CPU)^M^@ .. From owner-freebsd-stable@freebsd.org Thu Nov 21 10:48:00 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C38CB1B91C7 for ; Thu, 21 Nov 2019 10:48:00 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Jbr00dM4z4CcY for ; Thu, 21 Nov 2019 10:47:59 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id xALAhuKS054560 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 21 Nov 2019 10:43:56 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: bzeeb-lists@lists.zabbadoz.net Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id xALAhrRM046105 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 21 Nov 2019 17:43:53 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Logging boot messages from the loader To: "Bjoern A. Zeeb" References: Cc: FreeBSD stable From: Eugene Grosbein Message-ID: <4c45a371-c352-b8c4-65c6-d1bff55b4a2e@grosbein.net> Date: Thu, 21 Nov 2019 17:43:48 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=3.3 required=5.0 tests=BAYES_00,HELO_MISC_IP, LOCAL_FROM,RDNS_NONE,SPF_PASS,T_DATE_IN_FUTURE_96_Q autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 T_DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after * Received: date * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS * 1.1 HELO_MISC_IP Looking for more Dynamic IP Relays X-Spam-Level: *** X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on eg.sd.rdtc.ru X-Rspamd-Queue-Id: 47Jbr00dM4z4CcY X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.78 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-1.68)[ip: (-4.43), ipnet: 2a01:4f8::/29(-2.32), asn: 24940(-1.64), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2019 10:48:00 -0000 21.11.2019 17:22, Bjoern A. Zeeb wrote: > On 21 Nov 2019, at 9:51, Eugene Grosbein wrote: > >> Hi! >> >> How do I get boot messages generated by loader itself written to some log after successfull boot? >> This is generic BIOS-based 11.3-STABLE/amd64 system with syscons console driver. >> >> /var/run/dmesg.boot starts with kernel messages without previous lines from the loader. >> Specifically, I need messages about loading kernel modules with full paths. > > loader cannot write to the kernel message buffer as there is no kernel yet. There might be possible ways to preserve that but I doubt loader is fit for that task. Some hardware does not clear memory while warm booting, so KDB trace from previous panic sometimes survive reboot and ends up in /var/run/dmesg.boot for successful boot. I wonder why loader messages cannot be there, too. In fact, kernel message buffer can be found even in absence of kernel :-) I've already searched through the sources in stand/ and see there is no code to support this yet. > A boot -v (boot_verbose=“YES”) might spit the information out from the kernel as well. Not sure if that helps you. However on modern server machines a boot -v often overflows the message buffer.. so you might have to tune that as well. > > Extracted from an older console log of a random lab machine of someone else: > .. > WARNING: WITNESS option enabled, expect reduced performance.^M^@ > WARNING: DIAGNOSTIC option enabled, expect reduced performance.^M^@ > Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff80e66000.^M^@ > Preloaded elf obj module "/boot/kernel/if_igb.ko" at 0xffffffff80e66968.^M^@ > Preloaded elf obj module "/boot/kernel/nfscl.ko" at 0xffffffff80e66fd0.^M^@ > Preloaded elf obj module "/boot/kernel/nfslock.ko" at 0xffffffff80e67638.^M^@ > Preloaded elf obj module "/boot/kernel/nfssvc.ko" at 0xffffffff80e67ba0.^M^@ > Preloaded elf obj module "/boot/kernel/krpc.ko" at 0xffffffff80e68088.^M^@ > Preloaded elf obj module "/boot/kernel/nfscommon.ko" at 0xffffffff80e686b0.^M^@ > Preloaded elf obj module "/boot/kernel/ahci.ko" at 0xffffffff80e68d20.^M^@ > Preloaded elf obj module "/boot/kernel/cam.ko" at 0xffffffff80e69308.^M^@ > Preloaded elf obj module "/boot/kernel/mfi.ko" at 0xffffffff80e699f0.^M^@ > Preloaded elf obj module "/boot/kernel/geom_part_gpt.ko" at 0xffffffff80e6a058.^M^@ > Preloaded elf obj module "/boot/kernel/sysvsem.ko" at 0xffffffff80e6a648.^M^@ > Preloaded elf obj module "/boot/kernel/sysvshm.ko" at 0xffffffff80e6acb0.^M^@ > Calibrating TSC clock ... TSC clock: 3400185886 Hz^M^@ > CPU: Intel(R) Xeon(R) CPU X5272 @ 3.40GHz (3400.19-MHz K8-class CPU)^M^@ Yes, but this only shows results of loader's work and kldstat -v shows paths too. I need to know why changing modules_path in the nextboot.conf does not work and why does loader uses stock zfs.ko insted of my patched version /boot/nextboot/zfs.ko if I use nextboot.conf with module_path="/boot/nextboot;/boot/kernel;/boot/modules" I have panicing production server (ZFS-only) and cannot reboot it often and I don't wont to make cycle-boot by mistake due to wrong debugging code I might add to the module. From owner-freebsd-stable@freebsd.org Thu Nov 21 11:35:54 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 468421BA862 for ; Thu, 21 Nov 2019 11:35:54 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mx.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47JcvD3Qtzz4GBd for ; Thu, 21 Nov 2019 11:35:52 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id xALBZnKC058979; Thu, 21 Nov 2019 11:35:49 GMT (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id xALBZlwU058978; Thu, 21 Nov 2019 03:35:47 -0800 (PST) (envelope-from david) Date: Thu, 21 Nov 2019 03:35:47 -0800 From: David Wolfskill To: Eugene Grosbein Cc: FreeBSD stable Subject: Re: Logging boot messages from the loader Message-ID: <20191121113547.GI34976@albert.catwhisker.org> Reply-To: stable@freebsd.org Mail-Followup-To: stable@freebsd.org, Eugene Grosbein , FreeBSD stable References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="tgQ0jN0vM+YIGDGO" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.2 (2019-09-21) X-Rspamd-Queue-Id: 47JcvD3Qtzz4GBd X-Spamd-Bar: ------- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of david@catwhisker.org designates 198.144.209.73 as permitted sender) smtp.mailfrom=david@catwhisker.org X-Spamd-Result: default: False [-7.89 / 15.00]; ARC_NA(0.00)[]; HAS_REPLYTO(0.00)[stable@freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:198.144.209.73]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[catwhisker.org]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[73.209.144.198.list.dnswl.org : 127.0.10.0]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:7961, ipnet:198.144.208.0/20, country:US]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-3.49)[ip: (-9.32), ipnet: 198.144.208.0/20(-4.46), asn: 7961(-3.62), country: US(-0.05)] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2019 11:35:54 -0000 --tgQ0jN0vM+YIGDGO Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Nov 21, 2019 at 04:51:13PM +0700, Eugene Grosbein wrote: > Hi! >=20 > How do I get boot messages generated by loader itself written to some log= after successfull boot? > This is generic BIOS-based 11.3-STABLE/amd64 system with syscons console = driver. >=20 > /var/run/dmesg.boot starts with kernel messages without previous lines fr= om the loader. > Specifically, I need messages about loading kernel modules with full path= s. > .... Perhaps: Set up serial console on the machine in question, then connect a machine that's expected to be "sufficiently stable" to the other end of a "null modem" cable & run tip(1) on the second machine (perhaps within script(1)). Or something similar, anyway. Peace, david --=20 David H. Wolfskill david@catwhisker.org Why is Trump trying to keep salient information from House investigators? See http://www.catwhisker.org/~david/publickey.gpg for my public key. --tgQ0jN0vM+YIGDGO Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGTBAEBCgB9FiEE4owz2QxMJyaxAefyQLJg+bY2PckFAl3WdpNfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEUy OEMzM0Q5MEM0QzI3MjZCMTAxRTdGMjQwQjI2MEY5QjYzNjNEQzkACgkQQLJg+bY2 PckHeQgAiIR2Xmro7yUybv5+SFHg2G3WKpw5+7ehmWB1y14FQxkti+AVsXYeRM5O fpTJ7FO1bmj9zYhQdxSsT5elIDwLKoaWWUzFZSwqiX/HliidBsRbqfNZmZ/aAmFD BBP+vBugHt7tHlL45SDB1V0LtrxAKPMBFOCbBw8vf6HVS5zcgaBhGwpTLhyZGwMV vIE8hB3H9edEJOjh4/D9Gv7iTtKird5B+rcvkqSdJJwowxRMMxAdqHG9GSBV79UX VvlqdH3HEQevuf0W1ouR7HcQyJ5gfIsQmeemigTQikX6sxtVIIHjIKfRLTTgUT1F EpQd1pakbxt520rBLWbL7Ncu/Bp9xA== =poeI -----END PGP SIGNATURE----- --tgQ0jN0vM+YIGDGO-- From owner-freebsd-stable@freebsd.org Thu Nov 21 11:52:58 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9E3371BB1B2; Thu, 21 Nov 2019 11:52:58 +0000 (UTC) (envelope-from michal@microwave.sk) Received: from daemon.microwave.sk (daemon.microwave.sk [217.144.16.208]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47JdGw6bj8z4HCg; Thu, 21 Nov 2019 11:52:56 +0000 (UTC) (envelope-from michal@microwave.sk) Received: from [192.168.0.128] (static-dsl-28.87-197-110.telecom.sk [87.197.110.28]) by daemon.microwave.sk (Postfix) with ESMTPSA id 09C74289B; Thu, 21 Nov 2019 12:52:48 +0100 (CET) From: =?utf-8?Q?Michal_Van=C4=8Do?= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3601.0.10\)) Subject: device_attach: ixv0 attach returned 5 Message-Id: <2D60E558-2F48-40EA-BF0D-C350BD005215@microwave.sk> Date: Thu, 21 Nov 2019 12:52:47 +0100 To: freebsd-net@freebsd.org, freebsd-stable@freebsd.org X-Mailer: Apple Mail (2.3601.0.10) X-Rspamd-Queue-Id: 47JdGw6bj8z4HCg X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.44 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[microwave.sk:s=mail]; NEURAL_HAM_MEDIUM(-0.98)[-0.981,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-0.97)[-0.975,0]; MV_CASE(0.50)[]; DKIM_TRACE(0.00)[microwave.sk:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[microwave.sk,quarantine]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(0.02)[country: SK(0.09)]; ASN(0.00)[asn:31127, ipnet:217.144.16.0/20, country:SK]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2019 11:52:58 -0000 Hi, I=E2=80=99m trying to get SR-IOV working with my two port Intel 10G NIC: ix0@pci0:3:0:0: class=3D0x020000 card=3D0x15ad15d9 chip=3D0x15ad8086 = rev=3D0x00 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Ethernet Connection X552/X557-AT 10GBASE-T' class =3D network subclass =3D ethernet ix1@pci0:3:0:1: class=3D0x020000 card=3D0x15ad15d9 chip=3D0x15ad8086 = rev=3D0x00 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Ethernet Connection X552/X557-AT 10GBASE-T' class =3D network subclass =3D ethernet with this iovctl config: # cat /etc/iovctl_ix1.conf=20 PF { device: "ix1"; num_vfs: 4; } I get 4 PCI devices created: none57@pci0:3:0:129: class=3D0x020000 card=3D0x15ad15d9 = chip=3D0x15a88086 rev=3D0x00 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Ethernet Connection X552 Virtual Function' class =3D network subclass =3D ethernet none58@pci0:3:0:131: class=3D0x020000 card=3D0x15ad15d9 = chip=3D0x15a88086 rev=3D0x00 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Ethernet Connection X552 Virtual Function' class =3D network subclass =3D ethernet none59@pci0:3:0:133: class=3D0x020000 card=3D0x15ad15d9 = chip=3D0x15a88086 rev=3D0x00 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Ethernet Connection X552 Virtual Function' class =3D network subclass =3D ethernet none60@pci0:3:0:135: class=3D0x020000 card=3D0x15ad15d9 = chip=3D0x15a88086 rev=3D0x00 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Ethernet Connection X552 Virtual Function' class =3D network subclass =3D ethernet But the driver fails to attach with following errors: ixv0: at device = 0.129 on pci4 ixv0: ...reset_hw() failure: Reset Failed! ixv0: IFDI_ATTACH_PRE failed 5 device_attach: ixv0 attach returned 5 ixv0: at device = 0.131 on pci4 ixv0: ...reset_hw() failure: Reset Failed! ixv0: IFDI_ATTACH_PRE failed 5 device_attach: ixv0 attach returned 5 ixv0: at device = 0.133 on pci4 ixv0: ...reset_hw() failure: Reset Failed! ixv0: IFDI_ATTACH_PRE failed 5 device_attach: ixv0 attach returned 5 ixv0: at device = 0.135 on pci4 ixv0: ...reset_hw() failure: Reset Failed! ixv0: IFDI_ATTACH_PRE failed 5 device_attach: ixv0 attach returned 5 I=E2=80=99m running 12-STABLE. Is this a hardware related or possibly a = driver bug? regards Michal From owner-freebsd-stable@freebsd.org Thu Nov 21 12:23:07 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 330FA1BCDB2 for ; Thu, 21 Nov 2019 12:23:07 +0000 (UTC) (envelope-from egypcio@gmail.com) Received: from mail-yb1-xb30.google.com (mail-yb1-xb30.google.com [IPv6:2607:f8b0:4864:20::b30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Jdxk202wz4K8t for ; Thu, 21 Nov 2019 12:23:05 +0000 (UTC) (envelope-from egypcio@gmail.com) Received: by mail-yb1-xb30.google.com with SMTP id q18so1318793ybq.6 for ; Thu, 21 Nov 2019 04:23:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=hMgJG4WwpQ7Yxs6OCwWW1G3xo+R/Ua7uy9KDgkP7x10=; b=XPIgRrqzLndKuuZfoTWqDmSpCBtE6/L1yF7hiAbPyjLG47EJ/T0CfqOFQhw7d3ihqB z96Ovt2RGHf+0x3k8WRd07joCOJGmTy2zeBKZQ9nyy9uXPANhslKyfIsn2r3nsBVbiO4 +Kj//4KDQHZcPfTGeMjzqu/hNeflKPqBE/2XrpojP3rO8CwkUH/UpUT/rVorYWHgjYp0 wPznvcslxBBCLxYL/suB1UH8a9M+KH1u2WgSknLFlHH7OfE7VK6S41/sVNYT9F5espau 6ueQkir+b37x78Yo8eDNulxroQjK4fkYjj/pG502sNPJQUBUxPYNIUHWaF1IkdItHDDj WFSg== X-Gm-Message-State: APjAAAWpfnYd8UZxtFaOOAtNTlwvOpEw6GCoVCMEnOny1LbpqAmFUFGt k2fBVZ8MEj3YkO8Q8veNmu3R2lF82VLWKnzqKUchgFFM X-Google-Smtp-Source: APXvYqx7OGvkMEybLctH/i2o0COTPTj9JMaZZf9PGPtkIZXenpvhNnzwElaG2rVR/r4ygmuotvUdJemugK6+/yuX2Wc= X-Received: by 2002:a25:d117:: with SMTP id i23mr5849641ybg.134.1574338984390; Thu, 21 Nov 2019 04:23:04 -0800 (PST) MIME-Version: 1.0 From: =?UTF-8?Q?Vin=C3=ADcius_Zavam?= Date: Thu, 21 Nov 2019 12:22:53 +0000 Message-ID: Subject: [NTP] "Unexpected origin timestamp 0xe180e455.d15bf3fb does not match aorg 0000000000.00000000 from server" (kernel reports TIME_ERROR: 0x41: Clock Unsynchronized) To: freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 47Jdxk202wz4K8t X-Spamd-Bar: - X-Spamd-Result: default: False [-1.70 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[googlemail.com]; TO_DN_NONE(0.00)[]; URI_COUNT_ODD(1.00)[5]; DKIM_TRACE(0.00)[googlemail.com:+]; DMARC_POLICY_ALLOW(-0.50)[googlemail.com,quarantine]; FORGED_SENDER(0.30)[egypcio@googlemail.com,egypcio@gmail.com]; IP_SCORE(0.00)[ip: (-9.28), ipnet: 2607:f8b0::/32(-2.29), asn: 15169(-1.97), country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[egypcio@googlemail.com,egypcio@gmail.com]; DWL_DNSWL_NONE(0.00)[googlemail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[googlemail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[0.3.b.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2019 12:23:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242137 --=20 Vin=C3=ADcius Zavam keybase.io/egypcio From owner-freebsd-stable@freebsd.org Thu Nov 21 12:43:30 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 198231BD759; Thu, 21 Nov 2019 12:43:30 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 47JfPD6Jrdz4LSJ; Thu, 21 Nov 2019 12:43:28 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id xALCh5vU056217 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 21 Nov 2019 12:43:06 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: stable@freebsd.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id xALCh2kL046985 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 21 Nov 2019 19:43:02 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Logging boot messages from the loader To: stable@freebsd.org, FreeBSD stable References: <20191121113547.GI34976@albert.catwhisker.org> From: Eugene Grosbein Message-ID: <6ed2e034-8864-0427-cca6-4e347cf34dfb@grosbein.net> Date: Thu, 21 Nov 2019 19:42:57 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20191121113547.GI34976@albert.catwhisker.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 47JfPD6Jrdz4LSJ X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.78 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.68)[ip: (-4.45), ipnet: 2a01:4f8::/29(-2.32), asn: 24940(-1.64), country: DE(-0.01)]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2019 12:43:30 -0000 21.11.2019 18:35, David Wolfskill wrote: >> How do I get boot messages generated by loader itself written to some log after successfull boot? >> This is generic BIOS-based 11.3-STABLE/amd64 system with syscons console driver. >> >> /var/run/dmesg.boot starts with kernel messages without previous lines from the loader. >> Specifically, I need messages about loading kernel modules with full paths. >> .... > > Perhaps: Set up serial console on the machine in question, then > connect a machine that's expected to be "sufficiently stable" to > the other end of a "null modem" cable & run tip(1) on the second > machine (perhaps within script(1)). > > Or something similar, anyway. The box is located in remote hoster's datacenter, so I was looking for easier way. Bad luck. From owner-freebsd-stable@freebsd.org Thu Nov 21 13:54:21 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 150081BF4FB for ; Thu, 21 Nov 2019 13:54:21 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Jgz06mCjz4QB9 for ; Thu, 21 Nov 2019 13:54:20 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from mail-qv1-f54.google.com (mail-qv1-f54.google.com [209.85.219.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id BFC8A7FE0 for ; Thu, 21 Nov 2019 13:54:20 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-qv1-f54.google.com with SMTP id g18so1402992qvp.8 for ; Thu, 21 Nov 2019 05:54:20 -0800 (PST) X-Gm-Message-State: APjAAAU7EYEmiEQ0vSDu6mX3ZdLXbEykG/4Cx3ybty5hvezWyKrB7zTq RitYQsSdrp04cJmsQhKWGNHN+X1Fop0/ohIulh0= X-Google-Smtp-Source: APXvYqwt3zSh7vhutFNqxT+rrmCboCehPmnLeqq+NAutz7WaHFVa4vKqRobpS41+zNOjHR6hvInbbEJcjxxDO2m7PCE= X-Received: by 2002:a0c:8e87:: with SMTP id x7mr8168189qvb.112.1574344460212; Thu, 21 Nov 2019 05:54:20 -0800 (PST) MIME-Version: 1.0 References: <4c45a371-c352-b8c4-65c6-d1bff55b4a2e@grosbein.net> In-Reply-To: <4c45a371-c352-b8c4-65c6-d1bff55b4a2e@grosbein.net> From: Kyle Evans Date: Thu, 21 Nov 2019 07:54:07 -0600 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Logging boot messages from the loader To: Eugene Grosbein Cc: "Bjoern A. Zeeb" , FreeBSD stable Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2019 13:54:21 -0000 On Thu, Nov 21, 2019 at 4:48 AM Eugene Grosbein wrote: > > 21.11.2019 17:22, Bjoern A. Zeeb wrote: > > > On 21 Nov 2019, at 9:51, Eugene Grosbein wrote: > > > >> Hi! > >> > >> How do I get boot messages generated by loader itself written to some = log after successfull boot? > >> This is generic BIOS-based 11.3-STABLE/amd64 system with syscons conso= le driver. > >> > >> /var/run/dmesg.boot starts with kernel messages without previous lines= from the loader. > >> Specifically, I need messages about loading kernel modules with full p= aths. > > > > loader cannot write to the kernel message buffer as there is no kernel = yet. There might be possible ways to preserve that but I doubt loader is f= it for that task. > > Some hardware does not clear memory while warm booting, so KDB trace from= previous panic sometimes survive reboot > and ends up in /var/run/dmesg.boot for successful boot. I wonder why load= er messages cannot be there, too. > In fact, kernel message buffer can be found even in absence of kernel :-) > > I've already searched through the sources in stand/ and see there is no c= ode to support this yet. > > > A boot -v (boot_verbose=3D=E2=80=9CYES=E2=80=9D) might spit the informa= tion out from the kernel as well. Not sure if that helps you. However on= modern server machines a boot -v often overflows the message buffer.. so y= ou might have to tune that as well. > > > > Extracted from an older console log of a random lab machine of someone = else: > > .. > > WARNING: WITNESS option enabled, expect reduced performance.^M^@ > > WARNING: DIAGNOSTIC option enabled, expect reduced performance.^M^@ > > Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff80e66000.^M^@ > > Preloaded elf obj module "/boot/kernel/if_igb.ko" at 0xffffffff80e66968= .^M^@ > > Preloaded elf obj module "/boot/kernel/nfscl.ko" at 0xffffffff80e66fd0.= ^M^@ > > Preloaded elf obj module "/boot/kernel/nfslock.ko" at 0xffffffff80e6763= 8.^M^@ > > Preloaded elf obj module "/boot/kernel/nfssvc.ko" at 0xffffffff80e67ba0= .^M^@ > > Preloaded elf obj module "/boot/kernel/krpc.ko" at 0xffffffff80e68088.^= M^@ > > Preloaded elf obj module "/boot/kernel/nfscommon.ko" at 0xffffffff80e68= 6b0.^M^@ > > Preloaded elf obj module "/boot/kernel/ahci.ko" at 0xffffffff80e68d20.^= M^@ > > Preloaded elf obj module "/boot/kernel/cam.ko" at 0xffffffff80e69308.^M= ^@ > > Preloaded elf obj module "/boot/kernel/mfi.ko" at 0xffffffff80e699f0.^M= ^@ > > Preloaded elf obj module "/boot/kernel/geom_part_gpt.ko" at 0xffffffff8= 0e6a058.^M^@ > > Preloaded elf obj module "/boot/kernel/sysvsem.ko" at 0xffffffff80e6a64= 8.^M^@ > > Preloaded elf obj module "/boot/kernel/sysvshm.ko" at 0xffffffff80e6acb= 0.^M^@ > > Calibrating TSC clock ... TSC clock: 3400185886 Hz^M^@ > > CPU: Intel(R) Xeon(R) CPU X5272 @ 3.40GHz (3400.19-MHz K8-cl= ass CPU)^M^@ > > Yes, but this only shows results of loader's work and kldstat -v shows pa= ths too. > I need to know why changing modules_path in the nextboot.conf does not wo= rk > and why does loader uses stock zfs.ko insted of my patched version /boot/= nextboot/zfs.ko > if I use nextboot.conf with module_path=3D"/boot/nextboot;/boot/kernel;/b= oot/modules" > > I pointed this out on IRC, but for list's sake: for something like this, it's good to verify module_path that loader used by inspecting kenv. This is my guess at the output: $ kenv module_path /boot/kernel;/boot/nextboot;/boot/kernel;/boot/modules With lualoader, it would be slightly different as we go through the cosmetic exercise of removing the loaded kernel path from the middle of module_path if it's there. Both 4th and lua will prepend the kernel path to module_path once the kernel's loaded, which should always happen after nextboot processing. For anyone else doing the same, the best option here would likely be fully qualifying the module names for every module you want out of a separate path: zfs_name=3D"/boot/nextboot/zfs.ko" This should/will work with both 4thloader and lualoader. It's not ideal for general configuration, but if you're doing a one-off load of a module in loader it works. From owner-freebsd-stable@freebsd.org Thu Nov 21 18:10:46 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 58F891C6B03; Thu, 21 Nov 2019 18:10:46 +0000 (UTC) (envelope-from michal@microwave.sk) Received: from daemon.microwave.sk (daemon.microwave.sk [IPv6:2a01:108:1:1001::d0]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47Jnfr4g5Nz3FSR; Thu, 21 Nov 2019 18:10:44 +0000 (UTC) (envelope-from michal@microwave.sk) Received: from [192.168.0.128] (static-dsl-28.87-197-110.telecom.sk [87.197.110.28]) by daemon.microwave.sk (Postfix) with ESMTPSA id F2C712A9F; Thu, 21 Nov 2019 19:10:40 +0100 (CET) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3601.0.10\)) Subject: Re: device_attach: ixv0 attach returned 5 From: =?utf-8?Q?Michal_Van=C4=8Do?= In-Reply-To: Date: Thu, 21 Nov 2019 19:10:40 +0100 Cc: freebsd-net@freebsd.org, freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <01EC7302-24A6-4441-85A2-4D24C97696ED@microwave.sk> References: <2D60E558-2F48-40EA-BF0D-C350BD005215@microwave.sk> To: Richard Gallamore X-Mailer: Apple Mail (2.3601.0.10) X-Rspamd-Queue-Id: 47Jnfr4g5Nz3FSR X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.41 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[microwave.sk:s=mail]; NEURAL_HAM_MEDIUM(-0.98)[-0.985,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-0.95)[-0.948,0]; TO_DN_SOME(0.00)[]; DKIM_TRACE(0.00)[microwave.sk:+]; DMARC_POLICY_ALLOW(-0.50)[microwave.sk,quarantine]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(0.02)[country: SK(0.09)]; ASN(0.00)[asn:31127, ipnet:2a01:108::/32, country:SK]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Nov 2019 18:10:46 -0000 Well then this is really annoying. I can image few other applications = besides virtualization. Jails using vnet bound to VF instead of epair being one = of=20 examples. Any hope that someone will port the SR-IOV from Intel=E2=80=99s = code into the base driver on foreseeable future? regards Michal > On 21 Nov 2019, at 18:52, Richard Gallamore = wrote: >=20 > Hello Michal, >=20 >> I=E2=80=99m running 12-STABLE. Is this a hardware related or possibly = a driver > bug? > This is probably a driver / module bug. >=20 > [1] is a bug on this issue, that I opened some years ago. The last = time > I tested the sr-iov functionality it was working if you compile the = intel > module with sr-iov support enabled but not with the base module. This > was probably a couple years ago though and on 12-CURRENT before > 12 was released. >=20 > Personally I suggest you just avoid sr-iov though and use bridges. = sr-iov > was > a cool idea but it just failed to get an attraction. Probably several > reasons > for this, based on my experience I would say the primary reason is = because > its designed use case is for virtualization, and if you do use this = for > virtualization, you trade the ability to do live migrations due to = using > physical hardware. >=20 > Hope this helps, > Richard Gallamore >=20 > On Thu, Nov 21, 2019 at 3:53 AM Michal Van=C4=8Do via freebsd-stable < > freebsd-stable@freebsd.org> wrote: >=20 >> Hi, >>=20 >> I=E2=80=99m trying to get SR-IOV working with my two port Intel 10G = NIC: >>=20 >> ix0@pci0:3:0:0: class=3D0x020000 card=3D0x15ad15d9 chip=3D0x15ad8086 = rev=3D0x00 >> hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'Ethernet Connection X552/X557-AT 10GBASE-T' >> class =3D network >> subclass =3D ethernet >> ix1@pci0:3:0:1: class=3D0x020000 card=3D0x15ad15d9 chip=3D0x15ad8086 = rev=3D0x00 >> hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'Ethernet Connection X552/X557-AT 10GBASE-T' >> class =3D network >> subclass =3D ethernet >>=20 >> with this iovctl config: >>=20 >> # cat /etc/iovctl_ix1.conf >> PF { >> device: "ix1"; >> num_vfs: 4; >> } >>=20 >> I get 4 PCI devices created: >>=20 >> none57@pci0:3:0:129: class=3D0x020000 card=3D0x15ad15d9 = chip=3D0x15a88086 >> rev=3D0x00 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'Ethernet Connection X552 Virtual Function' >> class =3D network >> subclass =3D ethernet >> none58@pci0:3:0:131: class=3D0x020000 card=3D0x15ad15d9 = chip=3D0x15a88086 >> rev=3D0x00 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'Ethernet Connection X552 Virtual Function' >> class =3D network >> subclass =3D ethernet >> none59@pci0:3:0:133: class=3D0x020000 card=3D0x15ad15d9 = chip=3D0x15a88086 >> rev=3D0x00 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'Ethernet Connection X552 Virtual Function' >> class =3D network >> subclass =3D ethernet >> none60@pci0:3:0:135: class=3D0x020000 card=3D0x15ad15d9 = chip=3D0x15a88086 >> rev=3D0x00 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'Ethernet Connection X552 Virtual Function' >> class =3D network >> subclass =3D ethernet >>=20 >> But the driver fails to attach with following errors: >>=20 >> ixv0: at device = 0.129 >> on pci4 >> ixv0: ...reset_hw() failure: Reset Failed! >> ixv0: IFDI_ATTACH_PRE failed 5 >> device_attach: ixv0 attach returned 5 >> ixv0: at device = 0.131 >> on pci4 >> ixv0: ...reset_hw() failure: Reset Failed! >> ixv0: IFDI_ATTACH_PRE failed 5 >> device_attach: ixv0 attach returned 5 >> ixv0: at device = 0.133 >> on pci4 >> ixv0: ...reset_hw() failure: Reset Failed! >> ixv0: IFDI_ATTACH_PRE failed 5 >> device_attach: ixv0 attach returned 5 >> ixv0: at device = 0.135 >> on pci4 >> ixv0: ...reset_hw() failure: Reset Failed! >> ixv0: IFDI_ATTACH_PRE failed 5 >> device_attach: ixv0 attach returned 5 >>=20 >> I=E2=80=99m running 12-STABLE. Is this a hardware related or possibly = a driver bug? >>=20 >> regards >> Michal >>=20 >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" >>=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"