From owner-freebsd-virtualization@freebsd.org Sun May 16 21:01:07 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A624863133B for ; Sun, 16 May 2021 21:01:07 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4FjvnG4chpz3rG8 for ; Sun, 16 May 2021 21:01:06 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.nyi.freebsd.org (Postfix) id 1303F631242; Sun, 16 May 2021 21:01:05 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 122856312BD for ; Sun, 16 May 2021 21:01:05 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FjvnC5clcz3rFw for ; Sun, 16 May 2021 21:01:03 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0B5691F96F for ; Sun, 16 May 2021 21:00:51 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 14GL0oUx058077 for ; Sun, 16 May 2021 21:00:50 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 14GL0oMq058076 for virtualization@FreeBSD.org; Sun, 16 May 2021 21:00:50 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202105162100.14GL0oMq058076@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: virtualization@FreeBSD.org Subject: Problem reports for virtualization@FreeBSD.org that need special attention Date: Sun, 16 May 2021 21:00:50 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 May 2021 21:01:08 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- In Progress | 247208 | mpt(4): VMWare virtualized LSI controller panics New | 240945 | [hyper-v] [netvsc] hn network driver incorrectly 2 problems total for which you should take action. From owner-freebsd-virtualization@freebsd.org Mon May 17 04:12:26 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EF97A63B512 for ; Mon, 17 May 2021 04:12:26 +0000 (UTC) (envelope-from weh@microsoft.com) Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-eopbgr1310111.outbound.protection.outlook.com [40.107.131.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fk5Lx2SvTz4RJr; Mon, 17 May 2021 04:12:25 +0000 (UTC) (envelope-from weh@microsoft.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lSQgFhyk/EN/MkG3jVTZexsBgcawARrc1SMUuJ6T9QDQv4S1nxnWtFIbPu+On0zIptGcTNSJo/Lp+Jkomnk01jzYpc7C6DXAL6bnBxipCxVI53g9XeGJ0r+lVKA931nOCf62ceII88/HmOpvco4drc+CTROj9rOhyg2OoSFMB7H76QJyat0T2LeFBf6O3JODXuCm/yIL6Y3EZDSx6xsONZjUR5UuB/4flK+WmFdiTxoJctlnXVJHhE3E+9yH5+YfsRELC3vkhIDQaSe2hrD7bMei0ev5WRuPeKMOkeFWMgM+qgwKMbWckT0waSRI3d/vbywLNojgoIP7q72viQ9hEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ox5LpFQYHyMIoxUY9aXAdlEDv4/3HbSiSRpmcwFSZ8c=; b=jOXKfCElyJOpb8g4n86YIVDAq+EG0D4tWZ1bqb/74qQMtjvkJYQ+JTWxSMfj2nw2NtGpxAnoZI8c9hF1IqPzNlJKOrVElDBt4ZmmWthjuNgpz/2SOMJNUnYR+MrJ3KKZUcBNNPX0UXmMRcnRiXKz2EtVXYNNk3NfhmmJE1Haw90HabUGx7qiyWJ28/73HERiOA7BqGCeivPpD9etn5Q+rG3hOpo7HKAIIWyGatrwPtL74Ju+C6EjaH818ZCrn5JqAAhdSZXfT1aam6C4FC61aA/HUMmSHEAzyeZJjXDk5QSdHgR6AMUEbWdhGQJFpsAu2+k3y/LNuiIu4jTWyUC8Rw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none Received: from PSAP153MB0440.APCP153.PROD.OUTLOOK.COM (2603:1096:301:35::9) by PSAP153MB0487.APCP153.PROD.OUTLOOK.COM (2603:1096:301:75::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.2; Mon, 17 May 2021 04:12:20 +0000 Received: from PSAP153MB0440.APCP153.PROD.OUTLOOK.COM ([fe80::a1:5809:9657:cfdf]) by PSAP153MB0440.APCP153.PROD.OUTLOOK.COM ([fe80::a1:5809:9657:cfdf%5]) with mapi id 15.20.4173.007; Mon, 17 May 2021 04:12:20 +0000 From: Wei Hu To: Harry Schmalzbauer , Chris Knight , "freebsd-virtualization@freebsd.org" CC: Konstantin Belousov Subject: RE: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root Thread-Topic: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root Thread-Index: AQHWot5T3VubDGm01EyXk9C1ey9JFqmZ+PXwgUqCJACAAbbYgIACKD5A Date: Mon, 17 May 2021 04:12:20 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=d3caac43-4735-45c0-80e3-661cb388e253; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-05-17T03:51:48Z; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; x-originating-ip: [114.84.172.152] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 0248bdf8-fa54-4245-6916-08d918e9f75a x-ms-traffictypediagnostic: PSAP153MB0487: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2201; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 4biQ7/c3WgrUWrOcSxw1Q/WuOIjGhsRpVMxhjuHkIPykQc92EIDjiPwB1SVqyh83bMy77zTFSzQFbOl8V8UECy6qkZQZ6Hp3uIcAuq05N5qym7h2XfKHTFO5q8GwMI5kmgPBtl+a8GfklBrqdWXck+qpWIyFG+wN4g1qgOf2Yu8khcTMBhX5fWnquCCLCwNGv3kJiUjApz0QhLr5Bvw5kLeHlxZxTpbGscyaDZMUSnNUe+9pUPndq69i4UfWHuXuz1c9S2Uy4FK1amRkFychOIdp2tKhX7ts8W2ki+0wk9poz2EFQUNvB1JFnhf7oL6xmOZWKbkq1291440X1bl6i5GB0leBzK60Ud4IQ9/V1Sab5S//JN6uWkkeCFw52QFo3yKxcacQyPbHwYrAvnLRDBMCbf8BXpr1uMjCWNK9lZHwHgulHu3FcFCW2Bn5nfgWKBztHLBKhm2txJYXV9WrLkes84QzSMhDWxyQfm4sBReEHqSZWpyPG2kMb9KCieMbfudEvghMy91Q1iK42QBOJIpXlBEmKxOUeoUNs0Kms0UhcC+x5Gh8845qTESsT17E3x42R9zH3Ppstidn4u2gCtpZhAjnmEdXun2oO89ueE8TD+Qc/0YyFDSxv2rP5FzJue7ow90mhrYTD3/F+5wp40E0DpK2KBds+FiniDAcqYK/K3mzbHOYX0QeBjK4ejWJ9/G/3o+4pIoAPr84RdMloQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PSAP153MB0440.APCP153.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(4636009)(366004)(71200400001)(4326008)(8936002)(5660300002)(8676002)(52536014)(66476007)(66446008)(66946007)(66556008)(55016002)(82960400001)(82950400001)(53546011)(478600001)(33656002)(83380400001)(9686003)(86362001)(186003)(6506007)(8990500004)(966005)(38100700002)(316002)(110136005)(76116006)(10290500003)(26005)(7696005)(122000001)(2906002)(64756008); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: =?utf-8?B?Z2Jta2ExSXpRTFM1OGM5NVc5UHZjMXNPTlBVYWhhbGt3MlFFbjFUU3RwWUNS?= =?utf-8?B?ajBDcVdUQytJeklUbGQxMEd2ZHBuZm5sak9rYkh5MG0yeEdzaTI3N2NROHZY?= =?utf-8?B?SUVOblo2MlVwc3Y3VndQVFJ1WTBzM2xwZm9zWHJWc2tWZ3JvZm53VFN0V3JI?= =?utf-8?B?Rysrc0hSYndITVZMcklTMDVGQlg4aWVuMXhoZXFOQjFzWGh1dnFpdldJZlQy?= =?utf-8?B?U0VYWCtjL1lyeDYzcWVzZEFDc05hLzVrSXZBRkhQbFNZcUN3dU1mZjQ3aGUy?= =?utf-8?B?THpoN0ZadlNlL0Z4Nk5tam1uR29QTWJ4UmdpWXoyMGppWm1LZTdqSElLTEJs?= =?utf-8?B?Q1A1SXRwNE5KSWVMbFRiNEIxUXFSWEFTcGpVaXRNRmtvY09idmc3dzR2YVlq?= =?utf-8?B?VHJSTjgvbkNna2kwQ1lDNHpiNGxtSUxYblhxeXJnemlibFI3bWNOMWxyMW8r?= =?utf-8?B?Q0oraUlZYVBHbnFNUFdsQTJ5TVBXTkVUc3JmWmJxUjI5Q25POFR2TE9yR1ZE?= =?utf-8?B?SWp1LzRvWkNFUVNYYXV3VWVlcFlmOWNsTzBMampBQ3Btb0wzNmQ5RCsvNk0v?= =?utf-8?B?Qkp4bmtOc0U4ZkphbXRmTWFXWGorZE0yYWUveTRtZUs2bldBNXJ6eG12akNI?= =?utf-8?B?eGRuZWIvWWE0YitZN09yOFdXN1RCMEx4R3dIZkNXUWJxV0F5SG9QS0Z4eWhs?= =?utf-8?B?ZUc4NzVIbXRuM25MclZTQ0VmTzRreThuaXNmVGxSajVmaXd0QW9TVmtQSjNw?= =?utf-8?B?UmV3NlVGN1FnUU01cVVOZnVMY25MaEhRZEZvNGVxVWR1T0VyeVNWOWdTRU9j?= =?utf-8?B?MWUrNmxub1FUMWxvM05hYkM3b2xJdEwrVXdZRk9tR3dKLzdxZlFoU1FuYysx?= =?utf-8?B?NTZ0RzVpQnJyQmZxSXdLemc1QnJJNy9iT2dJYkd1WGJycHBIQWF5aEgxSldx?= =?utf-8?B?dXFHeFJaeVVkNzhCUWpkSXdSdXZPVHltdi9YWk1CSzR3UWQrUlFqMTlNUGwy?= =?utf-8?B?eU5wNU9KZlkzOGVhQW1UN1pMVWl2aWRYaisxbWwxeGh0MzZ3eExvU2wvWmk2?= =?utf-8?B?UUlUR0h5WlgvRTkyUngrN0YvQmFqTEFGMUg5dDR3OEhoZGhCd1ZscDRydmtY?= =?utf-8?B?S3RHbCtVT3gwR3FxdklJY0o5em1sck9qZnhudWJDOWZzMjJoc2JQZTBUTnl0?= =?utf-8?B?VUZHYWUvNlNzeEtkTm90VitwOFI5cHFOTkV5VWF5UXNkaUFmemR2NGdiUURr?= =?utf-8?B?OHdCYUt2d0RrRXl3bENjejdDQWZqTzJwTzhBN3VsNldVeFByREJVWDlkS2NZ?= =?utf-8?B?S2VXUUhxS2VlM0FkSU1yYkpvTnVsNjV1bitndUpBcUdyL002a29BdERBRWVY?= =?utf-8?B?RTJSYzM3a253eUlwS1BmaUR0YkJablRJR3ZsOHY2RnFmMFRKT1ZnTGF3dGVF?= =?utf-8?B?dEVYTGdzODZnZTJ6c09LQ1kyNWVXMXpMQU0zSmxYUWlDZmxNSSswVnZFbHVk?= =?utf-8?B?M3lKVzFac09rV3lZdDBiMEVVbGxpdndFL1lxMWhEd1g4aDFzcFdwQ1g2VVho?= =?utf-8?B?NDMxTEFmaUVwK2VaMUU3TWxiTmMxTExXVkVCNTF6ZWFvVkVsYmVqNElyUWNX?= =?utf-8?B?MlNwNGYwRXQrRExMbEc2SXRsV2JORmF5ZlBBR1lFZ0d6K2NhY3ZMN29zb0FG?= =?utf-8?B?ZHQvOURFUUo1ODFnMkdjWWo3Y2MyK2U0ZmRvdEZlMXpkaXQyb25VelJjcXYv?= =?utf-8?Q?CYm95KUbm2CDPEE8ik1r5CB1R8rtWMp+9Yz3kwm?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PSAP153MB0440.APCP153.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 0248bdf8-fa54-4245-6916-08d918e9f75a X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2021 04:12:20.2355 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: UnxjGo/XjiWGZgWjAzGqaR2QqTi4PomXDf63LZRkGagi7W9I7jTVWnyoT/1w6KGKR8419bzWrnunJoP7tzSwmw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PSAP153MB0487 X-Rspamd-Queue-Id: 4Fk5Lx2SvTz4RJr X-Spamd-Bar: -------- X-Spamd-Result: default: False [-8.00 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; HAS_XOIP(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[microsoft.com:+]; MIME_BASE64_TEXT(0.10)[]; DMARC_POLICY_ALLOW(-0.50)[microsoft.com,reject]; FREEMAIL_TO(0.00)[omnilan.de,gmail.com,freebsd.org]; RCVD_IN_DNSWL_LOW(-0.10)[40.107.131.111:from]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[40.107.131.111:from]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[microsoft.com:s=selector2]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; DWL_DNSWL_MED(-2.00)[microsoft.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_SHORT(1.00)[1.000]; SPAMHAUS_ZRD(0.00)[40.107.131.111:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.131.111:from]; WHITELIST_SPF_DKIM(-3.00)[microsoft.com:d:+,microsoft.com:s:+]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2021 04:12:27 -0000 Q29weSBLaWIgIHdobyB3YXMgdGhlIGF1dGhvciBvZiBQVEkgYW5kIG90aGVyIHJlbGF0ZSBwYXRj aGVzLg0KDQpUaGlzIGxvb2tzIHRvIGJlIGNhdXNlZCBieSBNZWx0ZG93biBhbmQgU3BlY3RyZSBt aXRpZ2F0aW9uIHBhdGNoZXMgd2hpY2ggd2VyZSANCmNoZWNrZWQgaW50byBoZWFkIGNvdXBsZSB5 ZWFycyBhZ28uIFNlZSB0aGlzIGxpbmsgZm9yIGRldGFpbHMgYWJvdXQgdGhlc2UgcGF0Y2hlczoN Cmh0dHBzOi8vd2lraS5mcmVlYnNkLm9yZy9TcGVjdWxhdGl2ZUV4ZWN1dGlvblZ1bG5lcmFiaWxp dGllcw0KDQpUaGUgYW1kNjQgIHBhdGNoZXMgYWxzbyBjYXVzZWQgcHJvYmxlbXMNCm9uIEh5cGVy LVYgKHRyaXBsZSBmYXVsdCkgd2hlbiB2bWJ1cyBhbmQgb3RoZXIgSHlwZXJWIGRyaXZlcnMgKG5l dHZzYywgc3RvcnZzYyBldGMpDQp3ZXJlIGxvYWRlZCBzZXBhcmF0ZWx5LiBUaGUgd29ya2Fyb3Vu ZCBmb3IgdGhpcyBvbiBhbWQ2NCBpcyBlaXRoZXIgYnVpbGQgdGhlc2UNCkh5cGVyLVYgbW9kdWxl cyBpbnRvIHRoZSBrZXJuZWwgb3IgY29tcGxldGVseSBkaXNhYmxlIG1pdGlnYXRpb24gYnkNCnNl dHRpbmcgdm0ucG1hcC5wdGk9IjAiIGluIC9ib290L2xvYWRlci5jb25mIGluIHRoZSBhZmZlY3Qg RnJlZUJTRCBndWVzdHMuDQoNCkl0IGxvb2tzIHRoZSBwcm9ibGVtIG9uIGkzODYgZ3Vlc3RzIGlz IGRpZmZlcmVudC4gV2hpbGUgbXVsdGlwbGUgcGF0Y2hlcyBoYXZlIGJlZW4gDQpjaGVja2VkIGlu IGZvciBhbWQ2NCwgdGhlcmUgd2FzIG9ubHkgb25lIHBhdGNoIHIzMzI0ODkgZm9yIGkzODYuICBJ IGFtIG5vdCBzdXJlDQp3aGV0aGVyIHRoZXJlIGlzIHNpbWlsYXIgdm0ucG1hcC5wdGkgb3B0aW9u IHRvIGRpc2FibGUgdGhlIHBhdGNoIGZvciBpMzg2IG9yIG5vdC4NCg0KSSBhbSBub3QgZmFtaWxp YXIgdG8gdGhlc2UgcGF0Y2hlcy4gDQoNCkAgS2liLCB3aWxsIHlvdSBiZSBhYmxlIHRvIGNvbW1l bnQgb24gdGhpcz8NCg0KVGhhbmtzLA0KV2VpDQoNCg0KDQo+IC0tLS0tT3JpZ2luYWwgTWVzc2Fn ZS0tLS0tDQo+IEZyb206IEhhcnJ5IFNjaG1hbHpiYXVlciA8ZnJlZWJzZEBvbW5pbGFuLmRlPg0K PiBTZW50OiBTdW5kYXksIE1heSAxNiwgMjAyMSAyOjU1IEFNDQo+IFRvOiBXZWkgSHUgPHdlaEBt aWNyb3NvZnQuY29tPjsgQ2hyaXMgS25pZ2h0IDxzdHJ5cXhAZ21haWwuY29tPjsgZnJlZWJzZC0N Cj4gdmlydHVhbGl6YXRpb25AZnJlZWJzZC5vcmcNCj4gU3ViamVjdDogUmU6IEZyZWVCU0QgMTIu MSBpMzg2IG9uIEh5cGVyLVYgRmFpbHMgdG8gTW91bnQgUm9vdA0KPiANCj4gQW0gMTQuMDUuMjAy MSB1bSAxODo0NCBzY2hyaWViIEhhcnJ5IFNjaG1hbHpiYXVlcjoNCj4gPiBBbSAxNi4xMC4yMDIw IHVtIDExOjM1IHNjaHJpZWIgV2VpIEh1IHZpYSBmcmVlYnNkLXZpcnR1YWxpemF0aW9uOg0KPiA+ Pj4gOg0KPiA+Pj4NCj4gPj4gRGlkIHlvdSB0cnkgMTIuMD8gTWF5YmUgc29tZXRoaW5nIGNoYW5n ZWQgaW4gMTIueCBjYXVzZXMgZmFpbHVyZS4NCj4gPj4gSTM4NiBpcyBub3Qgd2VsbCB0ZXN0ZWQg b24gSHlwZXItVi4gSSB3aWxsIHRyeSB0byByZXByb2R1Y2UgYW5kIHRha2UNCj4gPj4gYSBsb29r Lg0KPiA+DQo+ID4gVGhlIHJlZ3Jlc3Npb24gaGFwcGVuZWQgYmVmb3JlIDEyLjAuDQo+ID4gVG9k YXkgSSBzdHVtYmxlZCBhY3Jvc3MgdGhpcyBwcm9ibGVtIHRvbyBhbmQgdHJpZWQgMTEuNCAoL01h eSA4dGgsDQo+ID4gMjAyMCkgL3N1Y2Vzc2Z1bGx5LCB3aGlsZSAxMi4wICgvT2N0IDE5dGgsIDIw MTgpLyBpcyBmYWlsaW5nIHdpdGggaTM4Ng0KPiA+IHJlbGVhc2VzLg0KPiA+DQo+ID4gT25lIGNv bW1pdCB3aGljaCBpcyBpbiAxMiBidXQgbm90IGluIDExLjQgaXMgcjMzMjQ4OS4NCj4gPg0KPiA6 DQo+IDoNCj4gOg0KPiA+IFVuZm9ydHVuYXRlbHkgYmV5b25kIG15IHNraWxscy4NCj4gDQo+IElu IHN5cy94ODYveDg2L2xvY2FsX2FwaWMuYywgSSBhZGRlZCB0aGlzIGRlYnVnIHByaW50ZjoNCj4g IMKgwqDCoMKgwqDCoMKgIGZvciAoaWR4ID0gSVBJX0RZTl9GSVJTVDsgaWR4IDw9IElQSV9EWU5f TEFTVDsgaWR4KyspIHsNCj4gcHJpbnRmKCJERUJVRzogbmF0aXZlX2xhcGljX2lwaV9hbGxvYygp LCBpZHggPSAlaVxuIiwgaWR4KTsNCj4gIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBp cCA9ICZpZHRbaWR4XTsNCj4gIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBmdW5jID0g KGlwLT5nZF9oaW9mZnNldCA8PCAxNikgfCBpcC0+Z2RfbG9vZmZzZXQ7DQo+ICDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgaWYgKCghcHRpICYmIGZ1bmMgPT0gKHVpbnRwdHJfdCkmSURU VkVDKHJzdmQpKSB8fA0KPiAgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg KHB0aSAmJiBmdW5jID09ICh1aW50cHRyX3QpJklEVFZFQyhyc3ZkX3B0aSkpKSB7DQo+ICDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHZlY3RvciA9IGlkeDsN Cj4gIMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgc2V0aWR0 KHZlY3RvciwgaXBpZnVuYywgU0RUX0FQSUMsIFNFTF9LUEwsIEdTRUxfQVBJQyk7DQo+ICDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGJyZWFrOw0KPiAgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIH0NCj4gIMKgwqDCoMKgwqDCoMKgIH0NCj4gDQo+ IFRoaXMgaXMgbXkgbGF0ZXN0IGF0dGVtcHQgdG8gc2VlIGlmIGF0IGxlYXN0IHRoZSBvcmlnaW4g b2YgbXkgZmF1bHQgdHJhY2luZyBtYXRjaGVzLg0KPiBJdCBkb2VzLg0KPiANCj4gIEZyb20gI2Rl ZmluZSBJUElfRFlOX0ZJUlNUwqDCoCAoQVBJQ19JUElfSU5UUyArIDkpIGluIHN5cy94ODYvaW5j bHVkZS9hcGljdmFyLmgNCj4gSSBleHBlY3RlZCB0byBzZWUgMjUyLCAyNTMgYW5kIDI1NCwgd2hp Y2ggSSBzdWNlc3NmdWxseSB2ZXJpZmllZC4NCj4gDQo+IE5vdyBjb21lcyB0aGUgcGFydCBJIGNh bm5vdCByZWFkOiAodWludHB0cl90KSZJRFRWRUMocnN2ZCkNCj4gSURUVkVDKHJzdmQpIHNlZW1z IHRvIGJlIGEgYXNzZW1ibGVyIG1ha3JvIGZyb20gc3lzL2kzODYvaTM4Ni9leGNlcHRpb24ucw0K PiANCj4gQnV0IG5vdyBpdCBkb2Vzbid0IG1ha2UgYW55IHNlbnNlIGZvciBtZSB0byBkaWcgZnVy dGhlciwgSSBzaW1wbHkgZG8gbm90DQo+IHVuZGVyc3RhbmQgaXQuDQo+IA0KPiBVbmZvcnR1bmF0 ZWx5IGhhdmluZyBicm9rZW4gaTM4NiBzdXBwb3J0IG9uIEh5cGVyLVYgaXMgbm90IGEgY29zbWV0 aWMgaXNzdWUuDQo+IFJlbW92aW5nICJkZXZpY2UgaHlwZXJ2IiBmcm9tIGtlcm5lbCBnaXZlcyBh Y2Nlc3MgdG8gdGhlIHYtaGRkIHZpYSBsZWdhY3kNCj4gSURFL2FkYSwgYnV0IHRoZW4gSSBoYXZl IG5vIE5JQy4NCj4gQ2hvb3NpbmcgYmV0d2VlbiBtaXNzaW5nIEhERCBhbmQgbWlzc2luZyBuZXR3 b3JrIHN1cHBvcnQgaXMgaGFyZA0KPiBub3dhZGF5cyA7LSkNCj4gDQo+IEFueSBoZWxwIGdldHRp bmcgaTM4NiB1cCBhbmQgcnVubmluZyBiYWNrIG9uIEh5cGVyLVYgaGlnaGx5IGFwcHJlY2lhdGVk IcKgIDExLjQNCj4gaXMgdGhlIGxhdGVzdCB3b3JraW5nIHZlcnNpb24sIGFsbCBpMzg2LXJlbGVh c2VzIHN0YXJ0aW5nIHdpdGggMTIuMCBkb24ndCB3b3JrIQ0KPiANCj4gVGhhbmtzLA0KPiAtaGFy cnkNCg0K From owner-freebsd-virtualization@freebsd.org Mon May 17 05:47:30 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 74D1D63D0F0 for ; Mon, 17 May 2021 05:47:30 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fk7Sd1Y5Rz4dWx for ; Mon, 17 May 2021 05:47:28 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 14H5lBAb027500 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 17 May 2021 08:47:14 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 14H5lBAb027500 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 14H5lBdt027499; Mon, 17 May 2021 08:47:11 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 17 May 2021 08:47:11 +0300 From: Konstantin Belousov To: Wei Hu Cc: Harry Schmalzbauer , Chris Knight , "freebsd-virtualization@freebsd.org" Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4Fk7Sd1Y5Rz4dWx X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-2.91 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; FREEMAIL_FROM(0.00)[gmail.com]; NEURAL_HAM_LONG(-1.00)[-0.995]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; SPAMHAUS_ZRD(0.00)[2001:470:d5e7:1::1:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:470:d5e7:1::1:from]; NEURAL_HAM_SHORT(-0.91)[-0.912]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; FREEMAIL_CC(0.00)[omnilan.de,gmail.com,freebsd.org]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2021 05:47:30 -0000 On Mon, May 17, 2021 at 04:12:20AM +0000, Wei Hu wrote: > Copy Kib who was the author of PTI and other relate patches. > > This looks to be caused by Meltdown and Spectre mitigation patches which were > checked into head couple years ago. See this link for details about these patches: > https://wiki.freebsd.org/SpeculativeExecutionVulnerabilities > > The amd64 patches also caused problems > on Hyper-V (triple fault) when vmbus and other HyperV drivers (netvsc, storvsc etc) > were loaded separately. The workaround for this on amd64 is either build these > Hyper-V modules into the kernel or completely disable mitigation by > setting vm.pmap.pti="0" in /boot/loader.conf in the affect FreeBSD guests. > For amd64 I posted the patch to supposedly fix the issue, you did not responded. > It looks the problem on i386 guests is different. While multiple patches have been > checked in for amd64, there was only one patch r332489 for i386. I am not sure > whether there is similar vm.pmap.pti option to disable the patch for i386 or not. > > I am not familiar to these patches. > > @ Kib, will you be able to comment on this? > > Thanks, > Wei > > > > > -----Original Message----- > > From: Harry Schmalzbauer > > Sent: Sunday, May 16, 2021 2:55 AM > > To: Wei Hu ; Chris Knight ; freebsd- > > virtualization@freebsd.org > > Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root > > > > Am 14.05.2021 um 18:44 schrieb Harry Schmalzbauer: > > > Am 16.10.2020 um 11:35 schrieb Wei Hu via freebsd-virtualization: > > >>> : > > >>> > > >> Did you try 12.0? Maybe something changed in 12.x causes failure. > > >> I386 is not well tested on Hyper-V. I will try to reproduce and take > > >> a look. > > > > > > The regression happened before 12.0. > > > Today I stumbled across this problem too and tried 11.4 (/May 8th, > > > 2020) /sucessfully, while 12.0 (/Oct 19th, 2018)/ is failing with i386 > > > releases. > > > > > > One commit which is in 12 but not in 11.4 is r332489. > > > > > : > > : > > : > > > Unfortunately beyond my skills. > > > > In sys/x86/x86/local_apic.c, I added this debug printf: > >         for (idx = IPI_DYN_FIRST; idx <= IPI_DYN_LAST; idx++) { > > printf("DEBUG: native_lapic_ipi_alloc(), idx = %i\n", idx); > >                 ip = &idt[idx]; > >                 func = (ip->gd_hioffset << 16) | ip->gd_looffset; > >                 if ((!pti && func == (uintptr_t)&IDTVEC(rsvd)) || > >                     (pti && func == (uintptr_t)&IDTVEC(rsvd_pti))) { > >                         vector = idx; > >                         setidt(vector, ipifunc, SDT_APIC, SEL_KPL, GSEL_APIC); > >                         break; > >                 } > >         } > > > > This is my latest attempt to see if at least the origin of my fault tracing matches. > > It does. > > > > From #define IPI_DYN_FIRST   (APIC_IPI_INTS + 9) in sys/x86/include/apicvar.h > > I expected to see 252, 253 and 254, which I sucessfully verified. > > > > Now comes the part I cannot read: (uintptr_t)&IDTVEC(rsvd) > > IDTVEC(rsvd) seems to be a assembler makro from sys/i386/i386/exception.s > > > > But now it doesn't make any sense for me to dig further, I simply do not > > understand it. > > > > Unfortunately having broken i386 support on Hyper-V is not a cosmetic issue. > > Removing "device hyperv" from kernel gives access to the v-hdd via legacy > > IDE/ada, but then I have no NIC. > > Choosing between missing HDD and missing network support is hard > > nowadays ;-) > > > > Any help getting i386 up and running back on Hyper-V highly appreciated!  11.4 > > is the latest working version, all i386-releases starting with 12.0 don't work! > > > > Thanks, > > -harry > From owner-freebsd-virtualization@freebsd.org Mon May 17 06:33:25 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4848F63E2DB for ; Mon, 17 May 2021 06:33:25 +0000 (UTC) (envelope-from weh@microsoft.com) Received: from APC01-PU1-obe.outbound.protection.outlook.com (mail-eopbgr1320121.outbound.protection.outlook.com [40.107.132.121]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fk8Tc0hGSz4kjy for ; Mon, 17 May 2021 06:33:23 +0000 (UTC) (envelope-from weh@microsoft.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oB4ZAwuYmDQbs8lL9uskipaFoGVuLSduKBg6FpaAOuUxq/ypqXDi/Bn0NZNwG63nm4zr1vNoJJvtOnvsJqGS9q83vJkxY6QkTfego+nXaAL1Svctd/p35aAKiz7sQ/eCfPvqKfkV91nw4K2v8puUfCjDPNI3U8RkY6EQaOLW9S0OEcPn1UPzLKgT6OE/tkvCMMizdV0KR9+ImzZpDvcYfHhuiqH3csJ0EAgZQQn/VF+RWWZ0ILUvBc2PUlzXcD83ZtIiucGHhUrrxglohCnHWvTE326xJQCNXz4I2REnbfC7H7R3Wkc5M8Zm4A5aRSzGY2t/m/PXa8wzylPmsfrgGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vo/tTKBpnfSY6Ck5OjaZBJ744gxnzXRUe7+1JBNk1g0=; b=g0SCqfTm8nrEOieXfJf3J7Mxr9qLnk84SRtyMfyMH76fiQI0gxbpjnejfFN1opluq2IJBcyN0EHC1zAzxDZ5BYQy0FnIiKBZliaBasX6LLjDcB/mq182cROPZOj1OfhqSzFZCCUeM0+j+XlN5Ps9hAMq/OcTkYBvab8Qx5st1nqhqTddmTQkyZOnKBoW9QiXpGQu1/3rAHaZs4PL7Ue4TKURXFNQPuKvh2zHxid6Irb740L7CfVxhyABol8R2KS2cRtWnvbOxDp5n1C3Hzpy/wTE5xNaYMTOzjAQUaFfCDQ900d1B87NvgeQTvyRKNl54Pg3mtK3lB9cgNiC0ioAaA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none Received: from PSAP153MB0440.APCP153.PROD.OUTLOOK.COM (2603:1096:301:35::9) by PSBP153MB0357.APCP153.PROD.OUTLOOK.COM (2603:1096:301:1::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4150.7; Mon, 17 May 2021 06:33:19 +0000 Received: from PSAP153MB0440.APCP153.PROD.OUTLOOK.COM ([fe80::a1:5809:9657:cfdf]) by PSAP153MB0440.APCP153.PROD.OUTLOOK.COM ([fe80::a1:5809:9657:cfdf%5]) with mapi id 15.20.4173.009; Mon, 17 May 2021 06:33:19 +0000 From: Wei Hu To: Konstantin Belousov CC: Harry Schmalzbauer , Chris Knight , "freebsd-virtualization@freebsd.org" Subject: RE: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root Thread-Topic: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root Thread-Index: AQHWot5T3VubDGm01EyXk9C1ey9JFqmZ+PXwgUqCJACAAbbYgIACKD5AgAAgPYCAAAv1kA== Date: Mon, 17 May 2021 06:33:19 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=83fd88e0-dacb-4b23-b11a-c7722a6373da; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-05-17T06:29:58Z; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; x-originating-ip: [2404:f801:9000:1a:6fea::e0] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 42a2a9d2-4547-4dc5-e4c3-08d918fda966 x-ms-traffictypediagnostic: PSBP153MB0357: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7219; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: x+tBrkuwMXSUDUru/Q5a5ZVBf5ue85xlISqezCR8ibgqDGHz/Gbza3sQ/Dods03Xalyx+YIXgiGXSWcVCxF6BqyGfGl8dlngnhr10TiA1GkHDaTFYe23noNFCQEGacSSObvTAW5P4cTtxBlYeqwLwF3nlwf/l9ivun38NjVXQ70SQiGTp9mypg4AGqS8DLa5hh0apRFRDbi5bptncEPWbpf8VMyFLOHQSqrzbmIBHPxlgju3qZpnhdJa124D5J5LL/EnuONTxWaeZ/8nlO3WiCYgaI+fLkTSStqCosnDADyJDzsx1lYGrhsTFECW6nJAE22V8Bipo0pwudO19LHTK79EHl4PvVTLJgcaQUXu32AClfGwMhAPZdinW20XUlxXwqKZNfjiN6zO9wloGF68eHQEAFsBslUvFvwUdsAF3f9S8JycCIipyiV9BPcUzunhVh3bilTvf3e+oqKnFAQ5KyS8NgV3ehLkmKdr7FIG4Uy9gKv9C+NBjNwMxLtoBvVxF3uAW9JF+/P6S6T2N/QdyqjMcWi9gy5CvMgnoUTKJRqTljzqkaBDv/r0KfBSuN3Pclzz3NEyb5A2QHWTkUIierL+WsH00KyARxB9+Bobu08Y/k6f6cr8HvCfpzrFeUiR4vhUeOceJ4KFVEEwpXtjw3vcjJaGGaQ937CgbQxFbUezNK42zpCLyieHaGX919U/lwlASIogkbbJATw569smpzdYfX8DKFppIbRAdLtpDWE= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PSAP153MB0440.APCP153.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(4636009)(366004)(54906003)(76116006)(82950400001)(52536014)(82960400001)(83380400001)(10290500003)(122000001)(478600001)(6506007)(53546011)(6916009)(8676002)(66476007)(316002)(64756008)(5660300002)(4326008)(2906002)(66446008)(66556008)(8936002)(66946007)(86362001)(33656002)(9686003)(55016002)(8990500004)(71200400001)(966005)(186003)(7696005)(38100700002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?/l+LQhtPdyjI0m6JY55RTmlXAVobSQhbdcGcFn1aiX2BozwDDw1ynVOt2zP0?= =?us-ascii?Q?GOliA7Ap0oiZbSVh9+MkITvASWXa+R8Q9vakORMPCnpkkbiUyQ34TYYpwNR4?= =?us-ascii?Q?Vjvi6ytlQRmWuUdbpZK9pEH/spd2OVDsfmZ47fMWGfYYqw3BjVZwolm8QHth?= =?us-ascii?Q?eFn+4wOAJx1o1QxLnw+tFKtMC6oNS11cSZVpupiN5rRViFcAhJ1VLCe8JUEi?= =?us-ascii?Q?xw2fAMDjyPTxa4g2HExZw2XZ13EITTA7Kak3F3w6fhgEMN6VYiypU09hIKd1?= =?us-ascii?Q?5hzKe4Lu2wumJb7L/JBnsWWn0aUoSWiQFWWL7Ke5jsnosqKf196VelgupgI7?= =?us-ascii?Q?F1m7iRcc+8zkVOMcjef1EXZqPEYB04hpm4sZ9MGqk1YcVudSY3jX5FDjb08C?= =?us-ascii?Q?w0bLreK9vPob/37PmbGNkBm+gppNrlKBTM6EBnSeT7bmmLrEIy5Ues1BjU1r?= =?us-ascii?Q?/dnFzEz0gEqQPnVplTltZP1AAjrpNXJ5+OLbfYI/RyjXE4mGxxG7XxdUGH9H?= =?us-ascii?Q?IBSiR+Df8vgUUqcBR/objhoFADA3JDrIY0VZZs00fxtknc28WeD6haf8mMoO?= =?us-ascii?Q?D1ZM8fTJT8ApLIwQnU5XchQCtaXKDtF0YB2o+BqOqAEeTpT2kvfbc7xFenHL?= =?us-ascii?Q?Pd9Z+lFkuyLDNhG/yxliv4cHAvncTyJtXHid7ZE4fMzv839Y813yqu1a5JPm?= =?us-ascii?Q?FKBgnFcEP6U4JB0tXPSADUtnHZCXzyX7uUGJUkBW1qFnZJcFCtcVjERKSo5d?= =?us-ascii?Q?0p7W3A1UivYGJj5mCSvWcVi7SwxoyNsEoIsCgYAuC1pmCfqEO5fm7RykCUcY?= =?us-ascii?Q?bwYXMNTY8D+VbwzE1KOZzK2EFvV0fteqm114xKuehyDEbszRCMg2e38SxpYt?= =?us-ascii?Q?xNfCfgd8K5uMgEQuSOITb25erNXDTlYLQTjqNTziwhyFiQlVLcHAUAB5enO9?= =?us-ascii?Q?WS3I+bBq5r5uuYuWS6E2kBVMA8owXTFEhfCfTqUhOEvH0w5ubuTS1PZD5M4W?= =?us-ascii?Q?/w4qBvhoe1FdJ5x5+ViTuiXlnDBtlLt6yOlgiVE000BdAweAnY2TZvxvWwNv?= =?us-ascii?Q?1y9KZdCZ0SkjIUCI3Z0EO1Wr+U3Ik21FKz43ttIhEjSj2wZt69RKep+YPbVT?= =?us-ascii?Q?wqCfduskwiwcX65wxac3Ix4ZU94riPPkSnAzrwWilJxj2XV4NmzclH8+2w5g?= =?us-ascii?Q?4Rp55fYGTbJcO3/SSE2bnRrLw4M6+lD6mrUVAKbAYtfd+a4gysCIc78eYjnw?= =?us-ascii?Q?0DZY8Gglmb7Dpsm+pmu81eJjDGNdBiSO7A24/Ass5r6SuR0L2ggHoHTu0wNK?= =?us-ascii?Q?9XeDlWS4vU8vdDDvpdRA+PFsrlxQS7aoxmyW8ZUTdTJHnA=3D=3D?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PSAP153MB0440.APCP153.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 42a2a9d2-4547-4dc5-e4c3-08d918fda966 X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2021 06:33:19.5330 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: P53NNOIvn7odtkuak+sNEyIJyipm9o9IstzuK7pH6ZYJYFfBW/7NpZS5EF03TxCN2ycVLPtrD5tJLWwDeTfMjA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PSBP153MB0357 X-Rspamd-Queue-Id: 4Fk8Tc0hGSz4kjy X-Spamd-Bar: ---------- X-Spamd-Result: default: False [-10.10 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; HAS_XOIP(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[microsoft.com:+]; DMARC_POLICY_ALLOW(-0.50)[microsoft.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_IN_DNSWL_LOW(-0.10)[40.107.132.121:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[40.107.132.121:from]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[microsoft.com:s=selector2]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; DWL_DNSWL_MED(-2.00)[microsoft.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[40.107.132.121:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.132.121:from]; WHITELIST_SPF_DKIM(-3.00)[microsoft.com:d:+,microsoft.com:s:+]; FREEMAIL_CC(0.00)[omnilan.de,gmail.com,freebsd.org]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2021 06:33:25 -0000 > -----Original Message----- > From: Konstantin Belousov > Sent: Monday, May 17, 2021 1:47 PM > To: Wei Hu > Cc: Harry Schmalzbauer ; Chris Knight > ; freebsd-virtualization@freebsd.org > Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root >=20 > On Mon, May 17, 2021 at 04:12:20AM +0000, Wei Hu wrote: > > Copy Kib who was the author of PTI and other relate patches. > > > > This looks to be caused by Meltdown and Spectre mitigation patches > > which were checked into head couple years ago. See this link for detail= s about > these patches: > > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fwik= i > > .freebsd.org%2FSpeculativeExecutionVulnerabilities&data=3D04%7C01%7= C > > > weh%40microsoft.com%7Cd7b33cd5fc774c88310008d918f744c4%7C72f988bf > 86f14 > > > 1af91ab2d7cd011db47%7C1%7C0%7C637568272592647219%7CUnknown%7 > CTWFpbGZsb > > > 3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% > 3D% > > > 7C1000&sdata=3D5HjRc67IiHlrpR5bgWwJq9opMwBKudlHJpElsDZkS4A%3D& > amp;re > > served=3D0 > > > > The amd64 patches also caused problems on Hyper-V (triple fault) when > > vmbus and other HyperV drivers (netvsc, storvsc etc) were loaded > > separately. The workaround for this on amd64 is either build these > > Hyper-V modules into the kernel or completely disable mitigation by > > setting vm.pmap.pti=3D"0" in /boot/loader.conf in the affect FreeBSD gu= ests. > > > For amd64 I posted the patch to supposedly fix the issue, you did not > responded. >=20 I am sorry I might have missed your review request. Would you please let me= =20 know the link to the patch or the commit if you already have checked in? I = will need to test and inform the parties who got affected.=20 For the issue on i386, any plan to fix it. Thanks, Wei From owner-freebsd-virtualization@freebsd.org Mon May 17 06:47:31 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BE3D563E464 for ; Mon, 17 May 2021 06:47:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fk8nt6lKsz4m1J for ; Mon, 17 May 2021 06:47:30 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 14H6lMFv041780 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 17 May 2021 09:47:25 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 14H6lMFv041780 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 14H6lMGA041779; Mon, 17 May 2021 09:47:22 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 17 May 2021 09:47:22 +0300 From: Konstantin Belousov To: Wei Hu Cc: Harry Schmalzbauer , Chris Knight , "freebsd-virtualization@freebsd.org" Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4Fk8nt6lKsz4m1J X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-1.00 / 15.00]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none]; TO_DN_EQ_ADDR_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; FREEMAIL_FROM(0.00)[gmail.com]; RCVD_TLS_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:470:d5e7:1::1:from]; R_SPF_SOFTFAIL(0.00)[~all:c]; NEURAL_SPAM_SHORT(1.00)[0.998]; SPAMHAUS_ZRD(0.00)[2001:470:d5e7:1::1:from:127.0.2.255]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.996]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; FREEMAIL_CC(0.00)[omnilan.de,gmail.com,freebsd.org]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2021 06:47:31 -0000 On Mon, May 17, 2021 at 06:33:19AM +0000, Wei Hu wrote: > > -----Original Message----- > > From: Konstantin Belousov > > Sent: Monday, May 17, 2021 1:47 PM > > To: Wei Hu > > Cc: Harry Schmalzbauer ; Chris Knight > > ; freebsd-virtualization@freebsd.org > > Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root > > > > On Mon, May 17, 2021 at 04:12:20AM +0000, Wei Hu wrote: > > > Copy Kib who was the author of PTI and other relate patches. > > > > > > This looks to be caused by Meltdown and Spectre mitigation patches > > > which were checked into head couple years ago. See this link for details about > > these patches: > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki > > > .freebsd.org%2FSpeculativeExecutionVulnerabilities&data=04%7C01%7C > > > > > weh%40microsoft.com%7Cd7b33cd5fc774c88310008d918f744c4%7C72f988bf > > 86f14 > > > > > 1af91ab2d7cd011db47%7C1%7C0%7C637568272592647219%7CUnknown%7 > > CTWFpbGZsb > > > > > 3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% > > 3D% > > > > > 7C1000&sdata=5HjRc67IiHlrpR5bgWwJq9opMwBKudlHJpElsDZkS4A%3D& > > amp;re > > > served=0 > > > > > > The amd64 patches also caused problems on Hyper-V (triple fault) when > > > vmbus and other HyperV drivers (netvsc, storvsc etc) were loaded > > > separately. The workaround for this on amd64 is either build these > > > Hyper-V modules into the kernel or completely disable mitigation by > > > setting vm.pmap.pti="0" in /boot/loader.conf in the affect FreeBSD guests. > > > > > For amd64 I posted the patch to supposedly fix the issue, you did not > > responded. > > > I am sorry I might have missed your review request. Would you please let me > know the link to the patch or the commit if you already have checked in? I will > need to test and inform the parties who got affected. It was mailed for you after your query, then I pinged you several times. Anyway, look at and try https://reviews.freebsd.org/D30310 > > For the issue on i386, any plan to fix it. > > Thanks, > Wei From owner-freebsd-virtualization@freebsd.org Mon May 17 09:40:12 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8D472641ACB for ; Mon, 17 May 2021 09:40:12 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "CN", Issuer "CN" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FkDd73yXlz3hJT for ; Mon, 17 May 2021 09:40:10 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [78.138.80.135]) by mx0.gentlemail.de (8.15.2/8.15.2) with ESMTP id 14H9e0mL019074; Mon, 17 May 2021 11:40:01 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 11571F61; Mon, 17 May 2021 11:40:00 +0200 (CEST) Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root To: Konstantin Belousov , Wei Hu Cc: Chris Knight , "freebsd-virtualization@freebsd.org" References: From: Harry Schmalzbauer Organization: OmniLAN Message-ID: <22bbe4cd-82ed-b826-9ac0-05ef8aceba0e@omnilan.de> Date: Mon, 17 May 2021 11:39:59 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Greylist: inspected by milter-greylist-4.6.2 (mx0.gentlemail.de [78.138.80.130]); Mon, 17 May 2021 11:40:03 +0200 (CEST) for IP:'78.138.80.135' DOMAIN:'mh0.gentlemail.de' HELO:'mh0.gentlemail.de' FROM:'freebsd@omnilan.de' RCPT:'' X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx0.gentlemail.de [78.138.80.130]); Mon, 17 May 2021 11:40:03 +0200 (CEST) X-Rspamd-Queue-Id: 4FkDd73yXlz3hJT X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd@omnilan.de designates 2a00:e10:2800::a130 as permitted sender) smtp.mailfrom=freebsd@omnilan.de X-Spamd-Result: default: False [0.18 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_TO(0.00)[gmail.com,microsoft.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:61157, ipnet:2a00:e10:2800::/38, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:e10:2800::a130:from]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_SPAM_SHORT(0.77)[0.769]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[omnilan.de]; NEURAL_SPAM_MEDIUM(0.71)[0.712]; SPAMHAUS_ZRD(0.00)[2a00:e10:2800::a130:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2021 09:40:12 -0000 Am 17.05.2021 um 08:47 schrieb Konstantin Belousov: > On Mon, May 17, 2021 at 06:33:19AM +0000, Wei Hu wrote: >>> -----Original Message----- >>> From: Konstantin Belousov >>> Sent: Monday, May 17, 2021 1:47 PM >>> To: Wei Hu >>> Cc: Harry Schmalzbauer ; Chris Knight >>> ; freebsd-virtualization@freebsd.org >>> Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root >>> >>> On Mon, May 17, 2021 at 04:12:20AM +0000, Wei Hu wrote: : : : >>>> separately. The workaround for this on amd64 is either build these >>>> Hyper-V modules into the kernel or completely disable mitigation by >>>> setting vm.pmap.pti="0" in /boot/loader.conf in the affect FreeBSD guests. >>>> Just to share some hopefully helpful pre-check's I've done so far: Disabling PTI via vm.pmap.pti="0" in /boot/loader.conf doesn't make any difference for the i386 issue (at least x86/x86/local_apic.c always uses non-pti path; tried nontheless, with identical result). As far as my analysis isn't completely wrong, the problem is about IDT pointer. IDTVEC(rsvd) doesn't match in line 2180 of x86/x86/local_apic.c in native_lapic_ipi_alloc(): (ip->gd_hioffset << 16) | ip->gd_looffset; Workign 11.4 shows these values: 11.4-i386 idx 251 (252-1 because no r363527, IPI_SWI) : hi = (49417) 1100 0001 0000 1001 lo = (5472) 0001 0101 0110 0000 func = 1100 0001 0000 1001 0001 0101 0110 0000 IDTVEC(rsvd) = 1100 0001 0000 1001 0001 0101 0110 0000 == rsvd_pti stable/13: i386 idx 252 Nok: hi = (65472) 1111 1111 1100 0000 lo = (12456) 0011 0000 1010 1000 func = 1111 1111 1100 0000 0011 0000 1010 1000 IDTVEC(rsvd) = 0 1110 0100 1011 1011 1110 1000 == rsvd_pti My knowledge about IDT and it's handling by OS/FreeBSD is by far not enough to get a solution/explanation for the above. I couldn't find the relevant paths in our source code (starting with the lack of basic asm skills). For sure I'm missing the important path here, maybe I'm totaly wrong... if not this hopefully saves you some time. I noticed r332489 too as a candidate for the regression beteween 12(.0) and 11(.2+). I prepared a diff which applies to 11.4 (sources from iso) but recompiling 11.4 fails currently, I would need to rework the i386/i386/pmap.c adoption. : : : >> know the link to the patch or the commit if you already have checked in? I will >> need to test and inform the parties who got affected. > > It was mailed for you after your query, then I pinged you several times. > Anyway, look at and try https://reviews.freebsd.org/D30310 I applied it to stable/13 and this doesn't make any difference, still the (ip->gd_hioffset << 16) | ip->gd_looffset; and (uintptr_t)IDTVEC(rsvd) differ with exactly the same values as shown above. Thanks for your attention, best regards, -harry From owner-freebsd-virtualization@freebsd.org Tue May 18 01:56:29 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5DA5763A386 for ; Tue, 18 May 2021 01:56:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4FkfHd21Vjz3k2V for ; Tue, 18 May 2021 01:56:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 4362563A298; Tue, 18 May 2021 01:56:29 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4325863A251 for ; Tue, 18 May 2021 01:56:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FkfHd1DdZz3jsP for ; Tue, 18 May 2021 01:56:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0EB92170A6 for ; Tue, 18 May 2021 01:56:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 14I1uSFG001469 for ; Tue, 18 May 2021 01:56:28 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 14I1uS2S001468 for virtualization@FreeBSD.org; Tue, 18 May 2021 01:56:28 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 246168] Ubuntu 20.04 KVM / QEMU Failure with nested FreeBSD bhyve Date: Tue, 18 May 2021 01:56:29 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: drum@graphica.com.au X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 May 2021 01:56:29 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D246168 --- Comment #13 from John Hartley --- (In reply to Peter Grehan from comment #12) Hi Peter, I have been working with Ubuntu team again on this: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1876678 and based on further testing have raise issue with upstream QEMU: https://gitlab.com/qemu-project/qemu/-/issues/337 Just cross linking so issue can be tracked across various components. Thanks, John Hartley --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue May 18 06:03:29 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 666AA64057C for ; Tue, 18 May 2021 06:03:29 +0000 (UTC) (envelope-from meetshamsher@gmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Fklmd1DtFz3D90 for ; Tue, 18 May 2021 06:03:29 +0000 (UTC) (envelope-from meetshamsher@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 28A8264057B; Tue, 18 May 2021 06:03:29 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 286656404EC for ; Tue, 18 May 2021 06:03:29 +0000 (UTC) (envelope-from meetshamsher@gmail.com) Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fklmc2QS0z3DFL for ; Tue, 18 May 2021 06:03:25 +0000 (UTC) (envelope-from meetshamsher@gmail.com) Received: by mail-pf1-x42e.google.com with SMTP id d78so5759282pfd.10 for ; Mon, 17 May 2021 23:03:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:content-transfer-encoding:mime-version:subject:date:references :to:in-reply-to:message-id; bh=aLMSJ36GPnnkdsUSzkqcVDu7eONT0p0DpVd4FYClhLQ=; b=iSrPxMIBofIUKMUgUfZZlOWw9scKGSCuR3kQZvNb8AKXm9OQS2OiNkjMuvyUcbkLDK Yh9oTh7BChij7d8E7Ac7XUKOcA0MJRyy9a0wYpZPC9GkATg2aK4nRZErhd4yqZZBNjN+ s+jMvxugzmg+lbxFWX6gdbBj4EKATMFZ79esnFw6kpkJygiheqbYIYRCVVTeEYgBUyQk e5maUlq8031MB+pzPy34+XV2LH1Wb89iIzUdPBk1D9GhwO273UtY5qM+OQX0fA96abEL CEAGYcVh/c1WvESjphLhyqJWxhktcSxh8n3XQxsmcqsLheQhbgJPrIpLXDRYTnm3SnTX XTdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:date:references:to:in-reply-to:message-id; bh=aLMSJ36GPnnkdsUSzkqcVDu7eONT0p0DpVd4FYClhLQ=; b=fTfS+1NWaCcCPorDyJwz2rMbkdi8c0x4IOWhl/0FbDIDkXb9/gpFIcUQDboWLj1XMM GNLrGpp8G/ksNRQxnqXf/k0YYjzqJbrj8t/YNFAPMiPwvpxRcQrCpfd4YrVKNliiDese DwZedDHyOGoqKViyJ30UsuZfGx812ij4zEK+bYDVxtTcistbpo2hJVG1LFUS6f/I/Ptv 9VSvAaPxJZAATC4/Wl4fG5UmRPkp49z9qoIZf82RxQy+VAZATV+SK8MapySnLr8046Pz 0yDJAXVPzIH3NFTpDx9I9bxtGsq3VhAUAUNVqRXhHbN67/aJ5iRj11FBogQ6PjcoBw6K 2gFA== X-Gm-Message-State: AOAM530fsCdeMPBhBDWw80pO2srWsF3hVask6WaPWPUKzM7OcTGFnNel jBKvr0CPMwHkxVAmxKPKXt2DQDdahC4= X-Google-Smtp-Source: ABdhPJxEQ3wXCzRxVuCH4rJeDyK8TJjsVy9/j39s0TpM2uTjXKMQDWQD4Mij/t+KcP50pObprTBYlA== X-Received: by 2002:a62:6491:0:b029:28e:8c90:6b16 with SMTP id y139-20020a6264910000b029028e8c906b16mr3530319pfb.24.1621317803565; Mon, 17 May 2021 23:03:23 -0700 (PDT) Received: from [10.65.74.196] ([72.163.220.14]) by smtp.gmail.com with ESMTPSA id i2sm986961pjj.25.2021.05.17.23.03.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 17 May 2021 23:03:23 -0700 (PDT) From: Shamsher singh Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.6\)) Subject: Re: [Bug 255043] ena device is not getting enabled post addition in kernel Date: Tue, 18 May 2021 11:33:20 +0530 References: To: "virtualization@freebsd.org" In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3608.120.23.2.6) X-Rspamd-Queue-Id: 4Fklmc2QS0z3DFL X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=iSrPxMIB; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of meetshamsher@gmail.com designates 2607:f8b0:4864:20::42e as permitted sender) smtp.mailfrom=meetshamsher@gmail.com X-Spamd-Result: default: False [-3.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RECEIVED_SPAMHAUS_PBL(0.00)[72.163.220.14:received]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::42e:from]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[virtualization@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::42e:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::42e:from]; TO_DN_EQ_ADDR_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[virtualization] X-Mailman-Approved-At: Tue, 18 May 2021 09:13:45 +0000 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 May 2021 06:03:29 -0000 Hi, This is regarding C5 and above Nitro Hypervisor based instances. It seems we need to use KVM hypervisor at FreeBSD kernel level along = with ENA module. In FreeBSD code I am able to see ENA but not where either in kernel or = in port seeing KVM hypervisor support for FreeBSD. Can you please guide me how can I get KVM support in FreeBSD? Thanks & regards Shamsher > On 14-May-2021, at 5:39 PM, bugzilla-noreply@freebsd.org wrote: >=20 > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D255043 >=20 > Mark Linimon changed: >=20 > What |Removed |Added > = --------------------------------------------------------------------------= -- > Assignee|bugs@FreeBSD.org = |virtualization@FreeBSD.org >=20 > --=20 > You are receiving this mail because: > You reported the bug. From owner-freebsd-virtualization@freebsd.org Tue May 18 16:16:50 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EA3C9650FD7 for ; Tue, 18 May 2021 16:16:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Fl1NL66MPz3CYt for ; Tue, 18 May 2021 16:16:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id CFFEA651056; Tue, 18 May 2021 16:16:50 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CFC8F65116A for ; Tue, 18 May 2021 16:16:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fl1NL5SCqz3CLS for ; Tue, 18 May 2021 16:16:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A0298229A5 for ; Tue, 18 May 2021 16:16:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 14IGGoNT056957 for ; Tue, 18 May 2021 16:16:50 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 14IGGoZn056956 for virtualization@FreeBSD.org; Tue, 18 May 2021 16:16:50 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 254513] virtio_random: random_harvestq spinning on a CPU with Q35 virtio random device Date: Tue, 18 May 2021 16:16:50 +0000 X-Bugzilla-Reason: AssignedTo CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 13.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: deltatux@kwokinator.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 May 2021 16:16:51 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D254513 deltatux@kwokinator.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |deltatux@kwokinator.com --- Comment #5 from deltatux@kwokinator.com --- Just wanted to chime in that I have been experiencing the same issue as wel= l, it looks like this issue may be more widespread, I found someone writing ab= out this issue while trying to research a workaround for this problem: https://decomo.info/wiki/en/blog/2021/2021-05-02 I tried applying the workaround as suggested in this bug report by blacklis= ting virtio_random in rc.conf and it appears to be working, random_harvestq is no longer eating up 1 of my VM cores. However, I'm not sure what the impact is for blacklisting virtio_random... --=20 You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug.= From owner-freebsd-virtualization@freebsd.org Tue May 18 20:51:40 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8CE776571ED for ; Tue, 18 May 2021 20:51:40 +0000 (UTC) (envelope-from bsdlists@jld3.net) Received: from mail.jld3.net (mail.jld3.net [45.55.236.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fl7TR58Q0z3GQv for ; Tue, 18 May 2021 20:51:39 +0000 (UTC) (envelope-from bsdlists@jld3.net) Received: from localhost (localhost [127.0.0.1]) by mail.jld3.net (Postfix) with ESMTP id 694CE415B5 for ; Tue, 18 May 2021 14:51:33 -0600 (MDT) X-Virus-Scanned: amavisd-new at jld3.net Received: from mail.jld3.net ([127.0.0.1]) by localhost (mail.jld3.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 4Q882j1bsTZQ for ; Tue, 18 May 2021 14:51:32 -0600 (MDT) Received: from [172.21.35.202] (c-24-9-144-115.hsd1.co.comcast.net [24.9.144.115]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: jld) by mail.jld3.net (Postfix) with ESMTPSA id 00C9940573 for ; Tue, 18 May 2021 14:51:31 -0600 (MDT) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.jld3.net 00C9940573 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.jld3.net 00C9940573 From: "John Doherty" To: freebsd-virtualization@freebsd.org Subject: increasing bhyve VM_MAXCPU Date: Tue, 18 May 2021 14:51:22 -0600 X-Mailer: MailMate (1.13.2r5673) Message-ID: <7C6FC30B-BADC-4D32-AA2B-5F00B5E874F6@jld3.net> MIME-Version: 1.0 Content-Type: text/plain; format=flowed X-Rspamd-Queue-Id: 4Fl7TR58Q0z3GQv X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[45.55.236.93:from]; R_DKIM_ALLOW(-0.20)[jld3.net:s=8d052f02dde2]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[45.55.236.93:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[4]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DKIM_TRACE(0.00)[jld3.net:+]; DMARC_POLICY_ALLOW(-0.50)[jld3.net,quarantine]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:14061, ipnet:45.55.192.0/18, country:US]; MID_RHS_MATCH_FROM(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RECEIVED_SPAMHAUS_PBL(0.00)[24.9.144.115:received] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 May 2021 20:51:40 -0000 I'm experimenting with bhyve VMs with more than 16 vCPUs on FreeBSD 12.2-RELEASE. As I understand it, this limit is defined as VM_MAXCPU in /usr/src/sys/amd64/include/vmm.h. I also understand that as of about two years ago, it should be possible to increase it beyond 21[1]. I began working on a machine with only eight CPU cores/threads, so rather than trying to increase VM_MAXCPU, I tried to decrease it, which seemed to work fine. My procedure was: edit /usr/src/sys/amd64/include/vmm.h cd /usr/src/lib/libvmmapi ; make clean ; make ; make install cd /usr/src/usr.sbin/bhyve ; make clean ; make ; make install cd /usr/src/usr.sbin/bhyvectl ; make clean ; make ; make install cd /usr/src/usr.sbin/bhyveload ; make clean ; make ; make install cd /usr/src ; make buildkernel KERNCONF=JLD ; make installkernel KERNCONF=JLD reboot where JLD is a kernel config file that differs from GENERIC only in the "ident" line. I tried several values (1, 2, and 4) for VM_MAXCPU and verified that VMs with a number of vCPUs less than or equal to the max booted fine and that VMs with more didn't. So then I tried increasing VM_MAXCPU on a machine with two CPUs, 14 cores/28 threads each, using the same procedure. I set VM_MAXCPU to 28 and configured a VM with 24 vCPUs but that didn't boot. Also tried setting the max to 20 and booting a VM with 18 vCPUs but that didn't work either. In all cases, VMs with 16 or fewer vCPUs continued to boot fine. All the tests were done with VMs created before doing any of this stuff. I am using vm-bhyve to manage VMs. The VMs in these tests were running one of FreeBSD 12.2-RELEASE, alpine linux 3.13, and debian-testing. So it seems like I'm overlooking something but I don't know what, or maybe this just isn't going to work. Anybody have experience actually doing this? Thanks very much for any advice or further information. [1] From owner-freebsd-virtualization@freebsd.org Tue May 18 20:56:57 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A73A46575C2 for ; Tue, 18 May 2021 20:56:57 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-qv1-xf2a.google.com (mail-qv1-xf2a.google.com [IPv6:2607:f8b0:4864:20::f2a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fl7bX53c9z3HQV for ; Tue, 18 May 2021 20:56:56 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mail-qv1-xf2a.google.com with SMTP id a7so1387149qvf.11 for ; Tue, 18 May 2021 13:56:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=9ruPdqkyXBgHu+l2P9LlTPCw8TsOgn8ONWKvbVdQWcg=; b=LfXwYzy8U5O9Xt5Czl74cQizY8xnQIO0Hm4VCHiRGLeTyMCRvvzW66QKA1g4ic34Xk eivYLJAJ7VndegAeQ2jnKu4tAVQovcZGVpVLpAS9fU3+uO4LK9FfS+mdV8e2U0xUAzx4 CIBivgPWpnq5b8G8no5o1AAD8OuDB21ghAJy2QPmlADTwfegMyHz8AM+LSQxDn1fIkH+ yFQuFC3GgVsAA3tat0v5003wEPFVK5eH01bMQa8FgyzYLSF1rR0kv/y6ikWyWe8+ygOm lxiZY5bts9EUUzlq2baOGXGOtLUI2t8t+ogtaDPT0zhTNFVsfLNZTRk5ssVhcd5aG48W ljWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=9ruPdqkyXBgHu+l2P9LlTPCw8TsOgn8ONWKvbVdQWcg=; b=RIkLdfbe5241sXsQhhA0Sc2vJeVFOnzfEWWgTLv1xawcqfedikLzHtP4JOKLdhgR1i PI552ffCvkIOo/0Cs/+aREuNrZzW7gW2tnOJ2nvsBwYEJdnvLRy8UNCOkcv8GZAmsmp1 Xdp26j8jmrnWGY3jspauR6hbr/OX8DVjWcSjdDcUaoY+QWvwXBHTyF/PJz7CFHfth4Ts HV0cYU6H7RY3OrGkK6/iGfhV5K3W+jhZzsPmLsVFStuKt1DQSeGlwj4cSwbSKQYBePB5 r7B9mRxlIajK0OQE/44Ngfc5wX0B2FAMiExSU2E3dXgflpWAUeI12b0M2u5T2QxrmKLk JeFA== X-Gm-Message-State: AOAM531QFVGeks86QxMFg2rBkKwHAg7Fl9vTXI+6qhWd7jUseaqjZnnC MHCdzl/xf4EqpKJQqeEUEE3rFtPnnjoQOrZo X-Google-Smtp-Source: ABdhPJz54+Zzdf0gaPxCED0qp/g5OPH2zkVwn17BicW367i9c0kFIOgny+uTXPDtYjjchC/m0g5kAQ== X-Received: by 2002:ad4:43c7:: with SMTP id o7mr8359977qvs.42.1621371415745; Tue, 18 May 2021 13:56:55 -0700 (PDT) Received: from mutt-hbsd ([38.140.209.219]) by smtp.gmail.com with ESMTPSA id r10sm1888632qtm.6.2021.05.18.13.56.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 May 2021 13:56:54 -0700 (PDT) Date: Tue, 18 May 2021 16:56:55 -0400 From: Shawn Webb To: John Doherty Cc: freebsd-virtualization@freebsd.org Subject: Re: increasing bhyve VM_MAXCPU Message-ID: <20210518205655.2nnjt4yfdt43atby@mutt-hbsd> X-Operating-System: FreeBSD mutt-hbsd 14.0-CURRENT-HBSD FreeBSD 14.0-CURRENT-HBSD X-PGP-Key: https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/blob/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc References: <7C6FC30B-BADC-4D32-AA2B-5F00B5E874F6@jld3.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="wszted6l6snkae6g" Content-Disposition: inline In-Reply-To: <7C6FC30B-BADC-4D32-AA2B-5F00B5E874F6@jld3.net> X-Rspamd-Queue-Id: 4Fl7bX53c9z3HQV X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hardenedbsd.org header.s=google header.b=LfXwYzy8; dmarc=none; spf=pass (mx1.freebsd.org: domain of shawn.webb@hardenedbsd.org designates 2607:f8b0:4864:20::f2a as permitted sender) smtp.mailfrom=shawn.webb@hardenedbsd.org X-Spamd-Result: default: False [-4.99 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[hardenedbsd.org:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-1.00)[-1.000]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::f2a:from]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.89)[-0.893]; R_DKIM_ALLOW(-0.20)[hardenedbsd.org:s=google]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; DMARC_NA(0.00)[hardenedbsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::f2a:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::f2a:from]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 May 2021 20:56:57 -0000 --wszted6l6snkae6g Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, May 18, 2021 at 02:51:22PM -0600, John Doherty via freebsd-virtuali= zation wrote: > I'm experimenting with bhyve VMs with more than 16 vCPUs on FreeBSD > 12.2-RELEASE. >=20 > As I understand it, this limit is defined as VM_MAXCPU in > /usr/src/sys/amd64/include/vmm.h. I also understand that as of about two > years ago, it should be possible to increase it beyond 21[1]. >=20 > I began working on a machine with only eight CPU cores/threads, so rather > than trying to increase VM_MAXCPU, I tried to decrease it, which seemed to > work fine. My procedure was: >=20 > edit /usr/src/sys/amd64/include/vmm.h > cd /usr/src/lib/libvmmapi ; make clean ; make ; make install > cd /usr/src/usr.sbin/bhyve ; make clean ; make ; make install > cd /usr/src/usr.sbin/bhyvectl ; make clean ; make ; make install > cd /usr/src/usr.sbin/bhyveload ; make clean ; make ; make install > cd /usr/src ; make buildkernel KERNCONF=3DJLD ; make installkernel > KERNCONF=3DJLD > reboot >=20 > where JLD is a kernel config file that differs from GENERIC only in the > "ident" line. >=20 > I tried several values (1, 2, and 4) for VM_MAXCPU and verified that VMs > with a number of vCPUs less than or equal to the max booted fine and that > VMs with more didn't. >=20 > So then I tried increasing VM_MAXCPU on a machine with two CPUs, 14 cores= /28 > threads each, using the same procedure. I set VM_MAXCPU to 28 and configu= red > a VM with 24 vCPUs but that didn't boot. Also tried setting the max to 20 > and booting a VM with 18 vCPUs but that didn't work either. In all cases, > VMs with 16 or fewer vCPUs continued to boot fine. >=20 > All the tests were done with VMs created before doing any of this stuff. I > am using vm-bhyve to manage VMs. The VMs in these tests were running one = of > FreeBSD 12.2-RELEASE, alpine linux 3.13, and debian-testing. >=20 > So it seems like I'm overlooking something but I don't know what, or maybe > this just isn't going to work. Anybody have experience actually doing thi= s? >=20 > Thanks very much for any advice or further information. HardenedBSD bumped max vCPUs up to 48 in a recent commit[0]. Our 14-CURRENT/amd64 package building VM is set up with 30 (out of 36 available) vCPUs (`vmrun.sh -c 30`). We've completed one full package build (so, 31,000+ ports) without issue. The one package building VM we have on the system (so no other VMs) is running HardenedBSD 14-CURRENT. Meaning, we don't run anything other than HardenedBSD VMs. [0]: https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/issues/30 Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --wszted6l6snkae6g Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmCkKhQACgkQ/y5nonf4 4frv4g//WYnnfpb3LRxuTxRgwqKWZ9EGAH8TdKXbCKlQat4kcww9P78dzWNiR4/T EGm2r2IXMzY+f+tqECr1wu8Wuy+mJUTDotTm7ZP1+TdKJQsulBEDIZU5Quf+E/0e UfGegvaDySDD1cJ+dGV7H5dF8RRejZEwlZfyDXVpilL2jMaF6sjFI9H++9GHwOUF tj8ppTVxw65dviAWiu5Vw1/VgWDBfgif2cMf2iS+P/rna5Pcjt6KQbKgJBfm6zBy TgLtlf3YzmhJ94OveBoGmtoDUs5SP6wATni0JK5OsJgSRiGMGGpnBUCTNe7SEOJy ipbltjLtDojsmUwCeaMKQ9C2hR6moyCS2D5LNlSqE/hfG6FoNmKEAzgfmwaffxgb F7LqPbktzdj6RHdcLSI2bniLIQHat4EZTZ+SQU6HIqunKT1A3b9lfXJ0du9+Qk3Z HqUahFc0PNV+P2RD+rdhFHS21qR4y92kuWTz7buH6qjx5KV46TqDry+MooBGVRRg yD5IZA4EjeP84jsdEZEETctTX/D+4f6Axve2CjFYusafwSGROoHmJElEKhnReWBk 0CbskProCspXDa6yi9G3nIV35HBXWXR2xFuepCmCKO/qKtdCFPy3aS3vAP4dFoSq RtJJs/ATWdHD1uE3bTvOnCVTKjs9Nq2wtiKrdryNP7ZMmzX4iVE= =U1bc -----END PGP SIGNATURE----- --wszted6l6snkae6g-- From owner-freebsd-virtualization@freebsd.org Wed May 19 01:21:05 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D6F30634308 for ; Wed, 19 May 2021 01:21:05 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FlFSJ6qwbz3MF9 for ; Wed, 19 May 2021 01:21:04 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id 14J1L3Vv001127; Tue, 18 May 2021 18:21:03 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id 14J1L3BV001126; Tue, 18 May 2021 18:21:03 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <202105190121.14J1L3BV001126@gndrsh.dnsmgr.net> Subject: Re: increasing bhyve VM_MAXCPU In-Reply-To: <7C6FC30B-BADC-4D32-AA2B-5F00B5E874F6@jld3.net> To: John Doherty Date: Tue, 18 May 2021 18:21:03 -0700 (PDT) CC: freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 4FlFSJ6qwbz3MF9 X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd-rwg@gndrsh.dnsmgr.net has no SPF policy when checking 69.59.192.140) smtp.mailfrom=freebsd-rwg@gndrsh.dnsmgr.net X-Spamd-Result: default: False [1.64 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.89)[0.892]; NEURAL_HAM_LONG(-0.99)[-0.990]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[dnsmgr.net]; RBL_DBL_DONT_QUERY_IPS(0.00)[69.59.192.140:from]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.84)[0.842]; SPAMHAUS_ZRD(0.00)[69.59.192.140:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MIME_TRACE(0.00)[0:+]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2021 01:21:05 -0000 > I'm experimenting with bhyve VMs with more than 16 vCPUs on FreeBSD > 12.2-RELEASE. > > As I understand it, this limit is defined as VM_MAXCPU in > /usr/src/sys/amd64/include/vmm.h. I also understand that as of about two > years ago, it should be possible to increase it beyond 21[1]. > > I began working on a machine with only eight CPU cores/threads, so > rather than trying to increase VM_MAXCPU, I tried to decrease it, which > seemed to work fine. My procedure was: > > edit /usr/src/sys/amd64/include/vmm.h cp /usr/src/sys/amd64/include/vmm.h /usr/include/machine/vmm.h > cd /usr/src/lib/libvmmapi ; make clean ; make ; make install > cd /usr/src/usr.sbin/bhyve ; make clean ; make ; make install > cd /usr/src/usr.sbin/bhyvectl ; make clean ; make ; make install > cd /usr/src/usr.sbin/bhyveload ; make clean ; make ; make install All of this code you built used the old value as you did not propogate your change to vmm.h to the installed system, since your running individual make commands the includes in /usr/include are used. > cd /usr/src ; make buildkernel KERNCONF=JLD ; make installkernel > KERNCONF=JLD > reboot Your kernel has the new value you put in vmm.h, so now you have created a userland/kernel missmatch on some data structures. > > where JLD is a kernel config file that differs from GENERIC only in the > "ident" line. > > I tried several values (1, 2, and 4) for VM_MAXCPU and verified that VMs > with a number of vCPUs less than or equal to the max booted fine and > that VMs with more didn't. > > So then I tried increasing VM_MAXCPU on a machine with two CPUs, 14 > cores/28 threads each, using the same procedure. I set VM_MAXCPU to 28 > and configured a VM with 24 vCPUs but that didn't boot. Also tried > setting the max to 20 and booting a VM with 18 vCPUs but that didn't > work either. In all cases, VMs with 16 or fewer vCPUs continued to boot > fine. > > All the tests were done with VMs created before doing any of this stuff. > I am using vm-bhyve to manage VMs. The VMs in these tests were running > one of FreeBSD 12.2-RELEASE, alpine linux 3.13, and debian-testing. > > So it seems like I'm overlooking something but I don't know what, or > maybe this just isn't going to work. Anybody have experience actually > doing this? > > Thanks very much for any advice or further information. > > [1] > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Wed May 19 01:23:00 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A16D5634331 for ; Wed, 19 May 2021 01:23:00 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FlFVX00xDz3MJG for ; Wed, 19 May 2021 01:22:59 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id 14J1MwB7001153; Tue, 18 May 2021 18:22:58 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id 14J1Mwfa001152; Tue, 18 May 2021 18:22:58 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <202105190122.14J1Mwfa001152@gndrsh.dnsmgr.net> Subject: Re: increasing bhyve VM_MAXCPU In-Reply-To: <20210518205655.2nnjt4yfdt43atby@mutt-hbsd> To: Shawn Webb Date: Tue, 18 May 2021 18:22:58 -0700 (PDT) CC: John Doherty , freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 4FlFVX00xDz3MJG X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd-rwg@gndrsh.dnsmgr.net has no SPF policy when checking 69.59.192.140) smtp.mailfrom=freebsd-rwg@gndrsh.dnsmgr.net X-Spamd-Result: default: False [1.80 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-0.97)[-0.973]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[dnsmgr.net]; RBL_DBL_DONT_QUERY_IPS(0.00)[69.59.192.140:from]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.98)[0.976]; SPAMHAUS_ZRD(0.00)[69.59.192.140:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.90)[0.902]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MIME_TRACE(0.00)[0:+]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2021 01:23:00 -0000 -- Start of PGP signed section. > On Tue, May 18, 2021 at 02:51:22PM -0600, John Doherty via freebsd-virtualization wrote: > > I'm experimenting with bhyve VMs with more than 16 vCPUs on FreeBSD > > 12.2-RELEASE. > > > > As I understand it, this limit is defined as VM_MAXCPU in > > /usr/src/sys/amd64/include/vmm.h. I also understand that as of about two > > years ago, it should be possible to increase it beyond 21[1]. > > > > I began working on a machine with only eight CPU cores/threads, so rather > > than trying to increase VM_MAXCPU, I tried to decrease it, which seemed to > > work fine. My procedure was: > > > > edit /usr/src/sys/amd64/include/vmm.h > > cd /usr/src/lib/libvmmapi ; make clean ; make ; make install > > cd /usr/src/usr.sbin/bhyve ; make clean ; make ; make install > > cd /usr/src/usr.sbin/bhyvectl ; make clean ; make ; make install > > cd /usr/src/usr.sbin/bhyveload ; make clean ; make ; make install > > cd /usr/src ; make buildkernel KERNCONF=JLD ; make installkernel > > KERNCONF=JLD > > reboot > > > > where JLD is a kernel config file that differs from GENERIC only in the > > "ident" line. > > > > I tried several values (1, 2, and 4) for VM_MAXCPU and verified that VMs > > with a number of vCPUs less than or equal to the max booted fine and that > > VMs with more didn't. > > > > So then I tried increasing VM_MAXCPU on a machine with two CPUs, 14 cores/28 > > threads each, using the same procedure. I set VM_MAXCPU to 28 and configured > > a VM with 24 vCPUs but that didn't boot. Also tried setting the max to 20 > > and booting a VM with 18 vCPUs but that didn't work either. In all cases, > > VMs with 16 or fewer vCPUs continued to boot fine. > > > > All the tests were done with VMs created before doing any of this stuff. I > > am using vm-bhyve to manage VMs. The VMs in these tests were running one of > > FreeBSD 12.2-RELEASE, alpine linux 3.13, and debian-testing. > > > > So it seems like I'm overlooking something but I don't know what, or maybe > > this just isn't going to work. Anybody have experience actually doing this? > > > > Thanks very much for any advice or further information. > > HardenedBSD bumped max vCPUs up to 48 in a recent commit[0]. Our > 14-CURRENT/amd64 package building VM is set up with 30 (out of 36 > available) vCPUs (`vmrun.sh -c 30`). We've completed one full package > build (so, 31,000+ ports) without issue. The one package building VM > we have on the system (so no other VMs) is running HardenedBSD > 14-CURRENT. Meaning, we don't run anything other than HardenedBSD VMs. > > [0]: https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/issues/30 When I made the changes to fix the brokeness at 21 vCPU we (Michael Dexter and myself) did testing into the 200's of vcpu's. Note that many guests have problems at such large core counts, but bhyve itself behaved ok, but some what sluggish as iirc the testing was done on a 32 core system. > Thanks, > > -- > Shawn Webb > Cofounder / Security Engineer > HardenedBSD > > https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc -- End of PGP section, PGP failed! -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Wed May 19 02:45:52 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E2F0D635837 for ; Wed, 19 May 2021 02:45:52 +0000 (UTC) (envelope-from bsdlists@jld3.net) Received: from mail.jld3.net (mail.jld3.net [45.55.236.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FlHL74rbbz4W8R for ; Wed, 19 May 2021 02:45:51 +0000 (UTC) (envelope-from bsdlists@jld3.net) Received: from localhost (localhost [127.0.0.1]) by mail.jld3.net (Postfix) with ESMTP id 9C382415A2; Tue, 18 May 2021 20:45:50 -0600 (MDT) X-Virus-Scanned: amavisd-new at jld3.net Received: from mail.jld3.net ([127.0.0.1]) by localhost (mail.jld3.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id dZ-LfmkUYFQG; Tue, 18 May 2021 20:45:50 -0600 (MDT) Received: from [172.21.35.202] (c-24-9-144-115.hsd1.co.comcast.net [24.9.144.115]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: jld) by mail.jld3.net (Postfix) with ESMTPSA id E327C40573; Tue, 18 May 2021 20:45:48 -0600 (MDT) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.jld3.net E327C40573 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.jld3.net E327C40573 From: "John Doherty" To: "Rodney W. Grimes" Cc: freebsd-virtualization@freebsd.org Subject: Re: increasing bhyve VM_MAXCPU Date: Tue, 18 May 2021 20:45:42 -0600 X-Mailer: MailMate (1.13.2r5673) Message-ID: <13804A0B-E22F-4C02-BFFF-2C1EFF214EC3@jld3.net> In-Reply-To: <202105190121.14J1L3BV001126@gndrsh.dnsmgr.net> References: <202105190121.14J1L3BV001126@gndrsh.dnsmgr.net> MIME-Version: 1.0 Content-Type: text/plain X-Rspamd-Queue-Id: 4FlHL74rbbz4W8R X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.96 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[45.55.236.93:from]; R_DKIM_ALLOW(-0.20)[jld3.net:s=8d052f02dde2]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_MEDIUM(-0.96)[-0.961]; SPAMHAUS_ZRD(0.00)[45.55.236.93:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[jld3.net:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[jld3.net,quarantine]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:14061, ipnet:45.55.192.0/18, country:US]; MID_RHS_MATCH_FROM(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RECEIVED_SPAMHAUS_PBL(0.00)[24.9.144.115:received] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2021 02:45:52 -0000 On 18 May 2021, at 19:21, Rodney W. Grimes wrote: >> I'm experimenting with bhyve VMs with more than 16 vCPUs on FreeBSD >> I began working on a machine with only eight CPU cores/threads, so >> rather than trying to increase VM_MAXCPU, I tried to decrease it, which >> seemed to work fine. My procedure was: >> >> edit /usr/src/sys/amd64/include/vmm.h > > cp /usr/src/sys/amd64/include/vmm.h /usr/include/machine/vmm.h Thanks so much for this. Things are now working fine, e.g.: [root@fbsd1] # grep 'Multiprocessor System Detected' /var/run/dmesg.boot FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs From owner-freebsd-virtualization@freebsd.org Wed May 19 06:53:24 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A89916397CD for ; Wed, 19 May 2021 06:53:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4FlNqm46hkz4q4m for ; Wed, 19 May 2021 06:53:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 8B9106397CC; Wed, 19 May 2021 06:53:24 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8B581639A20 for ; Wed, 19 May 2021 06:53:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FlNqm3T20z4q7F for ; Wed, 19 May 2021 06:53:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 603966781 for ; Wed, 19 May 2021 06:53:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 14J6rOn3019434 for ; Wed, 19 May 2021 06:53:24 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 14J6rOnT019433 for virtualization@FreeBSD.org; Wed, 19 May 2021 06:53:24 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 221074] Hyper V Gen 2 install has no mouse Date: Wed, 19 May 2021 06:53:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: michael.adm@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2021 06:53:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221074 --- Comment #23 from Michael --- (In reply to Wei Hu from comment #21) I noticed an interesting point. We have only two Gen2 VMs on the hypervisor. We turn on the first VM. The keyboard works for a day or two normally, it d= oes not disappear. We turn on the second VM and run buildworld in it, for example. In the first VM, the keyboard disappears in just a minute. Maybe this will help to identify this nasty bug. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Wed May 19 11:26:22 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EACDC63EEBB for ; Wed, 19 May 2021 11:26:22 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "CN", Issuer "CN" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FlVtj5zwmz4kBh for ; Wed, 19 May 2021 11:26:21 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [IPv6:2a00:e10:2800:0:0:0:0:a135]) by mx0.gentlemail.de (8.15.2/8.15.2) with ESMTP id 14JBQAYi036042; Wed, 19 May 2021 13:26:11 +0200 (CEST) (envelope-from freebsd@omnilan.de) X-Authentication-Warning: mx0.gentlemail.de: Host ezra.dcm1.omnilan.net [IPv6:2a00:e10:2800:0:0:0:0:a135] claimed to be mh0.gentlemail.de Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 4CBE249B; Wed, 19 May 2021 13:26:10 +0200 (CEST) Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root From: Harry Schmalzbauer To: Konstantin Belousov , Wei Hu Cc: "freebsd-virtualization@freebsd.org" References: <22bbe4cd-82ed-b826-9ac0-05ef8aceba0e@omnilan.de> Organization: OmniLAN Message-ID: <7986b561-dd26-a134-e76d-530900363deb@omnilan.de> Date: Wed, 19 May 2021 13:26:09 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <22bbe4cd-82ed-b826-9ac0-05ef8aceba0e@omnilan.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Greylist: inspected by milter-greylist-4.6.2 (mx0.gentlemail.de [IPv6:2a00:e10:2800:0:0:0:0:a130]); Wed, 19 May 2021 13:26:13 +0200 (CEST) for IP:'2a00:e10:2800::a135' DOMAIN:'ezra.dcm1.omnilan.net' HELO:'mh0.gentlemail.de' FROM:'freebsd@omnilan.de' RCPT:'' X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx0.gentlemail.de [IPv6:2a00:e10:2800:0:0:0:0:a130]); Wed, 19 May 2021 13:26:13 +0200 (CEST) X-Rspamd-Queue-Id: 4FlVtj5zwmz4kBh X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd@omnilan.de designates 2a00:e10:2800::a130 as permitted sender) smtp.mailfrom=freebsd@omnilan.de X-Spamd-Result: default: False [-3.30 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; HAS_XAW(0.00)[]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com,microsoft.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:61157, ipnet:2a00:e10:2800::/38, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:e10:2800::a130:from]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-0.996]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[omnilan.de]; SPAMHAUS_ZRD(0.00)[2a00:e10:2800::a130:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 May 2021 11:26:23 -0000 Am 17.05.2021 um 11:39 schrieb Harry Schmalzbauer: > Am 17.05.2021 um 08:47 schrieb Konstantin Belousov: >> On Mon, May 17, 2021 at 06:33:19AM +0000, Wei Hu wrote: >>>> -----Original Message----- >>>> From: Konstantin Belousov >>>> Sent: Monday, May 17, 2021 1:47 PM >>>> To: Wei Hu >>>> Cc: Harry Schmalzbauer ; Chris Knight >>>> ; freebsd-virtualization@freebsd.org >>>> Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root >>>> >>>> On Mon, May 17, 2021 at 04:12:20AM +0000, Wei Hu wrote: > : > : > : >>>>> separately. The workaround for this on amd64 is either build these >>>>> Hyper-V modules into the kernel or completely disable mitigation by >>>>> setting vm.pmap.pti="0" in /boot/loader.conf in the affect FreeBSD >>>>> guests. >>>>> > > Just to share some hopefully helpful pre-check's I've done so far: > > Disabling PTI via vm.pmap.pti="0" in /boot/loader.conf doesn't make > any difference for the i386 issue (at least x86/x86/local_apic.c > always uses non-pti path; tried nontheless, with identical result). > > As far as my analysis isn't completely wrong, the problem is about IDT > pointer. > IDTVEC(rsvd) doesn't match in line 2180 of x86/x86/local_apic.c in > native_lapic_ipi_alloc(): > (ip->gd_hioffset << 16) | ip->gd_looffset; > I managed to setup an i386 vm with -current from 3 years ago (not the most straight forward task due to compiler infrastructure changes back then). Up to r332488 vmbus(4) resp. hyperv(4) is working fine. Updating r332488 to r332489 breaks like already described. Since r332489 is a huge patch with lots of x86 art, the only chance I see, how this can be resolved, is with Konstantin's help. I can just assure that I tracked it down to the single r332489 committ which makes vmbus(4) failing on i386. Verification done: - built/installed world&kernel on i386 HyV guest with r332488   reboot fine. - built/installed kernel on the same vm after svnlite up -r 332489   reboot fails with "vmbus0: cannot find free IDT vector" - loaded kernel.old/kernel, did revert 332489, by 'svnlite up -r 332488' and compiled/reinstalled kernel (again r332488)   reboot fine. I tried to revert r332489 on stable/13 but failed due to lots of conflicting changes (including CVE driven) affecting the same area. To summarize: r332489 only breaks i386 native_lapic_ipi_alloc(), amd64 is not affected (but haven't checked if PTI has any influence on amd64). The effect is 100% reproducable. Happy to test any suggestion or debug extension! Thanks, -harry From owner-freebsd-virtualization@freebsd.org Thu May 20 03:29:41 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E6AE4633086 for ; Thu, 20 May 2021 03:29:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4FlwGF5ZCDz4Sg4 for ; Thu, 20 May 2021 03:29:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id BA99A632ABC; Thu, 20 May 2021 03:29:41 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B99FD63302A for ; Thu, 20 May 2021 03:29:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FlwGF4P51z4SXY for ; Thu, 20 May 2021 03:29:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7F35E1EB7F for ; Thu, 20 May 2021 03:29:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 14K3TfRm069720 for ; Thu, 20 May 2021 03:29:41 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 14K3TfPV069719 for virtualization@FreeBSD.org; Thu, 20 May 2021 03:29:41 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 221074] Hyper V Gen 2 install has no mouse Date: Thu, 20 May 2021 03:29:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: weh@microsoft.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 May 2021 03:29:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221074 --- Comment #24 from Wei Hu --- (In reply to Michael from comment #23) I also have a Gen2 VM running for a while. I used it to developed a FreeBSD kernel module last year, and I left it running since then. I admit I don't directly use console often. But every time I open the console, the keyboard always works. I will leave the console open for a few days to see if it hit= the same problem you are seeing. The VM is at r366933 with some additional chan= ges I made irrelevant to keyboard. So it should has the same problem as you hav= e. In the meantime, if you can problem more information about how to reproduce= it, such as the kind of operation you are using on the consoles, and Hyper-V version, it may help. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu May 20 09:54:32 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 05D2A63A569 for ; Thu, 20 May 2021 09:54:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Fm4pH6Qtrz3Njg for ; Thu, 20 May 2021 09:54:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id DA0ED63A568; Thu, 20 May 2021 09:54:31 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D9D2063A683 for ; Thu, 20 May 2021 09:54:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fm4pH5l0Qz3Njf for ; Thu, 20 May 2021 09:54:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AE22423B79 for ; Thu, 20 May 2021 09:54:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 14K9sVjJ061311 for ; Thu, 20 May 2021 09:54:31 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 14K9sVDR061310 for virtualization@FreeBSD.org; Thu, 20 May 2021 09:54:31 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 221074] Hyper V Gen 2 install has no mouse Date: Thu, 20 May 2021 09:54:31 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: michael.adm@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 May 2021 09:54:32 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221074 --- Comment #25 from Michael --- (In reply to Wei Hu from comment #24) Windows Hyper-V 2019 Server 10.0.17763 (KB4586875, KB4494174, KB4523204, KB4539571, KB4549947, KB4558997, KB456160= 0, KB4562562, KB4580325, KB4587735, KB4592440) All VMs of the Gen2 version (about 10 VMs, the load on the hypervisor is ab= out 50%, 2x Intel Xeon E5-2623 v3). Most FreeBSD 13.0-CURRENT (GENERIC) VMs and two FreeBSD 14.0-CURRENT (GENER= IC) VMs have their kernel updated yesterday. All FreeBSD VMs in the Hyper-V con= sole lose their keyboard (there are Linux and Windows VMs - they have everything= in order with the keyboard in the Hyper-V console). Another point, I have seve= ral hypervisors with approximately the same VM composition and server hardware.= And this bug occurs on everyone. Sometimes the FreeBSD VM keyboard disappears even when not using it. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu May 20 10:17:40 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E515163AA56 for ; Thu, 20 May 2021 10:17:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4Fm5K05ylkz3pFR for ; Thu, 20 May 2021 10:17:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id CAD5F63ACA0; Thu, 20 May 2021 10:17:40 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CA97F63ADB8 for ; Thu, 20 May 2021 10:17:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fm5K059hRz3pVm for ; Thu, 20 May 2021 10:17:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9AF6E24034 for ; Thu, 20 May 2021 10:17:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 14KAHefR075685 for ; Thu, 20 May 2021 10:17:40 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 14KAHeNw075684 for virtualization@FreeBSD.org; Thu, 20 May 2021 10:17:40 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 221074] Hyper V Gen 2 install has no mouse Date: Thu, 20 May 2021 10:17:40 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: michael.adm@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 May 2021 10:17:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221074 --- Comment #26 from Michael --- (In reply to Michael from comment #25) I forgot to say that all VMs have Mellanox ConnectX-3 and Mellanox ConnectX= -4 network cards with SR-IOV enabled (all VMs have network cards with active SR-IOVs). --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu May 20 16:07:27 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2A4D06428F1 for ; Thu, 20 May 2021 16:07:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4FmF4Z5sRcz3Flm for ; Thu, 20 May 2021 16:07:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id BBB76642A40; Thu, 20 May 2021 16:07:26 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BB6FA6428EF for ; Thu, 20 May 2021 16:07:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FmF4Z1V6Bz3FpD for ; Thu, 20 May 2021 16:07:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1BDF12918B for ; Thu, 20 May 2021 16:07:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 14KG7QTF051581 for ; Thu, 20 May 2021 16:07:26 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 14KG7QrW051580 for virtualization@FreeBSD.org; Thu, 20 May 2021 16:07:26 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 221074] Hyper V Gen 2 install has no mouse Date: Thu, 20 May 2021 16:07:26 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: bugzilla.freebsd@omnilan.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 May 2021 16:07:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221074 Harald Schmalzbauer changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bugzilla.freebsd@omnilan.de --- Comment #27 from Harald Schmalzbauer --- (In reply to Michael from comment #25) Only asking about "keyboard disappears": Do you get detach kernel messages, or is it just unresponsive? The latter may be due to scrollLock state-sync by the HyV-Console-App. If I have no responsive keyboard, toggling ScrolLock fixes it. Also I regularly get CapsLock mismatch between guest and host console - meaning the tty inside the HyperV console has CapsLock active, while notepad on the des= k, which runs HyV Console, doesn't behave like CapsLock is active. If I toggle CapLock from inside tty, guest console is totally fine but everything else behaves like CapsLock is active. Might be special side effect caused by RDP (only use RDP to win hosts), nev= er investigated further because it's just a minor issue and isolated to HyV-Console-App. -harry --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Fri May 21 04:54:23 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A2086650825 for ; Fri, 21 May 2021 04:54:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4FmZ5W2Hnlz3NBr for ; Fri, 21 May 2021 04:54:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 4CA47650955; Fri, 21 May 2021 04:54:23 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4B1DA650954 for ; Fri, 21 May 2021 04:54:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FmZ5W1N7Zz3NPg for ; Fri, 21 May 2021 04:54:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1911C343A for ; Fri, 21 May 2021 04:54:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 14L4sN1D049006 for ; Fri, 21 May 2021 04:54:23 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 14L4sNk4049005 for virtualization@FreeBSD.org; Fri, 21 May 2021 04:54:23 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 221074] Hyper V Gen 2 install has no mouse Date: Fri, 21 May 2021 04:54:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: michael.adm@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 May 2021 04:54:23 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221074 --- Comment #28 from Michael --- (In reply to Harald Schmalzbauer from comment #27) > Do you get detach kernel messages, or is it just unresponsive? No messages. The keyboard just becomes unresponsive. > The latter may be due to scrollLock state-sync by the HyV-Console-App. > If I have no responsive keyboard, toggling ScrolLock fixes it. Not. Combinations and special keys do not react (you can dance on the keybo= ard - reactions are zero). In version 13.0, when the keyboard disappears, messages are displayed on the console that occur later, but in 14.0, messages freeze in the place where t= he keyboard disappeared. At the same time, everything works perfectly over SSH, like the system itself. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Fri May 21 12:50:35 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 98B576318AB for ; Fri, 21 May 2021 12:50:35 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "CN", Issuer "CN" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fmmfy4qT8z4TbT for ; Fri, 21 May 2021 12:50:34 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [IPv6:2a00:e10:2800:0:0:0:0:a135]) by mx0.gentlemail.de (8.15.2/8.15.2) with ESMTP id 14LCoMxE053098; Fri, 21 May 2021 14:50:23 +0200 (CEST) (envelope-from freebsd@omnilan.de) X-Authentication-Warning: mx0.gentlemail.de: Host ezra.dcm1.omnilan.net [IPv6:2a00:e10:2800:0:0:0:0:a135] claimed to be mh0.gentlemail.de Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 661348D6; Fri, 21 May 2021 14:50:22 +0200 (CEST) Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root From: Harry Schmalzbauer To: Konstantin Belousov , Wei Hu Cc: "freebsd-virtualization@freebsd.org" References: <22bbe4cd-82ed-b826-9ac0-05ef8aceba0e@omnilan.de> <7986b561-dd26-a134-e76d-530900363deb@omnilan.de> Organization: OmniLAN Message-ID: Date: Fri, 21 May 2021 14:50:21 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <7986b561-dd26-a134-e76d-530900363deb@omnilan.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Greylist: inspected by milter-greylist-4.6.2 (mx0.gentlemail.de [IPv6:2a00:e10:2800:0:0:0:0:a130]); Fri, 21 May 2021 14:50:25 +0200 (CEST) for IP:'2a00:e10:2800::a135' DOMAIN:'ezra.dcm1.omnilan.net' HELO:'mh0.gentlemail.de' FROM:'freebsd@omnilan.de' RCPT:'' X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx0.gentlemail.de [IPv6:2a00:e10:2800:0:0:0:0:a130]); Fri, 21 May 2021 14:50:25 +0200 (CEST) X-Rspamd-Queue-Id: 4Fmmfy4qT8z4TbT X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd@omnilan.de designates 2a00:e10:2800::a130 as permitted sender) smtp.mailfrom=freebsd@omnilan.de X-Spamd-Result: default: False [-2.38 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; HAS_XAW(0.00)[]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com,microsoft.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:61157, ipnet:2a00:e10:2800::/38, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:e10:2800::a130:from]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.09)[-0.087]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-0.99)[-0.993]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[omnilan.de]; SPAMHAUS_ZRD(0.00)[2a00:e10:2800::a130:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 May 2021 12:50:35 -0000 Am 19.05.2021 um 13:26 schrieb Harry Schmalzbauer: ... > I can just assure that I tracked it down to the single r332489 committ > which makes vmbus(4) failing on i386. > Verification done: > - built/installed world&kernel on i386 HyV guest with r332488 >   reboot fine. > - built/installed kernel on the same vm after svnlite up -r 332489 >   reboot fails with "vmbus0: cannot find free IDT vector" > - loaded kernel.old/kernel, did revert 332489, by 'svnlite up -r > 332488' and compiled/reinstalled kernel (again r332488) >   reboot fine. While having hard time catching up with x86 basics, I found another oddity: On Hyper-V gen1, the maximum ammount of RAM I can assign a FreeBSD guest beeing able to boot i386 is 3914MB, startring with r332489. As soon as I configure the guest to have 3916MB or more, FreeBSD-i386 r332489 booting leads to guest reset.  Loader starts kernel, machine reboots. Either reverting to r332488 or reducing guest-RAM solves this. The most interesting fact here is that I couldn't reproduce it on ESXi. I ducplicated the Hyper-V HDD and started the exactly same code on ESXi(v6) guest, where it doesn't crash!  I can boot FreeBSD-i386 r332489 with any ammount of guest-RAM on ESXi. (speaking for GENERIC, non-PAE) Host CPU differs significantly though and I have no possibility to test this influence.  Ther HyperV is Dual-Socket E5-2630 while the ESXi host is a single socket X3470 Xeon (pre SandyBridge). Maybe someone has an idea how this affects these symptoms. Thanks, -harry From owner-freebsd-virtualization@freebsd.org Fri May 21 14:09:17 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 13A366338B8 for ; Fri, 21 May 2021 14:09:17 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "CN", Issuer "CN" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FmpPm2Wxyz3Nhy for ; Fri, 21 May 2021 14:09:15 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [IPv6:2a00:e10:2800:0:0:0:0:a135]) by mx0.gentlemail.de (8.15.2/8.15.2) with ESMTP id 14LE9AxH053576; Fri, 21 May 2021 16:09:12 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id C4B378FA; Fri, 21 May 2021 16:09:10 +0200 (CEST) Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root From: Harry Schmalzbauer To: Konstantin Belousov , Wei Hu Cc: "freebsd-virtualization@freebsd.org" References: <22bbe4cd-82ed-b826-9ac0-05ef8aceba0e@omnilan.de> <7986b561-dd26-a134-e76d-530900363deb@omnilan.de> Organization: OmniLAN Message-ID: Date: Fri, 21 May 2021 16:09:10 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Greylist: inspected by milter-greylist-4.6.2 (mx0.gentlemail.de [IPv6:2a00:e10:2800:0:0:0:0:a130]); Fri, 21 May 2021 16:09:14 +0200 (CEST) for IP:'2a00:e10:2800::a135' DOMAIN:'mh0.gentlemail.de' HELO:'mh0.gentlemail.de' FROM:'freebsd@omnilan.de' RCPT:'' X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx0.gentlemail.de [IPv6:2a00:e10:2800:0:0:0:0:a130]); Fri, 21 May 2021 16:09:14 +0200 (CEST) X-Rspamd-Queue-Id: 4FmpPm2Wxyz3Nhy X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd@omnilan.de designates 2a00:e10:2800::a130 as permitted sender) smtp.mailfrom=freebsd@omnilan.de X-Spamd-Result: default: False [-2.80 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[omnilan.de]; ARC_NA(0.00)[]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:e10:2800::a130:from]; NEURAL_HAM_LONG(-0.98)[-0.980]; NEURAL_HAM_SHORT(-1.00)[-1.000]; SPAMHAUS_ZRD(0.00)[2a00:e10:2800::a130:from:127.0.2.255]; NEURAL_HAM_MEDIUM(-0.52)[-0.516]; FREEMAIL_TO(0.00)[gmail.com,microsoft.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:61157, ipnet:2a00:e10:2800::/38, country:DE]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 May 2021 14:09:17 -0000 Am 21.05.2021 um 14:50 schrieb Harry Schmalzbauer: > Am 19.05.2021 um 13:26 schrieb Harry Schmalzbauer: > > ... >> I can just assure that I tracked it down to the single r332489 >> committ which makes vmbus(4) failing on i386. >> Verification done: >> - built/installed world&kernel on i386 HyV guest with r332488 >>   reboot fine. >> - built/installed kernel on the same vm after svnlite up -r 332489 >>   reboot fails with "vmbus0: cannot find free IDT vector" >> - loaded kernel.old/kernel, did revert 332489, by 'svnlite up -r >> 332488' and compiled/reinstalled kernel (again r332488) >>   reboot fine. > > While having hard time catching up with x86 basics, I found another > oddity: > On Hyper-V gen1, the maximum ammount of RAM I can assign a FreeBSD > guest beeing able to boot i386 is 3914MB, startring with r332489. > As soon as I configure the guest to have 3916MB or more, FreeBSD-i386 > r332489 booting leads to guest reset.  Loader starts kernel, machine > reboots. > Either reverting to r332488 or reducing guest-RAM solves this. > > The most interesting fact here is that I couldn't reproduce it on ESXi. > I ducplicated the Hyper-V HDD and started the exactly same code on > ESXi(v6) guest, where it doesn't crash!  I can boot FreeBSD-i386 > r332489 with any ammount of guest-RAM on ESXi. > (speaking for GENERIC, non-PAE) Now that I have a living r332489 i386 available, I checked some sysctl. It seems on ESXi and i386, large page mappings aren't used anymore, or is it just wrong indicating? These valueas are suspicious to my eyes: kern.version: FreeBSD 12.0-CURRENT #0 r332489: Fri May 21 15:32:35 CEST 2021     root@esxtest:/usr/obj/altroot/usr/src/i386.i386/sys/GENERIC hw.pagesizes: 4096 0 hw.realmem: 0 vm.min_kernel_address: 0 vm.max_kernel_address: 4286578688 vm.kvm_free: -549453824 vm.kvm_size: -16777216 vm.pmap.pg_ps_enabled: 0 kern.version: FreeBSD 12.0-CURRENT #1 r332488: Fri May 21 17:35:58 CEST 2021     root@esxtest:/usr/obj/altroot/usr/src/i386.i386/sys/GENERIC hw.pagesizes: 4096 4194304 hw.realmem: 0 vm.min_kernel_address: 3220172800 vm.max_kernel_address: 4294963200 vm.kvm_free: 624947200 vm.kvm_size: 1073737728 vm.pmap.pg_ps_enabled: 1 Thanks for any hints, -harry