Date: Thu, 8 Apr 2021 02:09:28 -0400 From: Daniel Morante <daniel@morante.net> To: freebsd-arm@freebsd.org Subject: Re: LSI SAS3008 on 13-RC5 and 14-CURRENT Message-ID: <29bea062-00a0-3271-7359-d6be1dfe5b02@morante.net> In-Reply-To: <1db2b369-cec6-058a-67ad-f762064b3832@morante.net> References: <1db2b369-cec6-058a-67ad-f762064b3832@morante.net>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format. --------------ms050103010305070600020508 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Content-Language: en-US Yikes... let me try to reformat the message display: > I done extensive testing with many different configurations. I=20 > apologize in advance for the length of this =F0=9F=98=89. > > I have an arm64 system with an onboard SAS controller (Broadcom=C2=AE=20 > SAS3008) and expander (Broadcom=C2=AE SAS35x36R). This system also has = an=20 > onboard SATA controller that is currently unusable in any version of=20 > FreeBSD ARM64=20 > (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D254651).=C2=A0 As = a=20 > result I am limited to using the SAS3008. > > I've noticed strange behavior on 13-RC3/4/5 and on the latest=20 > 14-CURRENT snapshot (2021-04-01).In the controller firmware I can=20 > create a logical volume (RAID) or use the individual drives=20 > (non-RAID/pass thru). In RAID mode there is a only logical volume and=20 > no individual drives. > > I use the DVD (or disc1) ISO image as the installation media.=C2=A0 I l= oad=20 > the kernel module (kldload mpr) using a shell and then proceed to=20 > install FreeBSD using either ZFS or UFS on root. > > With 13.0 the drives or logical volumes are detected correctly. On=20 > 14.0 no logical volumes are detected. > > mpr0: <Avago Technologies (LSI) SAS3008> mem=20 > 0x42140000-0x4214ffff,0x42100000-0x4213ffff at device 0.0 numa-domain=20 > 0 on pci14 > mpr0: Firmware: 15.00.00.00, Driver: 23.00.00.00-fbsd > mpr0: IOCCapabilities:=20 > 6985c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR,MSIXIndex,FastP= ath,RDPQArray> > mpr0: Found device <81<SataDev>,End Device> <6.0Gbps> handle<0x000a>=20 > enclosureHandle<0x0002> slot 0 > mpr0: At enclosure level 0 and connector name ( ) > mpr0: Found device <81<SataDev>,End Device> <6.0Gbps> handle<0x000b>=20 > enclosureHandle<0x0002> slot 1 > mpr0: At enclosure level 0 and connector name ( ) > mpr0: Found device <4411<SmpInit,SspTarg,SepDev>,End Device>=20 > <12.0Gbps> handle<0x000c> enclosureHandle<0x0002> slot 16 > mpr0: At enclosure level 0 and connector name ( ) > > This volume only shows up on 13.0 and not on 14.0: > > da1 at mpr0 bus 0 scbus2 target 0 lun 0 > da1: <LSI Logical Volume 3000> Fixed Direct Access SPC-4 SCSI device > da1: Serial Number 2692521697153322079 > da1: 150.000MB/s transfers > da1: Command Queueing enabled > da1: 226974MB (464842752 512 byte sectors) > > In Non-RAID mode individual drives will be shown in both 13.0 and 14.0.= > > I can proceed through the OS installation, but it's painfully slow.=C2=A0=20 > Towards the end (at which point I assume it's writing bootcode...=20 > before asking to set the root password), the following messages begin=20 > to appear in the console: > > mpr0: IOC Fault 0x40002622, Resetting > mpr0: Reinitializing controller > mpr0: Firmware: 15.00.00.00, Driver: 23.00.00.00-fbsd > mpr0: IOCCapabilities:=20 > 6985c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR,MSIXIndex,FastP= ath,RDPQArray> > mpr0: Found device <81<SataDev>,End Device> <6.0Gbps> handle<0x000a>=20 > enclosureHandle<0x0002> slot 0 > mpr0: At enclosure level 0 and connector name (=C2=A0=C2=A0=C2=A0 ) > mpr0: Found device <81<SataDev>,End Device> <6.0Gbps> handle<0x000b>=20 > enclosureHandle<0x0002> slot 1 > mpr0: At enclosure level 0 and connector name (=C2=A0=C2=A0=C2=A0 ) > mpr0: Found device <4411<SmpInit,SspTarg,SepDev>,End Device>=20 > <12.0Gbps> handle<0x000c> enclosureHandle<0x0002> slot 16 > mpr0: At enclosure level 0 and connector name (=C2=A0=C2=A0=C2=A0 ) > mpr0: IOC Fault 0x40002622, Resetting > mpr0: Reinitializing controller > mpr0: Firmware: 15.00.00.00, Driver: 23.00.00.00-fbsd > mpr0: IOCCapabilities:=20 > 6985c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR,MSIXIndex,FastP= ath,RDPQArray> > mpr0: Found device <81<SataDev>,End Device> <6.0Gbps> handle<0x000a>=20 > enclosureHandle<0x0002> slot 0 > mpr0: At enclosure level 0 and connector name (=C2=A0=C2=A0=C2=A0 ) > mpr0: Found device <81<SataDev>,End Device> <6.0Gbps> handle<0x000b>=20 > enclosureHandle<0x0002> slot 1 > mpr0: At enclosure level 0 and connector name (=C2=A0=C2=A0=C2=A0 ) > mpr0: Found device <4411<SmpInit,SspTarg,SepDev>,End Device>=20 > <12.0Gbps> handle<0x000c> enclosureHandle<0x0002> slot 16 > mpr0: At enclosure level 0 and connector name (=C2=A0=C2=A0=C2=A0 ) > mpr0: IOC Fault 0x40002622, Resetting > mpr0: Reinitializing controller > mpr0: Firmware: 15.00.00.00, Driver: 23.00.00.00-fbsd > mpr0: IOCCapabilities:=20 > 6985c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR,MSIXIndex,FastP= ath,RDPQArray> > mpr0: Found device <81<SataDev>,End Device> <6.0Gbps> handle<0x000a>=20 > enclosureHandle<0x0002> slot 0 > mpr0: At enclosure level 0 and connector name (=C2=A0=C2=A0=C2=A0 ) > mpr0: Found device <81<SataDev>,End Device> <6.0Gbps> handle<0x000b>=20 > enclosureHandle<0x0002> slot 1 > mpr0: At enclosure level 0 and connector name (=C2=A0=C2=A0=C2=A0 ) > mpr0: Found device <4411<SmpInit,SspTarg,SepDev>,End Device>=20 > <12.0Gbps> handle<0x000c> enclosureHandle<0x0002> slot 16 > mpr0: At enclosure level 0 and connector name (=C2=A0=C2=A0=C2=A0 ) > mpr0: IOC Fault 0x40002622, Resetting > mpr0: Reinitializing controller > mpr0: Firmware: 15.00.00.00, Driver: 23.00.00.00-fbsd > mpr0: IOCCapabilities:=20 > 6985c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR,MSIXIndex,FastP= ath,RDPQArray> > (da1:mpr0:0:0:0): Invalidating pack > g_vfs_done():da1p1[WRITE(offset=3D512, length=3D4096)]error =3D 6 > g_vfs_done():da1p1[WRITE(offset=3D33792, length=3D4096)]error =3D 6 > da1 at mpr0 bus 0 scbus2 target 0 lun 0 > da1: <LSI Logical Volume 3000>=C2=A0 s/n 2692521697153322079 detached > g_vfs_done():da1p1[WRITE(offset=3D99840, length=3D16384)]error =3D 6 > mpr0: Found device <81<SataDev>,End Device> <6.0Gbps> handle<0x000a>=20 > enclosureHandle<0x0002> slot 0 > mpr0: At enclosure level 0 and connector name (=C2=A0=C2=A0=C2=A0 ) > mpr0: Found device <81<SataDev>,End Device> <6.0Gbps> handle<0x000b>=20 > enclosureHandle<0x0002> slot 1 > mpr0: At enclosure level 0 and connector name (=C2=A0=C2=A0=C2=A0 ) > mpr0: Found device <4411<SmpInit,SspTarg,SepDev>,End Device>=20 > <12.0Gbps> handle<0x000c> enclosureHandle<0x0002> slot 16 > mpr0: At enclosure level 0 and connector name (=C2=A0=C2=A0=C2=A0 ) > mpr0: IOC Fault 0x40002622, Resetting > mpr0: Reinitializing controller > (da1:mpr0:0:0:0): WRITE(10). CDB: 2a 00 00 00 0a cb 00 08 00 00 > (da1:mpr0:0:0:0): CAM status: SCSI Bus Reset Sent/Received > (da1:mpr0:0:0:0): Error 5, Periph was invalidated > g_vfs_done():da1p1[WRITE(offset=3D1394176, length=3D1048576)]error =3D = 5 > (da1:mpr0:0:0:0): WRITE(6). CDB: 0a 00 01 0b 20 00 > (da1:mpr0:0:0:0): CAM status: SCSI Bus Reset Sent/Received > (da1:mpr0:0:0:0): Error 5, Periph was invalidated > g_vfs_done():da1p1[WRITE(offset=3D116224, length=3D16384)]error =3D 5 > fsync: giving up on dirty (error =3D 5) (da1:mpr0:0:0:0): WRITE(10).=20 > CDB: 2a 00 01 d7 0d 08 00 00 08 00 > 0xffffa0002797b5b8: (da1:mpr0:0:0:0): CAM status: SCSI Bus Reset=20 > Sent/Received > type VCHR > (da1:mpr0:0:0:0): Error 5, Periph was invalidated > =C2=A0=C2=A0=C2=A0 usecount 1, writecount 0, refcount 6 seqc users 0(da= 1:mpr0:0:0:0):=20 > WRITE(10). CDB: 2a 00 01 52 b0 a8 00 00 08 00 > =C2=A0rdev 0xffffa0000af73400 > (da1:mpr0:0:0:0): CAM status: SCSI Bus Reset Sent/Received > =C2=A0=C2=A0=C2=A0 hold count flags () > mpr0: Firmware: 15.00.00.00, Driver: 23.00.00.00-fbsd > =C2=A0=C2=A0=C2=A0 flags ()mpr0: IOCCapabilities:=20 > 6985c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR,MSIXIndex,FastP= ath,RDPQArray> > > (da1:mpr0:0:0:0): Error 5, Periph was invalidated > =C2=A0=C2=A0=C2=A0 v_object 0xffffa00027a6f528 ref 0 pages 27 cleanbuf = 2 dirtybuf 1 > =C2=A0=C2=A0=C2=A0 lock type devfs: EXCL by thread 0xffffa0000aac3000 (= pid 271,=20 > syncer, tid 101634) > =C2=A0with exclusive waiters pending > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 dev da1p1 > mpr0: Found device <81<SataDev>,End Device> <6.0Gbps> handle<0x000a>=20 > enclosureHandle<0x0002> slot 0 > mpr0: At enclosure level 0 and connector name (=C2=A0=C2=A0=C2=A0 ) > mpr0: Found device <81<SataDev>,End Device> <6.0Gbps> handle<0x000b>=20 > enclosureHandle<0x0002> slot 1 > mpr0: At enclosure level 0 and connector name (=C2=A0=C2=A0=C2=A0 ) > mpr0: Found device <4411<SmpInit,SspTarg,SepDev>,End Device>=20 > <12.0Gbps> handle<0x000c> enclosureHandle<0x0002> slot 16 > mpr0: At enclosure level 0 and connector name (=C2=A0=C2=A0=C2=A0 ) > > The above continues indefinitely and the system eventually locks up=20 > (no kernel panic).=C2=A0 I've tried it countless times and it always en= ds=20 > the same.=C2=A0 Regardless of how I configure the controller (RAID 1, 0= , or=20 > pass thru). > > To rule out a bad controller I test with Ubuntu 20.10 for arm64. No=20 > issues, everything is detected, SAS, onboard SATA, and Ethernet.=C2=A0 = Not=20 > surprising since Gigabyte has Ubuntu on the compatibility list for=20 > this system.=C2=A0 All hardware is confirmed to be functional. > > This is were things get strange... > > I go back to test FreeBSD 13.0 and I no longer have issues with the=20 > SAS controller (onboard SATA still not working).=C2=A0 It seems that=20 > installing/booting Ubuntu did something to 'fix' the problem?=C2=A0 May= be=20 > it loaded some type of firmware or places some persistent data in=20 > UEFI? (I'm not sure how that works). Regardless I was able to install=20 > FreeBSD on to the SAS controller, and I can boot it. > > Has anyone seen this happen before? Should I file another bug report? > > 14.0 btw, still can't see logical volumes.=C2=A0 Should that be a separ= ate=20 > bug report? > > --------------ms050103010305070600020508 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DJgwggX/MIID56ADAgECAgIQADANBgkqhkiG9w0BAQsFADCBkjELMAkGA1UEBhMCVVMxEDAO BgNVBAgMB0Zsb3JpZGExDzANBgNVBAcMBk5hcGxlczEpMCcGA1UECgwgVGhlIERhbmllbCBN b3JhbnRlIENvbXBhbnksIEluYy4xGDAWBgNVBAsMD1BhY3kgV29ybGQsIExMQzEbMBkGA1UE AwwSUGFjeSBXb3JsZCBSb290IENBMB4XDTE4MDcwNzE3MzUzMVoXDTI4MDcwNDE3MzUzMVow gYkxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMSkwJwYDVQQKDCBUaGUgRGFuaWVs IE1vcmFudGUgQ29tcGFueSwgSW5jLjEYMBYGA1UECwwPUGFjeSBXb3JsZCwgTExDMSMwIQYD VQQDDBpQYWN5IFdvcmxkIEludGVybWVkaWF0ZSBDQTCCAiIwDQYJKoZIhvcNAQEBBQADggIP ADCCAgoCggIBAMU0vt8sGT82c8BLAT+otS3VQVw4rf566bmYzJ7wm+D/vcp5jiwL1PAnGFch bSSpe1KCFQ7LCnjrl7nQg1EglHKETfHI7QxVqE22q92G/2rIDfPnNTYlLJ0d5xmM0q1P+yEI XYZErQyBthDhHfrDIEdWJ5tWevfFAUFjXmWy5V7eHGgndB8TQFEp9ML6Bks4tKH3ykwAZbib hKd7L2EthiqSJvvyIP3Kg/AwVjX67JTbKgQfcsfYGm5dofHmL+3uIhovLL5HFbADU6AoqXlh lNY83RfaD60G6IhPZwzZboBi6qxHnjwqsbV+S81SZdoMEnHDMZNzP4RC27ZMaTmxH3IN8JN4 XuszONstvUAuzdhmcG4q+K7HrO+SG0eEhDBUrxKrmFCUh8pwYU2o4q52/yDrZasid1NBFHZt Wfsyoe1ArhqnO4yz95h53dR9aCISwOGIMExNUJlH91KYNrLeSbPMR7LXgu48hY+CGkSYXhcS fPF9p9OUjgUeZ02+K8iI4a0CYCjuJNF1N/rJ7UkT0b5hdkcLPrXCd8P6NfJotoX6cGITF7Rm z+fq/3NbjF2Gu3nWeCdOXyECW9/HBzTlILZI10VOmfqDGZK3z6SlRBlaXK4lUvbqfDM+E785 CxStIlNqx/FDM6O8F+OP17Yb1vNpDhAblV/eYFiGZOw0YZAzAgMBAAGjZjBkMB0GA1UdDgQW BBR8ncAAcpI2KRTREkGd4Qpdx7IpVTAfBgNVHSMEGDAWgBR6BEY736L1bCcrmtwFaEg93nZd uTASBgNVHRMBAf8ECDAGAQH/AgEAMA4GA1UdDwEB/wQEAwIBhjANBgkqhkiG9w0BAQsFAAOC AgEAs/eYLE+jKO3hpHJ8QYaL8FH7iFFNbB4SVBDyH0arb9uqK/FY8uGlOn2A9ul8cE7sSA3E dXptZzFqY+C69UjlV2kvw2+35gvKoiIncwYhCA2JOQgm0QaXBAxZijF5UOL3+i1IE8JpdShz IoUvPtXBQiVnW9HJ5jZnZJNy4Jid+RQd5fNL2WmJf1Ob48YoeB23Y0KUnUvhmmu52OUG+bni hbP4ULTL4egmR783ZA5CqBWFb0J7HhXDaPBM1dGf+gQ5YzJqGWZXZ3YRe83IlDXm76rK+wlO uPlRUGSBbkgv7YRKzBx2JNLdNPBXIDxT71b1kGip8q0Mk9f/VXeudgCLzqa1+gphE3lNYQMa 0CHYfezask6ee5999rezWDx/T2/U0xy8i+bBlRNL7QRwk5JRTOemxbJJ5fzdLgwxEjkASPWy nIrf1O6C9f2vkUVGAWEo8wqhO+iLLmTRTt0GTsaIzA5aBmMz4KoByjOVwy6wAwoLTcFQrbGE jk5Cve+1AlXylTyokaeiYnKBjPMPSa5e3gcr3DfYQN5SbN3hXc8OFfkYzwqkZz4kMIWlP7pQ O0NqX8N+IiHJFc8BB8T8P4GRjO7jJYGtCJeTSVrO5tacpvJvg0BsHf48IpETH7Hs0oy1IXdZ DJQOceJCuOwDe7Q1ELHtIrjDeloN1CR+7M3qNc8wggaRMIIEeaADAgECAgIQHTANBgkqhkiG 9w0BAQsFADCBiTELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExKTAnBgNVBAoMIFRo ZSBEYW5pZWwgTW9yYW50ZSBDb21wYW55LCBJbmMuMRgwFgYDVQQLDA9QYWN5IFdvcmxkLCBM TEMxIzAhBgNVBAMMGlBhY3kgV29ybGQgSW50ZXJtZWRpYXRlIENBMB4XDTIxMDMyOTA0Mjc0 NloXDTMxMDMyNzA0Mjc0NlowgaYxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMQ8w DQYDVQQHDAZOYXBsZXMxJjAkBgNVBAoMHVBlcnNvbmFsIElkZW50aXR5IENlcnRpZmljYXRl MRAwDgYDVQQLDAdIb21lIFBDMRcwFQYDVQQDDA5EYW5pZWwgTW9yYW50ZTEhMB8GCSqGSIb3 DQEJARYSZGFuaWVsQG1vcmFudGUubmV0MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKC AgEAyjTH7YpZsiRpbQUpldun7uSfij/VVxewKjq5HusXO81CWrth5nYIe754ASllND1BkJkI Y1eRDDp8fgvEDmddk8+8idUlhcKNhtuGs1Z43Bk6PjQTNefzT2agbgrS+4pZsDG2xEuNx28L y1A0pLzmsUtLDZpuW6wnrN+1KlFnDcBBRjC6cZxGYogcz/Kd35f4OtIXdxNOuBMyJcn2W/h9 YZi6msZejTbp/Hod8sqG5SnoZAVr9rn4X1+/KIDJDItHf10hzX65AgU0NJXIxS5sTP8DStst twAgAXyKjaIE2RTTvxqMhlT1wJCKG5t63PSZ7ZIN8Quy2F2nNJr5MCBkA1+cKC7UDXrHs9xT OsuCf0iOhR3KfN5aRNkR1gdqInqtd/oscWhKByqtIjo/bi77NDcChRMS6gYF+lo5STnhAsFz z9fjNMOQQ4cDdZ0VffTv8wgGUOdo6LHyXI38AQUrJMRFdjiD/ol1j8++rGH9veTtSoLtbKVl YsdYXJ7CQniqS/BuL0PH9q35aDxhz2Thgdb1WpOgb8lpwUliKFaitXk7HUy/t6ThlWDhWtCm bFJcWdTKZoPUIrDxSzHMiVglIhkO33DAYZYBRC9TlhTMYTJJHioXIX+pjZ+xSzklEZo7I5Ob /08lomPW8C9do76EJcqdMEIjbAFTtcLnoTeIXpkCAwEAAaOB4zCB4DAJBgNVHRMEAjAAMA4G A1UdDwEB/wQEAwID+DAdBgNVHQ4EFgQUneNjyWA303ludVIz+7MVlPRjTzMwHwYDVR0jBBgw FoAUfJ3AAHKSNikU0RJBneEKXceyKVUwHQYDVR0RBBYwFIESZGFuaWVsQG1vcmFudGUubmV0 MCcGA1UdJQQgMB4GCCsGAQUFBwMEBggrBgEFBQcDAgYIKwYBBQUHAwgwOwYIKwYBBQUHAQEE LzAtMCsGCCsGAQUFBzAChh9odHRwOi8vd3d3LnBhY3l3b3JsZC5jb20vY2EucGhwMA0GCSqG SIb3DQEBCwUAA4ICAQCNZqZqo1Y8DXGyt6xMuTTya5+aUoIMhcwPpDwsze9l4sjEdrbngjOl lC5Cx0h31mL41oyoPbO3K+qjvbZHORasZRcEA/1uM4v5yV8s9DMlboEDLCwe3sjG7IwNXhcU GvAoVEalg9mQpr0Qfdb/m4VhDvGsqZa0wW7+vl8Sof6tPqIh7blLKwLHQPt1sZ8yYX4Sm2Bk IwpCfuNAniZOWWPQEYw2hVEqh1nhyhKBxt7W8ioEK/we6Nd4ivBjzEo3SmVvGcJHlDBfEOmH TqhMJp+wA30UJx4+4o2Z4frhCVIzhy4mKhLkG9a0Jj/ihJ/Cn/7R4ezDJ+a+FHim+sfsUST1 QhThcj2SIyfDaYqjE+zVx5yfp5aCdKG4xQQzPWZC0FGaBQ4ON6CGUe+XuBssiOXK7tkO5OQf q/qE+82UiwpVrbQ0by12V22zwVk/DiCEVd2L6Fhs2tDbdte2uLPFAhOyc4E7AW7VGjL81arV 0/ACWBdhuCaNOqvsk7koukM2lksY5FGvz4zARKIySjEyj2uI+iRWQFOjO4cea7UZKjHc44gf JBehl/Gdp8cdrAxo7fCnMbH3fYbagq+pF94bLcQZm61zO01Qz7TFfbaDv4YjHOilEAB30JdH JqWjYBp8LFj75VR6Ixt+uUtWVfWKBzlpIBZufbLsWNRzJZo9+5+lzjGCBQEwggT9AgEBMIGQ MIGJMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTEpMCcGA1UECgwgVGhlIERhbmll bCBNb3JhbnRlIENvbXBhbnksIEluYy4xGDAWBgNVBAsMD1BhY3kgV29ybGQsIExMQzEjMCEG A1UEAwwaUGFjeSBXb3JsZCBJbnRlcm1lZGlhdGUgQ0ECAhAdMA0GCWCGSAFlAwQCAwUAoIIC QTAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0yMTA0MDgwNjA5 MjhaME8GCSqGSIb3DQEJBDFCBEB3lTJTP3QN4hvYGfm700XBXyCIyjnVVzz7wNR27X9gJ93+ nAbal6HNRmua+YEc9Pckvzx+o5G1+9yGwPPKh+QLMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZI AWUDBAEqMAsGCWCGSAFlAwQBAjAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZI hvcNAwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwgaEGCSsGAQQBgjcQBDGBkzCBkDCB iTELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExKTAnBgNVBAoMIFRoZSBEYW5pZWwg TW9yYW50ZSBDb21wYW55LCBJbmMuMRgwFgYDVQQLDA9QYWN5IFdvcmxkLCBMTEMxIzAhBgNV BAMMGlBhY3kgV29ybGQgSW50ZXJtZWRpYXRlIENBAgIQHTCBowYLKoZIhvcNAQkQAgsxgZOg gZAwgYkxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMSkwJwYDVQQKDCBUaGUgRGFu aWVsIE1vcmFudGUgQ29tcGFueSwgSW5jLjEYMBYGA1UECwwPUGFjeSBXb3JsZCwgTExDMSMw IQYDVQQDDBpQYWN5IFdvcmxkIEludGVybWVkaWF0ZSBDQQICEB0wDQYJKoZIhvcNAQEBBQAE ggIAx82YPG+HKyQQgqAPRYupouHQxbyRyMZCYlRDsMITWQFXheJrpC+6ZtKkn/ZMeWc3gOV8 jG2tnVuIyIT05kP+xiMeJkcHo+61UKH87P7fb3GV0ObQw3iM1hl7DOMhGegwu4SS3c+RDhQZ /hvLSgCty7GoSf++dT2h6WvT8HuDVDVWanEHI27PCHm/TL6kwqfzdEhdcd/KQHWIRUKM1BW7 dl+IeWgPCTV4i39hnKxAERZIpwS5JPgnLVFqTaUKf/oAJ31CAEXM+4SKiXdJcRkwYdOY/dC4 SsEaT3IX4aLBCvL8iZUeKJxBS6D7jWttmAqVTdJel8S7ovI1CtCwEwe2ECsu3LV9H79ZC8du KdfFBAU0noA7NXesVEtZy3zxOw9ljgmY+JU3QLGu4z9wgZWnUi1LG7VeAGI4bV8A4q8jCXwt 6GgdQy3tM/qQ5Zl9Jayb7ZqKoS8V4RAT8Fv7fFRPhroRbrtoo2hEW8hVC9DUPGodLLyQvfa0 vdqGc1uhD7wquY8Ksm6aJutYgwM2hGlqF1SIrqZ9HnzfJI5FcLsW1KUksKUdtNIIhOjPOIWM CLPzfZW8UwrXB5fA1kGmTRrgWMK1Odd3MZ53ggJ51hA48tzQNvrQHzvek7TyLs6uraYD5lpb uJ6uGmQrEP505a3TxvCNtRd8ri25qOOjEiGltWoAAAAAAAA= --------------ms050103010305070600020508--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?29bea062-00a0-3271-7359-d6be1dfe5b02>