Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 20 Mar 2019 10:44:10 -0500
From:      Karl Denninger <karl@denninger.net>
To:        Ian Lepore <ian@freebsd.org>, freebsd-arm@freebsd.org
Subject:   Re: Options for FBSD support with LCD device - new project [[Maybe related: I2c issues on the Pi2]]
Message-ID:  <7058a207-f92f-2997-92a6-9e9c35acb59b@denninger.net>
In-Reply-To: <b4bad5eb0571127c7b73b3b0bedd8150b954bf62.camel@freebsd.org>
References:  <ad61a598-53af-02a5-41db-0128da7d1a34@optiplex-networks.com> <CAF19XBLAjP4yKtGSBzA4QdT346Bnbnr8MutQNZgmERLbJkWAyA@mail.gmail.com> <8df902f6-20a3-31c4-71ac-91f5d5fdf50d@optiplex-networks.com> <0ecf23e129ca7ac6a92a01bbb34c03f1ac8c6dc8.camel@freebsd.org> <e5d42c67-e1f2-ede1-965f-c89226de46da@optiplex-networks.com> <89f5b8d1ab0614ac8d88b5d5f1afc63e640c3c17.camel@freebsd.org> <4EB5C6C1-7DB9-4DEE-BB23-CD1259581271@jeditekunum.com> <004ddba628b94b80845d8e509ddcb648d21fd6c9.camel@freebsd.org> <C68D7E6E-03C1-448F-8638-8BD1717DBF44@jeditekunum.com> <ac7d434f16f3a89f5ef247678d6becdbeded5c3f.camel@freebsd.org> <CE40E2B5-2244-4EF9-B67F-34A54D71E2E8@jeditekunum.com> <f60ea6d2-b696-d896-7bcb-ac628f41f7b8@denninger.net> <b4bad5eb0571127c7b73b3b0bedd8150b954bf62.camel@freebsd.org>

next in thread | previous in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format.

--------------ms090107030102050007020009
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 3/20/2019 10:00, Ian Lepore wrote:
> On Tue, 2019-03-19 at 09:55 -0500, Karl Denninger wrote:
>> On 3/19/2019 09:26, Jedi Tek'Unum wrote:
>>> On Mar 18, 2019, at 2:57 PM, Ian Lepore <ian@freebsd.org> wrote:
>>>> On Mon, 2019-03-18 at 14:51 -0500, Jedi Tek'Unum wrote:
>>>>> My impression wasn=E2=80=99t that support wasn=E2=80=99t there - bu=
t =E2=80=9Cout of
>>>>> the box=E2=80=9D
>>>>> configuration wasn=E2=80=99t there. In comparison, I didn=E2=80=99t=
 have to do
>>>>> anything to get I2C enabled in the binary distribution of Linux
>>>>> that
>>>>> comes through the manufacturer.
>>>>>
>>>>> Its the enabling part that isn=E2=80=99t obvious to most people IMO=
=2E
>>>>>
>>>>> Documentation/wiki is great. But even better would be all the
>>>>> enabling overlays already in place and the entries in
>>>>> loader.conf
>>>>> already there and commented out. It would be so much easier to
>>>>> go to
>>>>> a =E2=80=9Ccommon place=E2=80=9D (loader.conf), skim through the no=
tes, find
>>>>> the
>>>>> thing that one wants, and then just uncomment the referenced
>>>>> line!
>>>>> (Or any other similarly easy method.)
>>>>>
>>>>>
>>>>> For FBSD to get a better foothold in this space it needs to be
>>>>> better
>>>>> documented. For example, the wiki for NEO2 <
>>>>> http://wiki.friendlyarm.com/wiki/index.php/NanoPi_NEO2>; is a
>>>>> step-by-
>>>>> step guide for how to acquire and configure Linux for it.
>>>>>
>>>>>
>>>> On one of my imx6 boards I have 5 SPI devices.  Each device can
>>>> use 3
>>>> or 4 different sets of pins for clock, data-in, and data-
>>>> out.  Plus,
>>>> each can use literally any number of whatever gpio pins they want
>>>> as
>>>> chip selects.  Even limiting the chipsels to a handfull, there
>>>> would
>>>> literally be thousands of possible combinations of devices and
>>>> pin
>>>> configurations, each one needing to be a separate overlay.
>>>>
>>>> Maybe you have experience primarily with rpi or some similarly
>>>> crippled
>>>> devices that only offer one or two choices?
>>> If memory serves correctly, there are only 2 I2C devices on the
>>> H3/H5 and the NanoPi NEO/2 implementations only externalize 1.
>>> There is only 1 SPI AFAIK.
>>>
>>> I wouldn=E2=80=99t call that crippled. I chose this platform exactly
>>> because of its characteristics - small, fast, cheap. It fits the
>>> project I=E2=80=99m using it for perfectly. In fact, I can see uses f=
or
>>> even smaller (see Giant Board <https://groboards.com/giant-board/>)
>>> . I understand other projects may have different requirements and
>>> would drive one towards different solutions - and require more of
>>> the various interfaces. But they aren=E2=80=99t going to be typical o=
f
>>> hobbyist projects.
>>>
>>> Maybe I should pose the question in another way. What is the
>>> philosophy for choosing GPIO as default for all the pins? These
>>> boards have a very limited number of pins and my preference would
>>> be that the broadest range of interface types would be the default.
>>> There are 2 UARTs exposed so I would have picked 1 to be enabled by
>>> default. After that, with I2C and SPI enabled, there are still 6
>>> GPIO available. For a tiny board like this that seems to be
>>> reasonable. If people have a need for slightly more GPIO then I
>>> would expect they would be the ones configuring overlays.
>>>
>>> Apparently the developers of the Linux packages for these boards
>>> have chosen the diverse approach (=E2=80=9CFriendlyCore=E2=80=9D base=
d on
>>> UbuntuCore Xenial).
>>>
>>> IMHO, most =E2=80=9Chobbyists=E2=80=9D would prefer the diversity app=
roach. I=E2=80=99m
>>> completely capable of becoming an expert in FBSD and this sort of
>>> configuration stuff yet it isn=E2=80=99t a priority for me - I just w=
ant to
>>> use it like any other hobbyist. The way things are now pushes this
>>> type of user away from FBSD.
>>>
>>> If there is some philosophical perspective against the diversity
>>> approach then the next best thing is to have documentation that
>>> clearly and simply tells people how to enable the other
>>> functionality.
>>>
>>> Finally, I think there is an opportunity to grow FBSD in the
>>> hobbyist world of these small products. We are past the point where
>>> people can have a real operating system running on systems at
>>> Arduino size and cost. Linux has been aggressively deployed there
>>> but I can say from experience that it ain=E2=80=99t pretty - I won=E2=
=80=99t say
>>> more as everyone reading this has a clear understanding of why that
>>> is.
>> I'm currently working an issue similar to this, but one that rates
>> "highly annoying" right now rather than "catastrophically bad."
>>
>> The environment is a RPI2 which has GPIO and I2c configured; GPIO to
>> drive outputs, I2c is used to read analog channels.
>>
>> On 11.0 this code ran perfectly well.
>>
>> On 12-STABLE )FreeBSD 12.0-STABLE r344818 GENERIC)
>>  it also runs well *BUT* generates a huge number of console messages
>> about spurious interrupts:
>>
>> intc0: Spurious interrupt detected
>> local_intc0: Spurious interrupt detected
>> intc0: Spurious interrupt detected
>> intc0: Spurious interrupt detected
>> local_intc0: Spurious interrupt detected
>> local_intc0: Spurious interrupt detected
>>
>> ....
>>
>> The issue is coming from the i2c side as I have another one of these
>> that has no I2c defined in the configuration (but is running
>> identical
>> code) and no messages.
>>
>> Something is indeed generating an /insane /number of interrupts on
>> one
>> of the channels:
>>
>> Interrupts
>> 530k total
>>  1159 local_intc
>> 494k local_intc
>>  8047 local_intc
>>
>> For obvious reasons I'd like to track this down (it's also generating
>> a
>> load average of 1.0, where it should be 0.1 or thereabouts) but I'm
>> not
>> sure where to start looking.
>>
>> config.txt looks like this:
>>
>> root@Pool-MCP:/mnt # cat config.txt
>> init_uart_clock=3D3000000
>> enable_uart=3D1
>> kernel=3Du-boot.bin
>> kernel7=3Du-boot.bin
>> dtoverlay=3Dmmc
>> #audio_pwm_mode=3D2
>> dtparam=3Di2c_arm=3Don
>>
>> The only "change" from what is in the default is the i2c_arm=3Don line=
=2E
>>
>> The "something" appears to be the i2c code, *or* it's something
>> that's
>> gone wrong in the DTS changes that are in the newer way of building
>> the
>> boot area (where the grab is of the "standard" versions from ports
>> and
>> then just copied over.)
>>
>> I suspect this would bite you equally hard if you had a RTC
>> configured
>> on I2c as well.....
>>
>> Killing the process that has the I2c interface open (so the I2c
>> interface is not in active use, but is configured of course) does
>> *not*
>> stop the insane interrupt storm.
>>
> I'm aware of this (haven't forgotten that you reported it), but I
> haven't had time to look into it, because of a crazy $work schedule
> right now.  I did some work on the rpi i2c driver last year, so there's=

> a chance I caused this problem.  I only have an ancient rpi-b to test
> with, I wonder if this is a problem that only happens on rpi2 models?
>
> -- Ian

I don't think so -- there's another report (Bernd Walter on this list)
with a Pi1 running 12-Release and an I2c RTC on it getting the same
messages (and, I assume, the same insane interrupt counts.)

--=20
Karl Denninger
karl@denninger.net <mailto:karl@denninger.net>
/The Market Ticker/
/[S/MIME encrypted email preferred]/

--------------ms090107030102050007020009
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature

MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC
DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL
MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw
FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf
BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4
MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD
dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1
ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK
AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI
KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD
0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY
vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn
uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24
SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E
6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH
YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL
h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd
zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE
FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q
EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ
TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5
c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY
MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC
AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN
gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9
oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj
tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K
uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv
HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK
17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/
Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA
6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY
UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO
62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp
ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD
QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx
MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw
FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw
ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A
16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg
96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg
y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs
YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg
6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX
SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM
cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN
5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l
CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg
CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v
b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF
oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG
SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O
BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC
GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE
BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh
IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ
zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN
AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR
dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu
2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X
lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK
FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q
IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA
1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL
shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ
qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE
QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL
MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM
TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM
QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw
GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwMzIwMTU0NDE0
WjBPBgkqhkiG9w0BCQQxQgRAVFFSWRp0UEPlAwZJaZHFvKlFP24DD+yBzNtHgqi2yG7P1Li+
uJXMBj/ZJ4+zpkY+cuAJGJZrW83NyzbVO+qV+TBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl
AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3
DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL
MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM
TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM
QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg
gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz
dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0
ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF
AASCAgA0dyPfKKziciMq27oPD0lLC5jYmogxczKZORaQzbzzXLz3oEPghF372qx8IOAxNGG0
/8K8OJEQzfTyg3MoFJpfo6ybSsy0f1msYrwSw5QdZ+fMBJpSuVASbMgD3G6ql1U+5g/kCG+w
q+buLoiizjt9jRccj8JwflR21Fr+w950kp+E9rFXwwRqFVerj1pK54OcIZB59MietkZzmn2u
iChSJTPV4DkCeX1n0Dz/drl/iG4klsE8Fk5fBHWjKa5CtoP3RrePp3+nkXh+8X4ZqJ+KTVrk
f+Vt9tsyvp35HJ1h2Z3OGildQHQhGKMrP289bHgdxRqgMMwRFJ+8pu+s6+2EgqYpHDoXzCP5
5G6IjC6y9+m94jisteZfoDHUVjtWr6M0giv4JgfaClCcNc+lKtv7tSjLzoqPmVem7MjFF63q
9afDOsry+XwYF1PNYnjczSbQdfYSfoFpaFxndAC7ekcLKjLs2uqCd1prmakaJlEpQ6a5Nhr7
4ShMLFT3LN6VpuSHYvIClxPWwyeXXe+NrHf4LvDpFCF014ZLykmuOWVQg0Oj9K2oZ4PCeAY8
nybsFhyfKTFt9R8r2fsZ8MCwVOUEGAyp5FVxQ0CiUwhz9x7mwiMWpo4OXojmZ4wkpoziBS8X
CCL20vFi3rBVGCBbwp8ySIt/ypGVxOEhlSr2r9yRwwAAAAAAAA==
--------------ms090107030102050007020009--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7058a207-f92f-2997-92a6-9e9c35acb59b>