From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 00:33:59 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2D83F80; Sun, 29 Mar 2015 00:33:59 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 75C33181; Sun, 29 Mar 2015 00:33:59 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1Yc1Ap-0005Aa-8U; Sun, 29 Mar 2015 03:33:55 +0300 Date: Sun, 29 Mar 2015 03:33:55 +0300 From: Slawa Olhovchenkov To: Adrian Chadd Subject: Re: irq cpu binding Message-ID: <20150329003354.GK23643@zxy.spb.ru> References: <20150328201219.GF23643@zxy.spb.ru> <20150328221621.GG23643@zxy.spb.ru> <20150328224634.GH23643@zxy.spb.ru> <20150328230533.GI23643@zxy.spb.ru> <20150328234116.GJ23643@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 00:34:00 -0000 On Sat, Mar 28, 2015 at 04:58:53PM -0700, Adrian Chadd wrote: > Hi, > > * It turns out that fragments were being 100% handled out of order > (compared to non-fragments in the same stream) when doing fragment > reassembly, because the current system was assuming direct dispatch > netisr and not checking any packet contents for whether they're on the > wrong CPU. I checked. It's not noticable unless you go digging, but > it's absolutely happening. That's why I spun a lot of cycles looking > at the IP fragment reassembly path and which methods get called on the > frames as they're reinjected. In case of fragmented packet we have first fragment (may be arrived not first) contained L4 information and dispatchet to correct bucket and other fragments, don't contains this information and dispathed anywere. As I understund IP stack gather all packet before processing. All we need -- do processing on CPU arriving first segment. > * We're going to have modify drivers, because the way drivers > currently assign interrupts, pick CPUs for queues, auto-select how > many queues to use, etc is all completely adhoc and not consistent. So Yes. I don't see problem (except re-binding IRQ by cpuset). All interesting drivers give tunable to control how many queues to use. I don't know how automate this: - one 1-port card - one 2-port card - one port of 2-port card - two 1-port card - two different card .... Manual select is aceptable here. > yeah, we're going to change the drivers and they're going to be > consistent and configurable. That way you can choose how you want to > distribute work and pin or not pin things - and it's not done adhoc > differently in each driver. Even igb, ixgbe and cxgbe differ in how > they implement these three things. > > * For RSS, there'll be a consistent configuration for what the > hardware is doing with hashing, rather than it being driver dependent. > Again, otherwise you may end up with some NICs doing 2-tuple hashing > where others are doing 4-tuple hashing, and behaviour changes > dramatically based on what NIC you're using. What's problem there? I am don't intersting how NIC do hashing (anyway, hashing for direct and reflex traffic is different -- this is not Tilera). All I need -- distributing flow to CPU, for balance load and reduction lock congenstion. > * For applications - I'm not sure yet, but at the minimum the librss > API I have vaguely sketched out and coded up in a git branch lets you > pull out the list of buckets and which CPU it's on. I'm going to > extend that a bit more, but it should be enough for things like nginx > to say "ok, start up one nginx process per RSS bucket, and here's the > CPU set for it to bind to." You said it has worker groups - that's > great; I want that to be auto configured. For applications minimum is (per socket) select/kqueut/accept work only for flow, arrived at CPU matched CPU at time select/kqueut/accept (yes, for correct work application must pined to this CPU). And application don't need know anything about buckets and etc. After this, arrived packet activated IRQ handler, ithread, driver interrup thread, TCP stack, select/accept, read, write, tcp_output -- all on same cpu. I can be wrong, this is save L2/L3 cache. Where I missunderstund? From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 02:56:20 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 169C3FCF; Sun, 29 Mar 2015 02:56:20 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 986EBF0E; Sun, 29 Mar 2015 02:56:19 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DRDACMaBdV/95baINcgygwXASDD8JKCoUqSQKBawEBAQEBAX1BAoNSAQEEAQEBCxUrFwkLGw4EBgICDRkCKQEJGA4GCAcEARwEiA4NsgyYZAEBAQEBBQEBAQEBAQEbgSGKCIQgAQYBARs0B4ItDC8SgTMFlE2DXYNCOoVSjQ0ihAoiMQd7AQgXIn8BAQE X-IronPort-AV: E=Sophos;i="5.11,486,1422939600"; d="scan'208";a="200302379" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 28 Mar 2015 22:56:17 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id ED295B3F77; Sat, 28 Mar 2015 22:56:17 -0400 (EDT) Date: Sat, 28 Mar 2015 22:56:17 -0400 (EDT) From: Rick Macklem To: Konstantin Belousov Message-ID: <69948517.7558875.1427597777962.JavaMail.root@uoguelph.ca> In-Reply-To: <20150328171315.GU2379@kib.kiev.ua> Subject: Re: MAXBSIZE increase MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.10] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-fs@FreeBSD.org, freebsd-hackers@freebsd.org, Alexander Motin X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 02:56:20 -0000 Kostik wrote: > On Fri, Mar 27, 2015 at 10:57:05PM +0200, Alexander Motin wrote: > > Hi. > > > > Experimenting with NFS and ZFS I found an inter-operation issue: > > ZFS by > > default uses block of 128KB, while FreeBSD NFS (both client and > > server) > > is limited to 64KB requests by the value of MAXBSIZE. On file > > rewrite > > that limitation makes ZFS to do slow read-modify-write cycles for > > every > > write operation, instead of just writing the new data. Trivial > > iozone > > test show major difference between initial write and rewrite speeds > > because of this issue. > > > > Looking through the sources I've found and in r280347 fixed number > > of > > improper MAXBSIZE use cases in device drivers. After that I see no > > any > > reason why MAXBSIZE can not be increased to at least 128KB to match > > ZFS > > default (ZFS now supports block up to 1MB, but that is not default > > and > > so far rare). I've made a test build and also successfully created > > UFS > > file system with 128KB block -- not sure it is needed, but seems it > > survives this change well too. > > > > Is there anything I am missing, or it is safe to rise this limit > > now? > > This post is useless after the Bruce explanation, but I still want to > highlidht the most important point from that long story: > > increasing MAXBSIZE without tuning other buffer cache parameters > would dis-balance the buffer cache. Allowing bigger buffers > increases > fragmentation, while limiting the total number of buffers. Also, it > changes the tuning for runtime limits for amount of io in flight, see > hi/lo runningspace initialization. >From an NFS perspective, all it cares about is the maximum size of buffer cache block it can use. Maybe creating a separate constant that is specifically "max buffer cache block size", but does not define the maximum size of any file system's block would help? If the constant only defines maximum buffer cache block size, then it could be tuned based on architecture, so that amd64 could use much larger values for the buffer cache tunables. (As Bruce explained, i386 puts a very low limit on the buffer cache, due to KVM limitations.) Put another way, separate maximum buffer cache block from the maximum block size used by any on-disk file system. Other than KVM limits, I think the problems with increasing MAXBSIZE are because it is used as a maximum block size for file systems like UFS. Btw, since NFS already uses 64K buffers by default, there is already a dis-balanced buffer cache. Unfortunately, increasing BKVASIZE will make it allow even fewer buffers for i386. I don't know if buffer cache fragmentation has been causing anyone problems? Does anyone know if buffer cache fragmentation can cause any failure or will it just impact performance? (All I can see is that allocation of buffers > BKVASIZE can fragment the buffer cache's address space such that there might not be a contiguous area large enough for a buffer's allocation. I don't know what happens then?) I would like to see the NFS client be able to use 128K rsize/wsize. I would also like to see a larger buffer cache for machines like amd64 with a lot of RAM, so that wcommitsize (the size of write that the client can do asynchronously) can be much larger, too. (For i386, we probably have to live with a small buffer cache and maybe a 64K maximum buffer cache block size.) rick > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 05:46:55 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 12749614 for ; Sun, 29 Mar 2015 05:46:55 +0000 (UTC) Received: from mail-ig0-x231.google.com (mail-ig0-x231.google.com [IPv6:2607:f8b0:4001:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C53AAB1 for ; Sun, 29 Mar 2015 05:46:54 +0000 (UTC) Received: by igbud6 with SMTP id ud6so50384832igb.1 for ; Sat, 28 Mar 2015 22:46:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=oaCjc5j2KBw4djgruhCQL9/rMMKvlm2jIBomIWevq7Q=; b=gX5q5mGv+8wP4KzayOC2HZnWYCMRtZb0A1ahL9uyKAtJdE2JyxkjHAWTwNeSiIYzkQ 2EiIfWGHXKM16EkD3dU0BbpxYZ2xJAG7oxW84cTzvM+96bHeQToj4zJKX92YF82OFFSp CkiPpYetWKSU6kNyb5nWma90hlb0g6aJPhCyRkkGnixaemBQW6fu5bzmeQqEO+dsL3LW dvhnadyNhY4614zRdwncAj/GehI/6DrYE48TxKRrCLOuOzJnpgonUd1nYg3EbFcMX52P pxgryTjwEunWWe7eq4Ij2ItNvSGlUWJM8JnoQhPaLGzGxzzxZ3l9j07YevkSQ0D/4wsV 9pew== MIME-Version: 1.0 X-Received: by 10.107.155.13 with SMTP id d13mr39758797ioe.29.1427608014160; Sat, 28 Mar 2015 22:46:54 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Sat, 28 Mar 2015 22:46:54 -0700 (PDT) In-Reply-To: <20150329003354.GK23643@zxy.spb.ru> References: <20150328201219.GF23643@zxy.spb.ru> <20150328221621.GG23643@zxy.spb.ru> <20150328224634.GH23643@zxy.spb.ru> <20150328230533.GI23643@zxy.spb.ru> <20150328234116.GJ23643@zxy.spb.ru> <20150329003354.GK23643@zxy.spb.ru> Date: Sat, 28 Mar 2015 22:46:54 -0700 X-Google-Sender-Auth: YPCnY1-85xH-vpjeP6xivneSz18 Message-ID: Subject: Re: irq cpu binding From: Adrian Chadd To: Slawa Olhovchenkov Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 05:46:55 -0000 On 28 March 2015 at 17:33, Slawa Olhovchenkov wrote: > On Sat, Mar 28, 2015 at 04:58:53PM -0700, Adrian Chadd wrote: > >> Hi, >> >> * It turns out that fragments were being 100% handled out of order >> (compared to non-fragments in the same stream) when doing fragment >> reassembly, because the current system was assuming direct dispatch >> netisr and not checking any packet contents for whether they're on the >> wrong CPU. I checked. It's not noticable unless you go digging, but >> it's absolutely happening. That's why I spun a lot of cycles looking >> at the IP fragment reassembly path and which methods get called on the >> frames as they're reinjected. > > In case of fragmented packet we have first fragment (may be arrived > not first) contained L4 information and dispatchet to correct bucket > and other fragments, don't contains this information and dispathed > anywere. As I understund IP stack gather all packet before processing. > All we need -- do processing on CPU arriving first segment. I'm pretty sure that wasn't what was happening when i went digging. I was using UDP and varying the transmit size so I had exact control over the fragmentation. The driver rx path does direct dispatch netisr processing, and for fragments it was hashed on only L3 details not L4. Even the first frame is hashed on L3 only. So it'd go to a different queue compared to L4 hashing, and subsequent fragments would come in on the same queue. Once it was completed, it was processed up inline - it wasn't going back into netisr and getting re-checked for the right queue. >> * We're going to have modify drivers, because the way drivers >> currently assign interrupts, pick CPUs for queues, auto-select how >> many queues to use, etc is all completely adhoc and not consistent. So > > Yes. I don't see problem (except re-binding IRQ by cpuset). > All interesting drivers give tunable to control how many queues to > use. I don't know how automate this: > > - one 1-port card > - one 2-port card > - one port of 2-port card > - two 1-port card > - two different card > .... > > Manual select is aceptable here. > >> yeah, we're going to change the drivers and they're going to be >> consistent and configurable. That way you can choose how you want to >> distribute work and pin or not pin things - and it's not done adhoc >> differently in each driver. Even igb, ixgbe and cxgbe differ in how >> they implement these three things. >> >> * For RSS, there'll be a consistent configuration for what the >> hardware is doing with hashing, rather than it being driver dependent. >> Again, otherwise you may end up with some NICs doing 2-tuple hashing >> where others are doing 4-tuple hashing, and behaviour changes >> dramatically based on what NIC you're using. > > What's problem there? > I am don't intersting how NIC do hashing (anyway, hashing for direct > and reflex traffic is different -- this is not Tilera). > All I need -- distributing flow to CPU, for balance load and reduction > lock congenstion. Right, but you assume all packets in a flow go to the same CPU, and I discovered this wasn't the case. That's why I went down the path with RSS to make it right. > >> * For applications - I'm not sure yet, but at the minimum the librss >> API I have vaguely sketched out and coded up in a git branch lets you >> pull out the list of buckets and which CPU it's on. I'm going to >> extend that a bit more, but it should be enough for things like nginx >> to say "ok, start up one nginx process per RSS bucket, and here's the >> CPU set for it to bind to." You said it has worker groups - that's >> great; I want that to be auto configured. > > For applications minimum is (per socket) select/kqueut/accept work > only for flow, arrived at CPU matched CPU at time select/kqueut/accept > (yes, for correct work application must pined to this CPU). > > And application don't need know anything about buckets and etc. > > After this, arrived packet activated IRQ handler, ithread, driver > interrup thread, TCP stack, select/accept, read, write, tcp_output -- > all on same cpu. I can be wrong, this is save L2/L3 cache. > > Where I missunderstund? The other half of the network stack - the sending side - also needs to be either on the same or nearby CPU, or you still end up with lock contention and cache thrashing. -a From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 08:19:04 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BFBE638B; Sun, 29 Mar 2015 08:19:04 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 71699F9C; Sun, 29 Mar 2015 08:19:04 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1Yc8Qw-0009Ms-Ff; Sun, 29 Mar 2015 11:19:02 +0300 Date: Sun, 29 Mar 2015 11:19:02 +0300 From: Slawa Olhovchenkov To: Adrian Chadd Subject: Re: irq cpu binding Message-ID: <20150329081902.GN23643@zxy.spb.ru> References: <20150328221621.GG23643@zxy.spb.ru> <20150328224634.GH23643@zxy.spb.ru> <20150328230533.GI23643@zxy.spb.ru> <20150328234116.GJ23643@zxy.spb.ru> <20150329003354.GK23643@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 08:19:04 -0000 On Sat, Mar 28, 2015 at 10:46:54PM -0700, Adrian Chadd wrote: > >> * It turns out that fragments were being 100% handled out of order > >> (compared to non-fragments in the same stream) when doing fragment > >> reassembly, because the current system was assuming direct dispatch > >> netisr and not checking any packet contents for whether they're on the > >> wrong CPU. I checked. It's not noticable unless you go digging, but > >> it's absolutely happening. That's why I spun a lot of cycles looking > >> at the IP fragment reassembly path and which methods get called on the > >> frames as they're reinjected. > > > > In case of fragmented packet we have first fragment (may be arrived > > not first) contained L4 information and dispatchet to correct bucket > > and other fragments, don't contains this information and dispathed > > anywere. As I understund IP stack gather all packet before processing. > > All we need -- do processing on CPU arriving first segment. > > I'm pretty sure that wasn't what was happening when i went digging. I > was using UDP and varying the transmit size so I had exact control > over the fragmentation. > > The driver rx path does direct dispatch netisr processing, and for > fragments it was hashed on only L3 details not L4. Even the first > frame is hashed on L3 only. So it'd go to a different queue compared > to L4 hashing, and subsequent fragments would come in on the same > queue. Once it was completed, it was processed up inline - it wasn't > going back into netisr and getting re-checked for the right queue. Two case: 1) let this behavior 2) rewrite fo resheduling. I think 1) acceptable -- fragmented packets very rarely, compared to target data rate (2Mpps and more). > > What's problem there? > > I am don't intersting how NIC do hashing (anyway, hashing for direct > > and reflex traffic is different -- this is not Tilera). > > All I need -- distributing flow to CPU, for balance load and reduction > > lock congenstion. > > Right, but you assume all packets in a flow go to the same CPU, and I > discovered this wasn't the case. > That's why I went down the path with RSS to make it right. Only fragmented packets case or other case? > > > >> * For applications - I'm not sure yet, but at the minimum the librss > >> API I have vaguely sketched out and coded up in a git branch lets you > >> pull out the list of buckets and which CPU it's on. I'm going to > >> extend that a bit more, but it should be enough for things like nginx > >> to say "ok, start up one nginx process per RSS bucket, and here's the > >> CPU set for it to bind to." You said it has worker groups - that's > >> great; I want that to be auto configured. > > > > For applications minimum is (per socket) select/kqueut/accept work > > only for flow, arrived at CPU matched CPU at time select/kqueut/accept > > (yes, for correct work application must pined to this CPU). > > > > And application don't need know anything about buckets and etc. > > > > After this, arrived packet activated IRQ handler, ithread, driver > > interrup thread, TCP stack, select/accept, read, write, tcp_output -- > > all on same cpu. I can be wrong, this is save L2/L3 cache. > > > > Where I missunderstund? > > The other half of the network stack - the sending side - also needs to > be either on the same or nearby CPU, or you still end up with lock > contention and cache thrashing. For incoming connections this will be automatuc -- sending will be from CPU binding to receiving queue. Outgoing connections is more complex case, yes. Need to transfer FD (with re-binding) and signaling (from kernel to application) about prefered CPU. Prefered CPU is CPU give SYN-ACK. And this need assistance from application. But I am currently can't remember application massive servering outgouing connections. From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 13:01:32 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4A4BBF47; Sun, 29 Mar 2015 13:01:32 +0000 (UTC) Received: from mail-wi0-x229.google.com (mail-wi0-x229.google.com [IPv6:2a00:1450:400c:c05::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CD6A0EAD; Sun, 29 Mar 2015 13:01:31 +0000 (UTC) Received: by wicne17 with SMTP id ne17so1451791wic.0; Sun, 29 Mar 2015 06:01:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=khvIjErxUUULF5nFNMtGAymHhuz5iea9U0Z079fgK/U=; b=dO+5Jlx1S0TebzTRKBcsphd559BtsaoCWm/ML0VZ8zr6wYzC0ACltK0mCwWm0JTBbf I8/6FPw5fGBwiGqq7HyKonScnKjFBvpI4Qb0sJnGytUKR+fgWe8jiDrCo+LwZjalRigU l6r4KAFloR5cpOt01PfUkk856ntQQv0yQEYK2BzcDDkvqyvl6KTcFcyJACeZn8rNlowt s0rQhq+++h7/EPBpapaUqOhBAM/Pk9AAauItmACv3ubNm15+nI6FMAAwDe65RUaIjMU3 ntmuYgNviyaLQ4wGlvr3eNbamhaR3Uy+x3PCCl7+WeQ1y3bfcR0c08Wsp94/xOPYiwdV 2Dog== X-Received: by 10.194.200.229 with SMTP id jv5mr54394927wjc.59.1427634090358; Sun, 29 Mar 2015 06:01:30 -0700 (PDT) Received: from [192.168.190.100] (cpc74755-dals16-0-0-cust98.20-2.cable.virginm.net. [80.195.239.99]) by mx.google.com with ESMTPSA id lb6sm11160423wjb.22.2015.03.29.06.01.28 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 29 Mar 2015 06:01:29 -0700 (PDT) References: <55168627.5060509@riseup.net> <5516CF6C.6080808@riseup.net> Mime-Version: 1.0 (1.0) In-Reply-To: <5516CF6C.6080808@riseup.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: X-Mailer: iPhone Mail (11D257) From: Sevan / Venture37 Subject: Re: Libreboot X200 and FreeBSD Date: Sun, 29 Mar 2015 14:01:25 +0100 To: Piotr Kubaj X-Mailman-Approved-At: Sun, 29 Mar 2015 13:43:23 +0000 Cc: "freebsd-hackers@freebsd.org" , Adrian Chadd , "freebsd-current@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 13:01:32 -0000 > On 28 Mar 2015, at 15:57, Piotr Kubaj wrote: >=20 > It's a modified Thinkpad X200, so it uses the same chipset, i.e. GM45. Hi Piotr, Unfortunately its not possible to boot freebsd with the libreboot images due= to libreboot not have a binary blob vga bios & seabios payload. You should be able to run freebsd without issue using coreboot however (libr= eboot is a consumer of coreboot which produces images without any close comp= onents). I've managed to reflash a stock x60s with coreboot & run net/free & dragonfl= ybsd on there. I don't believer the laptops are modified physically in anyway & just have t= heir bios reflashed, might be worth asking them if you can get a laptop with= coreboot+vgabios+seabios. Alternatively, just get a thinkpad x200 & diy. You may have to resort to using linux for the initial flash. If it goes wrong & provided you made backup images before starting, you can r= ecover without issue using a raspberry pi or a beaglebone black as your repr= ogrammer & the flashrom utility. Dmesg of the x60s running coreboot is on the wiki & freebsd-acpi@ Regards Sevan / Venture37= From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 13:02:41 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D9C27DE; Sun, 29 Mar 2015 13:02:41 +0000 (UTC) Received: from mail-wg0-x22f.google.com (mail-wg0-x22f.google.com [IPv6:2a00:1450:400c:c00::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 685B5EBE; Sun, 29 Mar 2015 13:02:41 +0000 (UTC) Received: by wgra20 with SMTP id a20so143261955wgr.3; Sun, 29 Mar 2015 06:02:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=zeyb+mImK1X2kOgTADmg5xCxIzNlSE5hC7OCbTYPsTI=; b=YYKIwBh3zxxx2J7QnvNjJGiYvUnL+6odrvsUrKVyxtwXnuoJTyO8m/ZnVKcGfz31f2 3s4G9lEveqtq4o6C3P+Aw9AiXTugYKRwMDreWuv57ZXdzfFROlkis+njK28V75g3PBRw pXNEkj+ArcsMe/n3/nC5AzZktR+9Nnt3rw1PEAq7K+KNXr0f22FM39lK7tmnB0TPnYwv +OYAPzPmd/wPy27wCEXiJzYCcjDqHOcIulg0bA815i+vYUnJcprClxrdZ9TLbGiJrgPy 1tfpZejnXjaxvHXvu8mQDARTTRf/Lu82koiFGdoUDXl9lWxvjS/mSVfPXrIwsgwSvMCe rxkg== X-Received: by 10.180.89.34 with SMTP id bl2mr13415869wib.23.1427634159918; Sun, 29 Mar 2015 06:02:39 -0700 (PDT) Received: from [192.168.190.100] (cpc74755-dals16-0-0-cust98.20-2.cable.virginm.net. [80.195.239.99]) by mx.google.com with ESMTPSA id g2sm15520983wib.1.2015.03.29.06.02.38 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 29 Mar 2015 06:02:39 -0700 (PDT) References: <55168627.5060509@riseup.net> <5516CF6C.6080808@riseup.net> Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: X-Mailer: iPhone Mail (11D257) From: Sevan / Venture37 Subject: Re: Libreboot X200 and FreeBSD Date: Sun, 29 Mar 2015 14:02:35 +0100 To: Adrian Chadd X-Mailman-Approved-At: Sun, 29 Mar 2015 13:43:34 +0000 Cc: "freebsd-hackers@freebsd.org" , freebsd-current , Piotr Kubaj X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 13:02:42 -0000 > On 28 Mar 2015, at 17:42, Adrian Chadd wrote: > > Oh, in that case, someone should send me one so I can use it to verify > that my frame-buffer bootloader hack will work correctly on it. Why don't you reflash your existing one? :) Sevan / Venture37 From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 13:48:25 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A1E48813; Sun, 29 Mar 2015 13:48:25 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B70212EB; Sun, 29 Mar 2015 13:48:24 +0000 (UTC) Received: by wiaa2 with SMTP id a2so91369055wia.0; Sun, 29 Mar 2015 06:48:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=6SzejxQgaRrdx0NV6rOYs4C4QpFVkdrqnIaCL2cbbiA=; b=EVseqHFXAaeK7VJwKhGXM5zwJHQa4Qj3w2/t2eW15zvIU/TiASjH1fQvsroVR45qWo V8+vg81YL4XmIIFVi9HySem6AysONqk7vAABtpt6/f4PtI4aEbG9nINE8FhcrHKVV6ji UXp2ROlSmLZuDsVsnC2GRRq/XsW0BMh6A7/sG7r7vbAKjV2eLeR7KVdoAVfuPuh+xI9r AN0Fnf4R4muohjNjjWzDz24t89xk3kyMtX2Kx1OqdmivYzbZOPQwrYjrbcj6AUSzUSQj q1bkMlauFIkhjwi83KpZbJNtRgqaDg7rV05XMbPCyp//eO89AoRcIfziI0QtKogIz5Fr 3mJA== MIME-Version: 1.0 X-Received: by 10.181.8.76 with SMTP id di12mr5049177wid.75.1427636903050; Sun, 29 Mar 2015 06:48:23 -0700 (PDT) Received: by 10.28.228.196 with HTTP; Sun, 29 Mar 2015 06:48:22 -0700 (PDT) In-Reply-To: References: Date: Sun, 29 Mar 2015 21:48:22 +0800 Message-ID: Subject: Re: Self introduciton of Hao Sun and thoughts on LibNetstat From: Hao Sun To: Gabor Pali Content-Type: multipart/mixed; boundary=001a1135f0aa79a4e505126d9ec7 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: George Neville-Neil , Robert Watson , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 13:48:25 -0000 --001a1135f0aa79a4e505126d9ec7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi G=C3=A1bor, I've submitted the proposal of the libnetstat project on Friday. The attached PDF version of proposal is just FYI. Any questions or suggestions please feel free to contact me. Thanks, Hao 2015-03-24 23:00 GMT+08:00 Hao Sun : > Hi G=C3=A1bor, > > Thanks a lot for the advice and suggestions. Based on our discusstion and > initial investigation, I roughly draw a route map for the LibNetstat of > GSoC. I'd like to follow the following process to get down to more detail= s > from requirement to implementation. > > 0. Warm up. I'll build the FreeBSD, just as your suggestion, the "world" > version, and then install the system together with setting up the dev > environment. Hopefully after this step I'll get familar with the FreeBSD > dev process and be well prepared for the next steps. > > 1. Requirement gathering. The main task of our project is to create > wrapper libraries to support monitoring and management applications to > avoid direct use of kvm(3) and sysctl(3) interfaces. In this step, I plan > to read the kvm(3) and sysctl(3) implementations and filter out the > interfaces and functions which are related to networks. Then I'll classif= y > all the requirements in order to get different modules of the LibNetstat. > After this step, I should have a vague idea about creating the abstaction= s > for the modules. > > 2. Case study. In this step, I want to learn about the Libmemstat(3) and > existing LibNetstat implementations in order to get inspirations and > reuseable code snippets. After the case study, hopefully I'll be cleared > how to make the abstraction. > > 3. Create abstraction. After Step #2 and #3, I'll start to write code fro= m > sketch and the interfaces should be well defined after this step. > > 4. Core implementation. Based on the kvm(3) and sysctl(3), I'll create th= e > detailed implementation for the library. > > 5. Applications update preparation. Now our library is ready! In this ste= p > I'd like to read the implemation of netstat(1) and bsnmpd(1) and point ou= t > which part could be replaced with our new library. > > 6. Applications update. Update the netstat(1) and bsnmpd(1), also test th= e > new library. > > 7. Documentation. Maybe we need to create a manual page for the library. > > The above are my initial ideas. I'd like to dive into each step during th= e > next few days and add more details in my final proposal. Also I plan add > some buffer time in case there is a unexpected delay and some "stretch > goals" in case the abrove goals are done quickly :). > > I also feel excited to corporate and share my ideas with you, Robert > and George. Any suggestions or supplements are really welcomed :) > > Thanks, > Hao > > 2015-03-23 14:50 GMT+08:00 Gabor Pali : > >> Hi Hao, >> >> 2015-03-22 13:09 GMT+01:00 Hao Sun : >> > Following your guidance, I've cloned the FreeBSD mirror from GitHub an= d >> will get down to >> > have an initial scratch with the latest version. >> >> That is great. I forgot to mention, but maybe it is also said on the >> introductory wiki pages you have also cited originally, that it is >> probably the best if you try to build the userland ("world") and the >> kernel on your VM and install it first. This should give you the >> basic feel for the development cycle and help to spot problems with >> the clean system itself before you start hacking on it. The >> development branch of FreeBSD (that is called "current") shall build >> and install just fine for most of the time, but do not be discouraged >> if not, ask for help. >> >> > On the project's wiki page [1], I think the target for GSoC >> > 2015 is to finish the tasks haven't been done in the following table. >> But according to your >> > comments in the emails, it seems like I need to start the job from >> scratch. Thus the question >> > is should I keep the existing code and add new features to the previou= s >> version or just start >> > the project from the very beginning? >> >> I might have sounded a bit pessimistic, I do not necessarily insist on >> rewriting the entire library :-) I think it is just common sense: >> study the current implementation, take a look at the FreeBSD ecosystem >> and kernel, discuss the topic with the interested hackers, and work >> out your proposal based on your findings. You may find some of the >> old code base and concepts reusable, which is excellent, and you may >> decide to take another approach for the rest. It might be worthwhile >> to accommodate some "stretch goals" in your proposal if you >> accidentally completed your summer task too quickly :-) >> >> For making things a bit easier (hopefully) for you, I may also include >> George Neville-Neil in the conversation (see him CC'ed) who has shown >> some interest in driving this library into the base system in the past >> if I recall correctly. Along with Robert, he is also a high-profile >> src committer, with experience in networking and related areas. (And >> also a potential mentor for this project as well?) >> >> Cheers, >> G=C3=A1bor >> >> [1] https://wiki.freebsd.org/LibNetstat >> > > --001a1135f0aa79a4e505126d9ec7 Content-Type: application/pdf; name="FreeBSD_Proposal_Hao.pdf" Content-Disposition: attachment; filename="FreeBSD_Proposal_Hao.pdf" Content-Transfer-Encoding: base64 X-Attachment-Id: f_i7ui58s30 JVBERi0xLjUKJdDUxdgKMSAwIG9iago8PCAvUyAvR29UbyAvRCAoc2VjdGlvbi4xKSA+PgplbmRv YmoKNCAwIG9iagooSW50cm9kdWN0aW9uKQplbmRvYmoKNSAwIG9iago8PCAvUyAvR29UbyAvRCAo c2VjdGlvbi4yKSA+PgplbmRvYmoKOCAwIG9iagooUHJvamVjdCBHb2FscykKZW5kb2JqCjkgMCBv YmoKPDwgL1MgL0dvVG8gL0QgKHN1YnNlY3Rpb24uMi4xKSA+PgplbmRvYmoKMTIgMCBvYmoKKEdl bmVyYWwgVGFyZ2V0cykKZW5kb2JqCjEzIDAgb2JqCjw8IC9TIC9Hb1RvIC9EIChzdWJzZWN0aW9u LjIuMikgPj4KZW5kb2JqCjE2IDAgb2JqCihgYFN0cmV0Y2ggR29hbHMiKQplbmRvYmoKMTcgMCBv YmoKPDwgL1MgL0dvVG8gL0QgKHN1YnNlY3Rpb24uMi4zKSA+PgplbmRvYmoKMjAgMCBvYmoKKERl bGl2ZXJhYmxlcykKZW5kb2JqCjIxIDAgb2JqCjw8IC9TIC9Hb1RvIC9EIChzdWJzZWN0aW9uLjIu NCkgPj4KZW5kb2JqCjI0IDAgb2JqCihUZXN0IFBsYW5zKQplbmRvYmoKMjUgMCBvYmoKPDwgL1Mg L0dvVG8gL0QgKHN1YnNlY3Rpb24uMi41KSA+PgplbmRvYmoKMjggMCBvYmoKKEJlbmVmaXRzIGZv ciB0aGUgRnJlZUJTRCBQcm9qZWN0KQplbmRvYmoKMjkgMCBvYmoKPDwgL1MgL0dvVG8gL0QgKHNl Y3Rpb24uMykgPj4KZW5kb2JqCjMyIDAgb2JqCihJbXBsZW1lbnRhdGlvbiBEZXRhaWxzKQplbmRv YmoKMzMgMCBvYmoKPDwgL1MgL0dvVG8gL0QgKHNlY3Rpb24uNCkgPj4KZW5kb2JqCjM2IDAgb2Jq CihUaW1lIFNjaGVkdWxlKQplbmRvYmoKMzcgMCBvYmoKPDwgL1MgL0dvVG8gL0QgKHNlY3Rpb24u NSkgPj4KZW5kb2JqCjQwIDAgb2JqCihHZW5lcmFsIEluZm9ybWF0aW9uKQplbmRvYmoKNDEgMCBv YmoKPDwgL1MgL0dvVG8gL0QgWzQyIDAgUiAvRml0XSA+PgplbmRvYmoKNDQgMCBvYmogPDwKL0xl bmd0aCA0NDMgICAgICAgCi9GaWx0ZXIgL0ZsYXRlRGVjb2RlCj4+CnN0cmVhbQp42n1STW/TQBC9 +1fscVfCw854v8yNAi1CIKXEFQfEwU03iaGxxcZB4t8zzhoobcLJs+OZ9968GS02QourQp/5XjTF 80v0gjQ4R1Y0aw411J6EJwfeVKK5E5/lIn7r+o0qKx/kTd/9iGnfjT/Vl+bdsR09EFoztZeVAeKw tDUEa3P71TBs7mNuXx52u5im2MthnXOvhrv5L2m0f2ENVMbRBGsCOF0xOoLVIaM227npMsV4sXyd HwuFMikiOXyNq3HCKt40xfcCeWAtUIQaqCbBaBB4zNWumPIGHbgaBYJh4hTFurjO7hAKMuCDx0kH egfGWGG9BaxnIe+72z6O+7EdJwl2eqc2dXF/lt0aB5b9/h/7yV1R9dAVJFaByO4QVM5lNS8VWXkY t0N6kZ0ketijRUlsIta5+m07nPKbWKfj2mPN8tAfJ3ms6LEccg40b6liNXbG/xD78YyS0hIvtf5X z8fhVpXkZUzjyTvwYPQfYZ8Uopetck6O+6E/SYKBHQ+ZhOZzbDPHkE5SVEwRflMsVAiyvVclyu4M fj70EmnKmnnsNq0UU2z5IshJ8s/mYL7vJ17ylfwCy0fXugplbmRzdHJlYW0KZW5kb2JqCjQyIDAg b2JqIDw8Ci9UeXBlIC9QYWdlCi9Db250ZW50cyA0NCAwIFIKL1Jlc291cmNlcyA0MyAwIFIKL01l ZGlhQm94IFswIDAgNTk1LjI3NiA4NDEuODldCi9QYXJlbnQgNTEgMCBSCj4+IGVuZG9iago0NSAw IG9iaiA8PAovRCBbNDIgMCBSIC9YWVogODguMjkyIDc3OC43MjQgbnVsbF0KPj4gZW5kb2JqCjQ2 IDAgb2JqIDw8Ci9EIFs0MiAwIFIgL1hZWiA4OS4yOTIgNzQwLjg2MiBudWxsXQo+PiBlbmRvYmoK NDMgMCBvYmogPDwKL0ZvbnQgPDwgL0YxNyA0NyAwIFIgL0YyMSA0OCAwIFIgL0YyMyA0OSAwIFIg L0YyMiA1MCAwIFIgPj4KL1Byb2NTZXQgWyAvUERGIC9UZXh0IF0KPj4gZW5kb2JqCjU0IDAgb2Jq IDw8Ci9MZW5ndGggMTU5MSAgICAgIAovRmlsdGVyIC9GbGF0ZURlY29kZQo+PgpzdHJlYW0KeNqN V0uT2zYMvudXeHqSZ9aKnpbUWzLTZLanznQ7PSQ9cCXay64eLiltsv+++ADKlh2l6cU0QQAkgA8P RZvjJtp8fBP59f3Dm7cfkngTF2ES59nm4bApqzCpkk2RlGEVFZuHZvMpiLe7OE7y4L7fpnEw2mG7 o7WZ6tEM/favh19JSbKJ47DK8wRKos0uqcK4yET+4cm47S5Ji+Bkt3FA4nke/K3rEcR9MB+qR1qL YJhG2ddWE7caTX8UwherTifm0VYorXmERmWN9krGQVY3negA3GUwWH9Tr8ctSX/Bz2CfhdgNvRkH 62/ZB6pvYBOMIL+kaS5GdKpXR93pHsKkL63KgJ7TmlrBDU4ouJ5PwPXC95hGSI2xbHJaVcHktBCH g6zPL93nKI9S+omFBc/gI/fq6rG9Oi0DI8/Q9qBq7cK1B4vX0yLGMylmtNYQehKiOGFqcUsRBapt B/ENn45PxK/l//OWPd7rVlhNd2rPjmDjvcwg53KL6o/aq2ZLiGHh6ZX33jgzy0Vhtg8EF1o2+uuo +0Y3wvLFjE8MGOazWj1zHHF0q2/vH0JHVv8zGes5997Yzm+Gtcedn5DtIV7Dnd3JtLoJt7syj4IP 2zIhTIEhI4VwPVjh+F2eMdrviESBZcdrYaRjAfTgPAU2z4IE3RfT+EvdNEcPbEoW4F/ZV+E4XK7X azZco3+LwO6ynACGILrR1E72h6mvvdewBfJMqGFnGuFG0gORMyIFlWBVXkTJwsbjsoPs57Ue+lqf xB+AzRF5To+2a69GhCkyCSWqplizwgYBpZUtxtHjZNpG4kk7ZzrTKis84iKDCN9KDXyniFCJCyZi eqR0G3UnKXVbHOlpWRpmqa+Lia+Lv8HOfepLWlrkwcdBte6/KmM65yiim6dXOBHKkVXQ3wT+klyj rOezcRY7AIdz5rL9oCrnq2Q9up/pZWUZHHWvrWq9uLJH+GN0CEFJ2BqtHs/wohv58ru1cDS6NS9S ENRjq6EB/hu1G+XfqVW9d7TkW3JO4F5/juJsdEL0YSjEGlDqoeugeqIygfUVObUIRBZSgs5uTOlR +9IHIpxb1MfZTkThYVtRptgjGQol57Z3GxQojkqKTB5G+d73vPBKZPOJsBoF7xryGxmVUJYtyyL2 1HF8LnO78bmMVX+9OWjUqPAvRR1nipKFHOrk37v39zune2dG724hf0FRUOIcJkxOGhcecNMoED0y rcqLm0KLsCQ3TYfSO6MQMCBxBgTIP27PvrUlaEyMtxKdAtuRPCJCFVxSnl1Ce+4UqJTXzWKprYaI bxagI913a8CTsaCsFlUeu+sqzxQxL1pWedAJZp3/N3hNc1EXboIfKjoifAeaB/BZoooWtq9NCItB ggF+qaOcIoPA/dLNkqtuxqU5Ds4NDayXhka8S1Nv0Lmb40wQzqtszoo1CP+JpLJm5AckAF6apFIl 6Xb0EuzlsQnVJe0uIw8RfLICuQfMcrabJwCiHRUqqliTkKEC04VYEqACWKkVUoi0V2x6/6D+u8Bl ZzlDZYdY8710MoV4vAqBX53femqXJ9k8DdHgKet+rZN53vtROGbORlskIUG7EcLBDp2/cb3VumGy NY+kBHG0Paz+NvIaLvRtEwfS4jiZ5kzqjPOJjbR66bxQehHyMK++nQ858/x8iJDyiOiFGmBtrcsi bgKzXusGY82PAZauAuwPKXZU46AQte16ECMKHqvDI9ydlcEiCt4pLMXNg8aaR7KB23PPMxdP9M2Z +SLAweeK6O8VzC2nJGxMv/sWYN4g6vzKtywezMkMykApUT4NCz/Ht6/C5x09yO5qVOaRpRvs61tq 4G7qLvMJR8ytdjaMB8k+v+1v86jxOU6z36Vb0yfY02La+Ol748aiqd2zUTxrARhadgzzJEJ6WeUZ TKeFqA6jzEgxp25v3OyJaE7qmKcLKOTGC3FpuneS0W7o/McElFwM8N8ipMgbwAzK6vW5w5ljr310 1HmUmwvybUU893SaDcr/1dPTuMBHHrRNbEhMwwx98rEZcRVgYuaOwo+gw4bbSFBP3N5K396cHAp8 wUUQ8DTGVFxcPq7i4lKFMAL/uHfPUJ4Ho4XBxT7EmAoAlWU5f7kveX55ePMv4RxC7gplbmRzdHJl YW0KZW5kb2JqCjUzIDAgb2JqIDw8Ci9UeXBlIC9QYWdlCi9Db250ZW50cyA1NCAwIFIKL1Jlc291 cmNlcyA1MiAwIFIKL01lZGlhQm94IFswIDAgNTk1LjI3NiA4NDEuODldCi9QYXJlbnQgNTEgMCBS Cj4+IGVuZG9iagoyIDAgb2JqIDw8Ci9EIFs1MyAwIFIgL1hZWiA4OS4yOTIgNzQwLjg2MiBudWxs XQo+PiBlbmRvYmoKNiAwIG9iaiA8PAovRCBbNTMgMCBSIC9YWVogODkuMjkyIDU3Ni4yNjMgbnVs bF0KPj4gZW5kb2JqCjEwIDAgb2JqIDw8Ci9EIFs1MyAwIFIgL1hZWiA4OS4yOTIgNDg3LjYwNCBu dWxsXQo+PiBlbmRvYmoKNTUgMCBvYmogPDwKL0QgWzUzIDAgUiAvWFlaIDg5LjI5MiA0NjIuMDI5 IG51bGxdCj4+IGVuZG9iago1NiAwIG9iaiA8PAovRCBbNTMgMCBSIC9YWVogODkuMjkyIDM4NC44 OTUgbnVsbF0KPj4gZW5kb2JqCjU3IDAgb2JqIDw8Ci9EIFs1MyAwIFIgL1hZWiA4OS4yOTIgMzA5 LjYyMSBudWxsXQo+PiBlbmRvYmoKMTQgMCBvYmogPDwKL0QgWzUzIDAgUiAvWFlaIDg5LjI5MiAy NTEuOTg0IG51bGxdCj4+IGVuZG9iago1OCAwIG9iaiA8PAovRCBbNTMgMCBSIC9YWVogODkuMjky IDE4OC4wMTYgbnVsbF0KPj4gZW5kb2JqCjUyIDAgb2JqIDw8Ci9Gb250IDw8IC9GMjEgNDggMCBS IC9GMjIgNTAgMCBSID4+Ci9Qcm9jU2V0IFsgL1BERiAvVGV4dCBdCj4+IGVuZG9iago2MSAwIG9i aiA8PAovTGVuZ3RoIDE0MTcgICAgICAKL0ZpbHRlciAvRmxhdGVEZWNvZGUKPj4Kc3RyZWFtCnja lVdLk9s2DL7nV+gozUSqnpbc227TdNKZTjONe0p6oGXaVqKHS1K73fz6AgQoy1pt0l7WFAiQwIcP ADf2Tl7s/fIqXvze71798DZNvSSJtkWRerujl8RZtPHKtIq2centDt5HP42Cv3a/TkbexzDfxv5v Mkh8dZJBmKWlb852sfHbZt9Lo40wn+IizuBPMm0ooZ5IvemDtPLNsDDeC80r/aSN7OzF4GHiJXmU 5RvrYYjruPLCrIrSvHI+ZkGYJGnhv5Ft8xBkiS+V2LdS3/j+LFo+Ki2iuNjQUclquH9AtPIShOD2 oBszUCQb/7ExZ1pxFKWFALWfo1BOKFSpv7gmBi/KKN1W3wJ9h4ZSG7qxBrg0QziQqBNfAvh0KI5K rjj3vfxAVHjIoL7o2ads22+7nK26/CeCVvoHYeQhCPNN4s/uT+h+lIqet/e679jmRiX1xeXSNoBs LUwz9CQbddOfyNDGiIvVGGkjRBcx7duiBCaVUZbl5P1LeQmd8jzUfDXU9wqcg0QUhf9Z1sZlpEf0 RtHS90W4mnkEbWYPxCqNaFqECJUOUtequWCYnAILDyoSNFKdxYW39NBJZ9YN+v8UTc5Fswu2OfOq LPz3reg1n3JTLvFtpews4EnmG2uKq8/DXtNqOPIeKeXgOPSLOTq4Ww8WHYi9Z/XAIoNlRoKLUEb/ GISbeGvvoXRfD85eSDdt2KRiymPOdkGuM5yb65E3jagbiIDNpzjJXVKe8xZtppPWeWtzh7wl0kbE vzIqiu2tR7/3CL6r1BLKr5ckOcMVr2FZbbkQSXyBLJGmLX6QPKrG8ObYN4Y3IUBNwuOgSCYfEGBp qx0+XbRjy9aYO2t7ZgFBTMUTrnn/rKHE81aH3qepgyp2OMdz9OPrdu3QdzuPTdvSak/sd3r9YayN ZDPddLbSWiN6OYy65XIO8yyzXF3hgfzH4pRs/VrZcpSqESSwIMCvAbn1F9br3TOpZt0TtBpNQiRP 3+gzxQEbwoYBO8YdeJTCQJNmA0IAxBehNVtV1mrF9YmshgdCSQPhOxRLE2cJvQpWipaOZJCmiWQg Fq0eaDWVxPFpccpaXYAlwEYJvbb0alEapLWsjlWC7c6IapoDNoo8qaVmCU2rsT3Q58QR/DjL9nIc W/qwBYCLiXf248y6atiP2vTTuciAG4XV9PMGdjdL94QmyDJdd4Rkll/xBVr20rIXpIZ31diz4IyP CF5TimHlZj4sLUEXiiu5ACnxChYv9ChwxF0/e0DAl31ArHNv8VK4fShwawS/joYYVtou+MQlvbmy t5eP60+j5QiLl8Or4OF1L3uJpUZtrnBtrnDtq/Df4nhTUt5/eMMTDjm0yWgQ/cdBZ3s8UWt2H3zS ffOY3iILpvtw6NvBl0/PAizV/uZIkuqx64TCN85XyfNe8JQ/Dm072OGoF4+P6RE7e6C8/IidcTwH L63zuLi7liEJ7pvedrS8iv13NKWlOopa0jby5+7+HU/bCh8+UN8PlgB8pGk6Ce232lb+HUnmE3nt EdbM73mNNhuoKupSeYEZRdhyQFKQgP5PIFENjdLalCiuTYsuVuwg7D8EaAf9Hf4t4LMHPpc4r3rZ kmoHD6n1h0Nzi0RWZtTuy9zRLfMP3KDGTrL2gTUX4WVVwbU5Owbefs2pdyY02/MZTUBIjYo2xAne Ttqs+VoPXUfZLPwabwb0TlKTAA/G35vQ4Rse6oLA0UbBdB2xseFUCbdJ6n+wb03cdXqF01Pu5JNi nq75dFTy75FhabGH5EzxvPJ3P70ngQ2JZNQeQOYmJcJWZP5+NKRAqOcIt1J8MO1cSZAFlq8JyZcp BNtm1dd+4Df8hDfWI+PNFS945YI252V1lpsoKXIPoImSlBtKeqPz8+7Vv7wlG5gKZW5kc3RyZWFt CmVuZG9iago2MCAwIG9iaiA8PAovVHlwZSAvUGFnZQovQ29udGVudHMgNjEgMCBSCi9SZXNvdXJj ZXMgNTkgMCBSCi9NZWRpYUJveCBbMCAwIDU5NS4yNzYgODQxLjg5XQovUGFyZW50IDUxIDAgUgo+ PiBlbmRvYmoKNjIgMCBvYmogPDwKL0QgWzYwIDAgUiAvWFlaIDg4LjI5MiA3NzguNzI0IG51bGxd Cj4+IGVuZG9iago2MyAwIG9iaiA8PAovRCBbNjAgMCBSIC9YWVogODkuMjkyIDc0MC44NjIgbnVs bF0KPj4gZW5kb2JqCjE4IDAgb2JqIDw8Ci9EIFs2MCAwIFIgL1hZWiA4OS4yOTIgNzA5LjE4OSBu dWxsXQo+PiBlbmRvYmoKNjQgMCBvYmogPDwKL0QgWzYwIDAgUiAvWFlaIDg5LjI5MiA2ODcuOTE5 IG51bGxdCj4+IGVuZG9iago2NSAwIG9iaiA8PAovRCBbNjAgMCBSIC9YWVogODkuMjkyIDY1Ny42 MzMgbnVsbF0KPj4gZW5kb2JqCjY2IDAgb2JqIDw8Ci9EIFs2MCAwIFIgL1hZWiA4OS4yOTIgNjMw LjMzNSBudWxsXQo+PiBlbmRvYmoKNjcgMCBvYmogPDwKL0QgWzYwIDAgUiAvWFlaIDg5LjI5MiA1 ODYuMzY3IG51bGxdCj4+IGVuZG9iagoyMiAwIG9iaiA8PAovRCBbNjAgMCBSIC9YWVogODkuMjky IDU0Ny4zMjEgbnVsbF0KPj4gZW5kb2JqCjI2IDAgb2JqIDw8Ci9EIFs2MCAwIFIgL1hZWiA4OS4y OTIgMzI4LjAwMyBudWxsXQo+PiBlbmRvYmoKNjggMCBvYmogPDwKL0QgWzYwIDAgUiAvWFlaIDg5 LjI5MiAyNzYuNDQ2IG51bGxdCj4+IGVuZG9iago1OSAwIG9iaiA8PAovRm9udCA8PCAvRjIyIDUw IDAgUiAvRjIxIDQ4IDAgUiA+PgovUHJvY1NldCBbIC9QREYgL1RleHQgXQo+PiBlbmRvYmoKNzEg MCBvYmogPDwKL0xlbmd0aCAyMDY1ICAgICAgCi9GaWx0ZXIgL0ZsYXRlRGVjb2RlCj4+CnN0cmVh bQp42qUY2XLcNvJdX8HKS6iUhiEAgkftkx3LiZOVa9cerx/slAsiIQ1LPCYAqVnt1283GuQc5jjR 7sMMu3E0Gn034uA+iIOfL+KT78v1xY+vOQ8YiwopebC+C1gsojTIeB4VcRasq+BTyKPL39e/zpuC T6ukiMN3lyzsb0c7dNray5VIs/CuNwikYVV/jpnQRneXPA8Hmt0h3JuKMFv/R9vociVlEgq+uq0H 2tr2XT30pu7uCR/6yxXuayzhdtOPTUXwLUxloSZE3TYa+QxYApfJghXLIiEkXQGpCJ4iDxnw8EBY 3+E3CxWhaeLZAPgBF2rT6ebk8quJPE8jkXgJiUUJvWm3ICPTI6lHRw/4TnIeln1nazvornyigd0G JOWgFoU0NkO9hdu4EduPptSWEGX8qCKiUoaqbvDqIMoi5uHrS5hENeAi/W/VAp2r82LZaEdQFOFo HXcAoazwO0sXEUWfDo8dHZN+2tBEf+c3jfeWoLqb7903j06fjoIe7KCGz7GMGfwxHM1DM3bdvATV gt/SKLtxvMfHbFdju7VXoCiRoOjwCkLw0OkOvru6aQjqu3KaQ1UL4VSNSzZ142fAoJDHHWF+HiSb hKoZ/Zo707cEgcQIKEfjzDsL/anTFQ75TYjfvTGhcTFgBpSv0Ig5n+RYKtAATSua6PSOcJQXmEtd WpogN8OJJ1sODYoyJ1Hi7EZZmr11jqN156lWla4I3NXDxp/RL8mXBO/8AEVNTuLZS0mU8G3rrnJ2 DyGEBbCfM5lgCFnxIuKpDECJUc5T7yKXK8a4RKdodAtMCRbivZwPZjJ8pQewZOvpHYWk2FFkmZfm 2qkgLQ6u728PY3djV3qigBk9mFo/orlqO28adEtw3YEoW7Vfr7rKA01DVuF3bU3v1ASOaCdKXoq4 ertaEiNs2ppaDehCSQJo/QhWd+98GXDnZ/C1ekBAzqzh4AFr4NlZntO1cRlNQWTxjOOYwiuS0VI4 ICrObhc4O5UcL/IpKFnC0J/xS148ANfGeVwM/A4YnQkTJDJeQNhQt8BT44J6Aubx5lC4SKp1MeuJ kDm4IDKriUIkDiHZJca1j8/eI5G3jbfLsm9bUiAZqePb3KnSz4+WkorbA6xqghecCNYOdQPhxPEL gTSiCJpFUhbHDK2n021936lhNJ4qys8Nn9oo8r63UeTU+iRk/Ww/2x5llev1xR8XDCQRByzIwbUK HiQpuBZkn7K9+PR7HFQwB7KKRJEHO7eyDRKWRmmB+5rg/cU/L+JIBtPP3AeH6LupEpCBjIosS9Hr MhmlWRYkCZyY5XRbdpTmiCOk5QCg8uNrkUIFAaUDHIzFBNQSDM6g3XU3HO//Jkcn7Kw8rRVci4MW qDI5ovft0/ea8JtOmcfABz4w/MkdIblLLr+6jB/+oVMtJFwHj19wkQNxtNHd1dmj3arHvq4WifZN tfVEsW764qniOFDderL/tzjFM9UrkogLMQn4GcI7e1HIeUsXhWG4J3rQ3867RJyDvSfPcom/Ug6n US5ZIPI0ihNx6PQZJvLtiAEp4y5JT6OWvpW2pal9pKto2RSEMkrz8PnMRHKjOnXvsqKvJ2D8KH7i wEtl9XdEBa345s1LF1IwMq2WQpOl+IW5gLPMmSCEZgGhek01E2c5FZQ4nRM+9NtV4zNmQ0O4EaM9 F5hQ7dbXK0NNAbt5opwtskO58RQ4ifPJNn5a//3Lq+uXH35eyu9SRiKdl14tkZOgBHFI7F+v3y+R goojTsXzSP3ycYmSSKOC8W9SWiVQU4vTmhpJ/nb97u0ie2CqLP0mUTg2P2Lv5sVPv7y6/scSOZBx mrHnkXt7vf4fBfcVqQ/vr98t6pNHBeQmv9Jn8q8MhEeyYEcqvTmrh3kdGjDj4dve+RrDelwNE6SX zsnjiOd/QQDgVxARYemRMmuflI1uwL0rXz34ZhJaGeocMN9jp/cAvn1Fa6DBKNGJNoR2Wld2KkOO qWynsggRNQw+CMz1wdD/SfWRwPKHxxZjggAufFMFg019a5R5IgQrWKwR68rVn9T14mfsaldPuqLt pGxKXD1i/PoSq1/q0vBIFyOoqUH8sTbDqJrz8ajVbY/ciBTasRbinSshsxgOLZux8h0/B7YpsBBG RbErudlUn/O5QwHQNYP4kJDw8IVjkdY6+Z4lt1BbOl1Di/hYK2ot9xWD8MUbzFLzBYDtW03rpnrO Vb4yBa2o4XtPbLeZtIvIxAjAaBG03TEKI6XRlEKwTb2FKlsRVX8InLvANFW/mZgtwHOa8Vn9OUO3 kVkSfpiSDz+/T+z38RAVlCd7zcFsqToC9o8vQM5QO5v5fgCBnanRmK+WuD5qiCVWyS2OEMneDWbY sJqp3YI1dLDMDg6G0Yb6sv5hah5wxbilSf05ZmlZe49q5iuliWcT1h73gBLfkYiiS+zS9b3mTHd3 3BIirMAB/aPSnk1wonvlnlqq86784m7AtxRR0HMUtkjQEwG5O8K8Z/pDq3HqIAofSYqpEwLAjrcG +K87xxdsdk9vmmBveAf7ZsPDsw8ML1rwZd+Df0RRui0yrOrHmbyc+HSkEyiBqKt3CMkcADtCp2ag vqMtnm+wM+XeFnDpsf3DyOw7AE9XkPM5+ysk2EgtqOurtmouPTGJQ6WCD3pJzn2js/igd4P68K4A 0UZN0SZhJ3JbJRlWWS6iJPjeZzA9UO0EvXVX+V2Offw6CQBw4o0wcuzFSPcjwvr7pjm5RsEiBpkS 30ryPF+q6KFs/i/Wh6STCmVuZHN0cmVhbQplbmRvYmoKNzAgMCBvYmogPDwKL1R5cGUgL1BhZ2UK L0NvbnRlbnRzIDcxIDAgUgovUmVzb3VyY2VzIDY5IDAgUgovTWVkaWFCb3ggWzAgMCA1OTUuMjc2 IDg0MS44OV0KL1BhcmVudCA1MSAwIFIKPj4gZW5kb2JqCjcyIDAgb2JqIDw8Ci9EIFs3MCAwIFIg L1hZWiA4OC4yOTIgNzc4LjcyNCBudWxsXQo+PiBlbmRvYmoKNzMgMCBvYmogPDwKL0QgWzcwIDAg UiAvWFlaIDg5LjI5MiA3NDAuODYyIG51bGxdCj4+IGVuZG9iago3NCAwIG9iaiA8PAovRCBbNzAg MCBSIC9YWVogODkuMjkyIDcwNy4wNjYgbnVsbF0KPj4gZW5kb2JqCjMwIDAgb2JqIDw8Ci9EIFs3 MCAwIFIgL1hZWiA4OS4yOTIgNTk2LjAzNyBudWxsXQo+PiBlbmRvYmoKNzUgMCBvYmogPDwKL0Qg WzcwIDAgUiAvWFlaIDg5LjI5MiA0NjIuMDU0IG51bGxdCj4+IGVuZG9iago3NiAwIG9iaiA8PAov RCBbNzAgMCBSIC9YWVogODkuMjkyIDQ2NS41MzcgbnVsbF0KPj4gZW5kb2JqCjc5IDAgb2JqIDw8 Ci9EIFs3MCAwIFIgL1hZWiA4OS4yOTIgNDQ5LjI3OCBudWxsXQo+PiBlbmRvYmoKODAgMCBvYmog PDwKL0QgWzcwIDAgUiAvWFlaIDg5LjI5MiA0MzMuMDE5IG51bGxdCj4+IGVuZG9iago4MiAwIG9i aiA8PAovRCBbNzAgMCBSIC9YWVogODkuMjkyIDE3My4zODIgbnVsbF0KPj4gZW5kb2JqCjY5IDAg b2JqIDw8Ci9Gb250IDw8IC9GMjIgNTAgMCBSIC9GMjEgNDggMCBSIC9GMjUgNzcgMCBSIC9GMzYg NzggMCBSIC9GMzcgODEgMCBSID4+Ci9Qcm9jU2V0IFsgL1BERiAvVGV4dCBdCj4+IGVuZG9iago4 NSAwIG9iaiA8PAovTGVuZ3RoIDIwNTMgICAgICAKL0ZpbHRlciAvRmxhdGVEZWNvZGUKPj4Kc3Ry ZWFtCnjapVhLk9s2Er77V6j2EqgqYvgA+Dg6tmd3UpWsy1Yqh3hrC0NBEnf4UPiYyfz77Rc4osz4 sHsRgQbQALr7+7qhcHPahJu/vwnl++P+zQ93cbyJoqAwJt7sj9DMA2OyTRbnQRFmm/1h87sa+5ft Ls4TNXb8nQYngjM1YlVXD41rhtGOX0ITJvAT8QwY6K1fbwf+Nrbdxrm6kT5v40yJ4tK2wXan40jd H3kH2KqXQesbsGUHuv+1/wnus4uyIEkMH9mWpRtAb5Jlqnd/TFXvGtfiBqNIx7MdsZWrchsp27ad dB9AN50DZzVuhHOYPFe/bfNIue/qmgfK3tlRJrXumRv2YRh7W45V1w6s7Nj1a8eDy6AJkzhVwwXv ohyeovoSRrpEOWxgBzcEuHj2F+jQ4Klss4tNoBPxTnwz6fedLkL1CdR101i1J1Af5YujwY2SolD7 c4WWj2Cvru8dHSRTXXsYeAl5O8rIybDDD3ew41WoxDoP8lDDpnSMj3f//vTPX/cfeOoyqkwU5JHx M2v3RJ6uWX13lO0olkAwvAzlWM9xRNeDK+cmSNN4acc9rUkS1c93hU7T0U0OU02jWg3nbqoPPDi7 Fzv2oZYWXRam9m6c+lZkXrtETtWv7Ta+KgFnkxLL3WF62K15f3AQajF4gK4O32qkYE/V2+0uUjW4 bsBG50f5O98C2rauOzzTM3f5pNAAZPYiktV/TA7hl4Qh3JY9PI03q6oW4rSxGByi/nXmyvkb3Hmq xwpidOQoPoFJLtwEUOK42LkfztVl4JBG7TjDEoigUbFdXX+0AFeWoQ1x8kwR2JHYdOUrQm5XfxMq OR88WYXKL+AO2Irop+sfORovPfABuwAYoqsFKd+AERPJjKR8RlImSMr/EklFEWTGXCHp/pcP+zUg acTADDnbHlbVRYFOoxtt6Zq6XQLwKuJk6d4Zn+iRXFOU4pfCBRtfAXSXhrm6b/20ShY2bL0DmAWg +D2by7M8TmidO8iiTlRPF6GhfmSJ5c+DHUtceV6eqXXjKsRenZloQ85kTkBP4kEMcHlwQtQVibr/ uIUEg2Kdqv07OO5HJPvvefGv76979+9+XnQ/PqW8zo2lqPsNtTmWNhYP8sKTbT10LLaHA4skIr5K X8tEgrA4ChLkHqj2EcPWjYKuStAl9kckiv0nuFHtAomAaAM7xZHRFAFxEcSp2ex0EiQm5d01kE8U G7WvGspwRn0ut0mkzo4odSWQQlIUZQkr8BbQkBMP1VN1kDZFEDaGqWmQqLANzoEMa9R/XDleSfCS lMJRUhEZoA+xl4sS5EgEqT0h/Hc5ZOm932GsYKDx29WdqEbI8LgdHtcsf3BD2VcXpsJrwztb4hnO 4oYRNhU2630mPzMle5YTvxy7mauF129YCJgqCXOwYBRkkST1aJWpEHJi/mRRtRWBmQH/9tJjIJpQ RRhjhdphvIfqZ/vC/disuTBPgyKflXABh9cgX9q+4d50Qd2JERd/55n8YaowNXH8uUXDqC9RogWQ 9eFvPMgU0w9iZyN2TtUdoqt37kfG9a6IApPopZM+vycy0ezOpKBU3bKoasE3mEJr7vMpCmKt0TUy vTs5LCh55BljZTzLNDdybscOIudyo+eA9Ai4q7uLLynXAomHnqq+a2kahC9WkjpW/+gkmx2nukaf pBDCx5FOk8Yev9CC81649Uw7srWhe8ICAhtH21R1ZXtW8lzRJUiH48Ydugmt+fn9erQL1ctd4JAc tEsIUjiTqV8TO0uZzX0UXHp3AThIIBx9Kp8jonV/Sr1A+Pk/yttvISEsfBRTyMdZouKUvzv8aPXT 1DoWJKtQAC6Ml0iIdaI+Xb0jyFAoPFmMIwgYxEWYSUWq8Y2DhIwtohpqUb2H3wmLNAiFJffhENUS MAJcR+GfZ0Ge3VS9/u0Ra4jd3krGpMIPUgq/uCrHbIMSyn36NrniSNO1FZU5PZezmKTQzzgbHmng oznESQZ71VD1cV5ae9fgTgaTNsY+Qb46sOgApqNLwnX5/Wg4jaPg8am5fjcaQTY2iDFfPHViiXVV evDq20Jwp6OIyhFcz3DCaQgn4I04LQRPLL7U1k/t1u4ExqZk7SlA3x4XhxgdMHRTG/Fo1Vzq2ZLW Z/XrZVjW1kwBIKYK3ZcHrGFZKa+VDJzZ4Mk5tb5w0Blc9Vz5xIVdylY4rXc1BJGs4cxaYDnFmQrL 1/6RsmoRU1BHqhWFBHniIlxa1nYYquML9yyLi78qbL5+i8eAZkIKfLv+wM+XUJ4vIVMdNix/lm87 ENTVIDOIb2jteR57gBvd/ivBA/SvBJAjvby0eksEvPpUJzo2wlmYeFL9WsWadH6XYfv8Gvkyavnz tMWgPk0ihZJIBq4fZYmRPxY4ARmfQ2naoiI06Tf+Wriq/thS60SbL4k2+d9LDuZTDXGq8UM8iz0v T1QUrhFtkQRpoZdMqyH03lkkCGwN43Twbso0oxrl7BWewV7RYN97FlH4Ws9b2leO8K2d7UXDtd11 jAELIe6k5siDWKc3r6K1f7iSsBAMR6Fyf0IosuciH3uet6hMHXlknQ3Cgiv4aIYBNuldBEOc8XFx O1yq1SdPP6sq/KGg0eOtpoH/o0BBKWEh3aGtfP4YEe6pSQQLNCxVT05/R8kKdgk9gXQEId/JP1i+ nIHZ91yqwGxfLawcuERv+GLh7P/OoMqx48KhsY9C1Isy4goLt6V0lgb4ssHXCDwH/HPmes6H/Zv/ AkH7chcKZW5kc3RyZWFtCmVuZG9iago4NCAwIG9iaiA8PAovVHlwZSAvUGFnZQovQ29udGVudHMg ODUgMCBSCi9SZXNvdXJjZXMgODMgMCBSCi9NZWRpYUJveCBbMCAwIDU5NS4yNzYgODQxLjg5XQov UGFyZW50IDUxIDAgUgo+PiBlbmRvYmoKODYgMCBvYmogPDwKL0QgWzg0IDAgUiAvWFlaIDg4LjI5 MiA3NzguNzI0IG51bGxdCj4+IGVuZG9iago4NyAwIG9iaiA8PAovRCBbODQgMCBSIC9YWVogODku MjkyIDY4Ny44NjEgbnVsbF0KPj4gZW5kb2JqCjg4IDAgb2JqIDw8Ci9EIFs4NCAwIFIgL1hZWiA4 OS4yOTIgNjEwLjQxOSBudWxsXQo+PiBlbmRvYmoKMzQgMCBvYmogPDwKL0QgWzg0IDAgUiAvWFla IDg5LjI5MiA1MjIuMjk4IG51bGxdCj4+IGVuZG9iago4OSAwIG9iaiA8PAovRCBbODQgMCBSIC9Y WVogODkuMjkyIDQ0Ni44MzIgbnVsbF0KPj4gZW5kb2JqCjkwIDAgb2JqIDw8Ci9EIFs4NCAwIFIg L1hZWiA4OS4yOTIgMzY5LjM5IG51bGxdCj4+IGVuZG9iago5MSAwIG9iaiA8PAovRCBbODQgMCBS IC9YWVogODkuMjkyIDIyNC45MzMgbnVsbF0KPj4gZW5kb2JqCjgzIDAgb2JqIDw8Ci9Gb250IDw8 IC9GMjIgNTAgMCBSIC9GMzcgODEgMCBSIC9GMjEgNDggMCBSIC9GMjMgNDkgMCBSID4+Ci9Qcm9j U2V0IFsgL1BERiAvVGV4dCBdCj4+IGVuZG9iago5NyAwIG9iaiA8PAovTGVuZ3RoIDE2NjYgICAg ICAKL0ZpbHRlciAvRmxhdGVEZWNvZGUKPj4Kc3RyZWFtCnjarVdLk9s2DL7nV7in0NOVKlLvWzeb R3enSXZmN+0h6UGWaJtZWfRQ0rr+9wUISrY8rtM2PZEEQAjE4wMUzFazYPbuRXCyvnp88dNbIWac +3kci9njcsaD0E9mqcj8PEhnj9XsM4v8+R+Pd+Ol2WcvygP2JYgDpIOCcKIg92MOgvbqXd/IuRem gnGOa8g8Ot71c4+zgSlip2liSp74Io0GVfA9VBFzdmNk0UnaF4u2M8Wcs7JTuvHnXhzk7HrZSUP8 h05ucRcw4S40FW3CK1xT9vs8E0y+rGsit11hOtp2Gq2aeTwI/EyksEn9MIzJnJ1R1oYsZaWeeyJl lTsujd7gLmHt0xzositxWROXvg+bbi1JSjXIBouXRSlbYrZr3ddOcGG1O+W7uciYtMZm+MkvAY8a 6SQL926rXbXW+mBqdgv+OAmnxyNwOTxPpL7IM5KLvzfmAl0e0+rRctfX+5FxJt5Z6nORT+MteMZu tEGFPGVqs63lRpLDChfxMM/Yq6JFJ6C0bmi1/sVLT88btDs8KMR8aZx4u2/Lrh4FrijkOfeTTExd d5wo8P1yyEKRuFjCZtkPbGcqfulgLbGWGqiG9uPNWi1MYfb4jYvxcTWZfE98MAxRnDCR4BpjfPA4 MGIW8rP1CMiQhtP4RBFn19ttrUr7wJYoPdQc5GlVdPBSiTTBtkZuCzMELU1z9kGjZ3Z0RfeG5Kwj 0EF7Ors89vLMFyKZhgRCUO1/gKKHFLhF72YBZT7suM11ou1sHb6s6FQrqkuSgiq3VNTlKFSZwZl0 I7pekmAjO4CLDr3OHTwBFyv8TOUt2mazJaA4uiAg3hYSBOgjtnaAQEzdd8TcrdWAI0jfWphCRklQ gcQRKvAA/q4BUZzunerWg0ZDpEbuiPJPc89hQ/rfc+96LmLWrxCnIc2sw2z6IQ4CLwEeHXjwNwiR pelJR0hil4GQNEMSIrF33oYqtd8L2KcDhUQo0kl0LpLAHnAC9xg+z2Wi8KNwGtzTwF5RURd1q12d y7Y7wQrrfuT9y9LP/g/3p1FsW3IU5bb8o2xgEk1EZxEg8wFtTxAgDNlrbeu97G2xZGOxRGFE2Y1C ix6aVYgNCk+d2mBYUujA7wvM6/2YvMjezYc9BgdhtKLDoSkLMItPwzBicpxD8OyywaywBdUXNZG2 xcrJLLUdE1yzQMokFl4MlhDsO50UT9gZ2UrzbI2sSEWrN07q8FA84UPd7e4cLji8goSADFxJ3/md z0BI8DhCv3si90USzyA0fshdHkBz5RxC9k420ti3pTGgILxpQ+4/zhLSeIhkYHVyGK+srg8FGHki P4l8FPgCdi7yvxSazARbjoQEWJeNDeKhv2yCl0R+Al6YYMubTaHqi5aEuZ/l45TQ9s260AL8/fMK r/olDF8Xv+ruT0rqfq2bb7xf+FmUDF/9MUsoZDwLAi5EnqT55a+6+5O3Xj/PE0B/VRcLVatuHgq2 v2hEGvpxNA7EUUA2wKhoWkISgiGbecCgGpJPly1zSif+eKX0yhTb9Tzk3zApSfwx4O/tdJezBpPJ 7jCzccV8gU2GOXGFNoZYnsiB7DcV8fbW3MIQA75f9baY8dR2feWghQjYgnG9t5eeVLOi86dGUVUa B9Zh6geYZJMhGJyNiIOVjUgNWHizBpxQTQFFH8Fogk8JYSrZgJlxxL5anICzLdUogJ672fY0bAO1 LZVsSmlVOcBD8i0tO2VnQrh1eJPV1dBKYxfuwBlgxdm5/cg50TAxwMb9reAW5xhcbdS1eaITfgTX t/hWmAYWFqjph0WfyHwsO+1A2KAfgpQ65pHyZhjZ8HBLy9rC97MDbKREo176sWnatdqee5X8c8hY 60CHg4CtND9K+KkDh5Bq6yQcrBuSeq9Ko1u9dG0V+5+N6k6V8qgFX1f68AMlhv8vvN/XnSrh18I6 rDCEvBYYBYBs4nJa274NOWCHPlg3lDp2vzUaJwv2FSxtiVQYSRv7Z6baNfWHiFW9sVmKe9duYuui bHQRmpxgYpR1X43Ch4ee8eDD0IWwrSYJ1gP4zdgmAB4l4nX1rFrtJNBRD/dzmP+uXQOH6Sb00V8w vLCPLiLNza8k/6YpFjU+Ag9DGtHptexkObSaU8vu637lDcGiRgt6b+Evt/lNyd0wIQ3hOI3TjXau GYZhe9vR6lqtpH0eUh/RKJfS9ZPqTuYjniY+9lEvSf2U86F5Hsu8eXzxF3dzRCoKZW5kc3RyZWFt CmVuZG9iago5NiAwIG9iaiA8PAovVHlwZSAvUGFnZQovQ29udGVudHMgOTcgMCBSCi9SZXNvdXJj ZXMgOTUgMCBSCi9NZWRpYUJveCBbMCAwIDU5NS4yNzYgODQxLjg5XQovUGFyZW50IDUxIDAgUgov QW5ub3RzIFsgOTIgMCBSIDEwNCAwIFIgOTMgMCBSIDEwNSAwIFIgOTQgMCBSIF0KPj4gZW5kb2Jq CjkyIDAgb2JqIDw8Ci9UeXBlIC9Bbm5vdAovQm9yZGVyWzAgMCAxXS9IL0kvQ1swIDEgMV0KL1Jl Y3QgWzQ1Ni44MDIgMjIwLjcxNCA1MDYuOTggMjMzLjMzM10KL1N1YnR5cGUvTGluay9BPDwvVHlw ZS9BY3Rpb24vUy9VUkkvVVJJKGh0dHBzOi8vbXNkbi5taWNyb3NvZnQuY29tL2VuLXVzL2xpYnJh cnkvd2luZG93cy9oYXJkd2FyZS9kbjQ4MTUyMi5hc3B4KT4+Cj4+IGVuZG9iagoxMDQgMCBvYmog PDwKL1R5cGUgL0Fubm90Ci9Cb3JkZXJbMCAwIDFdL0gvSS9DWzAgMSAxXQovUmVjdCBbMTE3LjU2 MSAyMDIuNzE1IDMyMy42NTYgMjE2LjY2Ml0KL1N1YnR5cGUvTGluay9BPDwvVHlwZS9BY3Rpb24v Uy9VUkkvVVJJKGh0dHBzOi8vbXNkbi5taWNyb3NvZnQuY29tL2VuLXVzL2xpYnJhcnkvd2luZG93 cy9oYXJkd2FyZS9kbjQ4MTUyMi5hc3B4KT4+Cj4+IGVuZG9iago5MyAwIG9iaiA8PAovVHlwZSAv QW5ub3QKL0JvcmRlclswIDAgMV0vSC9JL0NbMCAxIDFdCi9SZWN0IFszMzAuOTA0IDIwMi43MTUg NTA2Ljk4IDIxNi42NjJdCi9TdWJ0eXBlL0xpbmsvQTw8L1R5cGUvQWN0aW9uL1MvVVJJL1VSSSho dHRwczovL3d3dy55b3V0dWJlLmNvbS93YXRjaD92PWFjbzhKQ0R0cmJnJmxpc3Q9UExBNTU4MUU0 RTRGRjA1MDYxKT4+Cj4+IGVuZG9iagoxMDUgMCBvYmogPDwKL1R5cGUgL0Fubm90Ci9Cb3JkZXJb MCAwIDFdL0gvSS9DWzAgMSAxXQovUmVjdCBbMTE3LjU2MSAxODYuMDQ0IDIyOC45NjEgMTk4LjY2 M10KL1N1YnR5cGUvTGluay9BPDwvVHlwZS9BY3Rpb24vUy9VUkkvVVJJKGh0dHBzOi8vd3d3Lnlv dXR1YmUuY29tL3dhdGNoP3Y9YWNvOEpDRHRyYmcmbGlzdD1QTEE1NTgxRTRFNEZGMDUwNjEpPj4K Pj4gZW5kb2JqCjk0IDAgb2JqIDw8Ci9UeXBlIC9Bbm5vdAovQm9yZGVyWzAgMCAxXS9IL0kvQ1sw IDEgMV0KL1JlY3QgWzI1Ni44ODQgMTg2LjA0NCA0MzkuMzM2IDE5OC42NjNdCi9TdWJ0eXBlL0xp bmsvQTw8L1R5cGUvQWN0aW9uL1MvVVJJL1VSSShodHRwczovL3d3dy55b3V0dWJlLmNvbS93YXRj aD92PUlVblZlNWtNN3FNKT4+Cj4+IGVuZG9iago5OCAwIG9iaiA8PAovRCBbOTYgMCBSIC9YWVog ODguMjkyIDc3OC43MjQgbnVsbF0KPj4gZW5kb2JqCjk5IDAgb2JqIDw8Ci9EIFs5NiAwIFIgL1hZ WiA4OS4yOTIgNzQwLjg2MiBudWxsXQo+PiBlbmRvYmoKMTAwIDAgb2JqIDw8Ci9EIFs5NiAwIFIg L1hZWiA4OS4yOTIgNjg2LjkzMSBudWxsXQo+PiBlbmRvYmoKMTAxIDAgb2JqIDw8Ci9EIFs5NiAw IFIgL1hZWiA4OS4yOTIgNjQyLjI5OCBudWxsXQo+PiBlbmRvYmoKMTAyIDAgb2JqIDw8Ci9EIFs5 NiAwIFIgL1hZWiA4OS4yOTIgNTc5LjY2NyBudWxsXQo+PiBlbmRvYmoKMTAzIDAgb2JqIDw8Ci9E IFs5NiAwIFIgL1hZWiA4OS4yOTIgNTM1LjAzNCBudWxsXQo+PiBlbmRvYmoKMzggMCBvYmogPDwK L0QgWzk2IDAgUiAvWFlaIDg5LjI5MiA0NjAuNjk3IG51bGxdCj4+IGVuZG9iago5NSAwIG9iaiA8 PAovRm9udCA8PCAvRjIyIDUwIDAgUiAvRjIzIDQ5IDAgUiAvRjIxIDQ4IDAgUiAvRjE3IDQ3IDAg UiA+PgovUHJvY1NldCBbIC9QREYgL1RleHQgXQo+PiBlbmRvYmoKMTA4IDAgb2JqIDw8Ci9MZW5n dGggODI0ICAgICAgIAovRmlsdGVyIC9GbGF0ZURlY29kZQo+PgpzdHJlYW0KeNqNVMlu2zAQvecr dAuF1IpIiVqObdAECVAgaA300PbA2rTFVotLUgn8953hUIYTpEAv4uhxlsfZ8mSf5MndRR7PD+uL 61shEs6zVkqRrHcgNpmUdVKLJmvzOllvk2/s/nJIV6Uo2U4NpjfK0t+z8R1Kkm2mYUg5m0b6Pdhp b1WAzLgnrFfjflZ77d7Bf9kwne2zdFWLlt1c31xdBbRlj0ffgZcf6wcguOJ1VhSSOKhxm64KWbAH lYqaPaVSMgUeJJfsfe8mugxMUVCbP7MyI6p6HS2JL0rTIV3BjbbKB4KIuaPzekB6hWzxRYfZa0t3 o/bo6Rk/k/0N73L/5CjgcWbcTXZQHhOASUHM6c1sTfBzTBvBIJCoIImd2aSiYR1pKatJGBA8krw1 Vm/8ydO0w7Nmy/Osdj5axxMDd9GRU0OUvBl0tvAWIit4RbzXnXGg0rZsOT1aF3nOvue8tME9oia4 CvcqYvd0qMOhP5KJnwj6NZkxXsZzSftIim6agafdxFChieDR83iWpQy7p16iaGgg+1bmMWZRFayL zYFRCIlJmo035/BO6x5K7+gPM1pAMc4JAgwEiR3Im7n3s9Wkp4B36EeQl9CuUzbqDkSf4m+1ctEK TN7g/gxdgYa8hhSE+Fvt6D92LEdvkBnIV/jxqtdnrQ1InDigNWjrMGmFiGXF6+V87hZmPOQ0mPZY HugFegkgSy1BhDZ+M92hPUSFcWEEJfsF/ekICrmE8+7LdBO67bRuYNfw810DJc1EJeGsM9E25Pox LWCROGd+9hijluwTvBUwT1TOvb3cXDnP8ryBu+Dn8/QzzrhHs+tbXp9rF23WlMWi/DXl+O60giQ4 2j+v3cs8y6uT97iNJLtTFIbIvY4i26zifDF6TBsY0XTFWW/eCiFKWLknTriIKkE7R0NhL0/tC2G3 xm1m52izlbCNwCkqkY5VPakZtwrlg/GvMg70X1TRzaHPSs6WR8we//NYX7ig+pahvnT1ZBRd6UGZ njDKBmD38Z/2MQBhKLaEnqYC8GUqzo1x1Yzwhm2cS4Cgl/63+wiJAwMSKA0v2y/hNSRBlgkMRwUN V5K36oXSx/XFXw3d1TgKZW5kc3RyZWFtCmVuZG9iagoxMDcgMCBvYmogPDwKL1R5cGUgL1BhZ2UK L0NvbnRlbnRzIDEwOCAwIFIKL1Jlc291cmNlcyAxMDYgMCBSCi9NZWRpYUJveCBbMCAwIDU5NS4y NzYgODQxLjg5XQovUGFyZW50IDExMCAwIFIKPj4gZW5kb2JqCjEwOSAwIG9iaiA8PAovRCBbMTA3 IDAgUiAvWFlaIDg4LjI5MiA3NzguNzI0IG51bGxdCj4+IGVuZG9iagoxMDYgMCBvYmogPDwKL0Zv bnQgPDwgL0YyMiA1MCAwIFIgL0YyMSA0OCAwIFIgL0YxNyA0NyAwIFIgPj4KL1Byb2NTZXQgWyAv UERGIC9UZXh0IF0KPj4gZW5kb2JqCjExMSAwIG9iagpbNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYg NTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0 LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYg NTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0 LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjYgNTE0LjZd CmVuZG9iagoxMTIgMCBvYmoKWzUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUy NSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1 IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUg NTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1 MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUy NSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1IDUyNSA1MjUgNTI1 IDUyNSA1MjVdCmVuZG9iagoxMTMgMCBvYmoKWzYxMS4xIDYxMS4xIDYxMS4xXQplbmRvYmoKMTE0 IDAgb2JqCls1NzEuMiA1NDQgNTQ0IDgxNiA4MTYgMjcyIDI5OS4yIDQ4OS42IDQ4OS42IDQ4OS42 IDQ4OS42IDQ4OS42IDczNCA0MzUuMiA0ODkuNiA3MDcuMiA3NjEuNiA0ODkuNiA4ODMuOCA5OTIu NiA3NjEuNiAyNzIgMjcyIDQ4OS42IDgxNiA0ODkuNiA4MTYgNzYxLjYgMjcyIDM4MC44IDM4MC44 IDQ4OS42IDc2MS42IDI3MiAzMjYuNCAyNzIgNDg5LjYgNDg5LjYgNDg5LjYgNDg5LjYgNDg5LjYg NDg5LjYgNDg5LjYgNDg5LjYgNDg5LjYgNDg5LjYgNDg5LjYgMjcyIDI3MiAyNzIgNzYxLjYgNDYy LjQgNDYyLjQgNzYxLjYgNzM0IDY5My40IDcwNy4yIDc0Ny44IDY2Ni4yIDYzOSA3NjguMyA3MzQg MzUzLjIgNTAzIDc2MS4yIDYxMS44IDg5Ny4yIDczNCA3NjEuNiA2NjYuMiA3NjEuNiA3MjAuNiA1 NDQgNzA3LjIgNzM0IDczNCAxMDA2IDczNCA3MzQgNTk4LjQgMjcyIDQ4OS42IDI3MiA0ODkuNiAy NzIgMjcyIDQ4OS42IDU0NCA0MzUuMiA1NDQgNDM1LjIgMjk5LjIgNDg5LjYgNTQ0IDI3MiAyOTku MiA1MTYuOCAyNzIgODE2IDU0NCA0ODkuNiA1NDQgNTE2LjggMzgwLjggMzg2LjIgMzgwLjggNTQ0 IDUxNi44IDcwNy4yIDUxNi44IDUxNi44IDQzNS4yXQplbmRvYmoKMTE1IDAgb2JqClszNTAgMzAw IDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAgMzAwIDMwMCAzMDAg NzUwIDUwMCA1MDAgNzUwIDcyNi45IDY4OC40IDcwMCA3MzguNCA2NjMuNCA2MzguNCA3NTYuNyA3 MjYuOSAzNzYuOSA1MTMuNCA3NTEuOSA2MTMuNCA4NzYuOSA3MjYuOSA3NTAgNjYzLjQgNzUwIDcx My40IDU1MCA3MDAgNzI2LjkgNzI2LjkgOTc2LjkgNzI2LjkgNzI2LjkgNjAwIDMwMCA1MDAgMzAw IDUwMCAzMDAgMzAwIDUwMCA0NTAgNDUwIDUwMCA0NTAgMzAwIDQ1MCA1MDAgMzAwIDMwMCA0NTAg MjUwIDgwMCA1NTAgNTAwIDUwMCA0NTAgNDEyLjUgNDAwIDMyNSA1MjUgNDUwIDY1MCA0NTAgNDc1 XQplbmRvYmoKMTE2IDAgb2JqCls2MjUgNjI1IDkzNy41IDkzNy41IDMxMi41IDM0My43IDU2Mi41 IDU2Mi41IDU2Mi41IDU2Mi41IDU2Mi41IDg0OS41IDUwMCA1NzQuMSA4MTIuNSA4NzUgNTYyLjUg MTAxOC41IDExNDMuNSA4NzUgMzEyLjUgMzQyLjYgNTgxIDkzNy41IDU2Mi41IDkzNy41IDg3NSAz MTIuNSA0MzcuNSA0MzcuNSA1NjIuNSA4NzUgMzEyLjUgMzc1IDMxMi41IDU2Mi41IDU2Mi41IDU2 Mi41IDU2Mi41IDU2Mi41IDU2Mi41IDU2Mi41IDU2Mi41IDU2Mi41IDU2Mi41IDU2Mi41IDMxMi41 IDMxMi41IDM0Mi42IDg3NSA1MzEuMiA1MzEuMiA4NzUgODQ5LjUgNzk5LjggODEyLjUgODYyLjMg NzM4LjQgNzA3LjIgODg0LjMgODc5LjYgNDE5IDU4MSA4ODAuOCA2NzUuOSAxMDY3LjEgODc5LjYg ODQ0LjkgNzY4LjUgODQ0LjkgODM5LjEgNjI1IDc4Mi40IDg2NC42IDg0OS41IDExNjIgODQ5LjUg ODQ5LjUgNjg3LjUgMzEyLjUgNTgxIDMxMi41IDU2Mi41IDMxMi41IDMxMi41IDU0Ni45IDYyNSA1 MDAgNjI1IDUxMy4zIDM0My43IDU2Mi41IDYyNSAzMTIuNSAzNDMuNyA1OTMuNyAzMTIuNSA5Mzcu NSA2MjUgNTYyLjUgNjI1IDU5My43IDQ1OS41IDQ0My44IDQzNy41IDYyNSA1OTMuNyA4MTIuNSA1 OTMuNyA1OTMuN10KZW5kb2JqCjExNyAwIG9iagpbNTUyLjggNTUyLjggNTUyLjggNTUyLjggNTUy LjggNTUyLjggNTUyLjggNTUyLjggNTUyLjggNTUyLjggMzE5LjQgMzE5LjQgODQ0LjQgODQ0LjQg ODQ0LjQgNTIzLjYgODQ0LjQgODEzLjkgNzcwLjggNzg2LjEgODI5LjIgNzQxLjcgNzEyLjUgODUx LjQgODEzLjkgNDA1LjYgNTY2LjcgODQzIDY4My4zIDk4OC45IDgxMy45IDg0NC40IDc0MS43IDg0 NC40IDgwMCA2MTEuMSA3ODYuMSA4MTMuOSA4MTMuOSAxMTA1LjUgODEzLjkgODEzLjkgNjY5LjQg MzE5LjQgNTUyLjggMzE5LjQgNTUyLjggMzE5LjQgMzE5LjQgNjEzLjMgNTgwIDU5MS4xIDYyNC40 IDU1Ny44IDUzNS42IDY0MS4xIDYxMy4zIDMwMi4yIDQyNC40IDYzNS42IDUxMy4zIDc0Ni43IDYx My4zIDYzNS42IDU1Ny44IDYzNS42IDYwMi4yIDQ1Ny44IDU5MS4xIDYxMy4zIDYxMy4zIDgzNS42 IDYxMy4zIDYxMy4zXQplbmRvYmoKMTE4IDAgb2JqIDw8Ci9MZW5ndGgxIDIwNDAKL0xlbmd0aDIg MTMxMzUKL0xlbmd0aDMgMAovTGVuZ3RoIDE0MzY3ICAgICAKL0ZpbHRlciAvRmxhdGVEZWNvZGUK Pj4Kc3RyZWFtCnjajbYFUJvrFqgNxd3dgjvB3bW4tlDcAgR3dy0uxaW4Fnd3Le5SpDjFXQqUy5az u8/5/5l7JzNJnnf5etf6EmpyFXVmMVM7Y5C0na0zMxsLKz9AQlFci40dwMrKwcLKyo5ITa0BdrYG /ecckfo9yNEJbGfL/y8NCUeQkfPrmaSR86uiop0tQM7FGsDGAWDj5mfj4WdlBbCzsvL9R9HOkR8g aeQKNgUosgDk7GxBTojUEnb2Ho5gcwvn1zj/+QqgM6EHsPHx8TD9aQ4QswE5gk2MbAGKRs4WIJvX iCZG1gB1OxMwyNnjv1zQCVo4O9vzA4Fubm4sRjZOLHaO5sL0TAA3sLMFQA3kBHJ0BZkC/igZoGRk A/q7NBZEaoCGBdjpL4G6nZmzm5EjCPB6YA02Adk6vZq42JqCHAGv0QHqsgoAZXuQ7V/KCn8pMAH+ bg6AjYXtH3d/W//hCGz7p7GRiYmdjb2RrQfY1hxgBrYGAZSlFVic3Z2ZAEa2pn8oGlk72b3aG7ka ga2NjF8V/kzdCCAtpgoweq3w7/qcTBzB9s5OLE5g6z9qBP7h5rXNUramEnY2NiBbZyfEP/KTBDuC TF777gH8+3KtbO3cbL3+Q2ZgW1OzP8owdbEHvrMFO7iAZCX/1nk9Qvx9Zg5yBnCxsrLycPMBQA4A kLuJBfCPABoe9qA/hWx/HL/W4ONlb2cPMHstA+QDNgO9fiB6ORm5ggDOji4gH69/C/6bENnYAKZg E2eAMcgcbIv42/vrMcjsL369f0ewO0CH9XX82ACsf7z++ab3OmGmdrbWHr/V/7xioMoHMYm3Mox/ l/yPUFzczh3gxczFAWBm52IDsLFx8AF4uFgBPv/t558O/Kf6P09VjMB/Z8f626OsrZkdgO+vIl67 959CXP+eDLq/14Ye8N8RlOxe5xkEoPs9/rqsXKwmr29s/89L8KfJ/9/s/+Hl/zr+/5uRtIu19Z9y ur8U/j9yIxuwtcffGq/z7OL8uhuKdq8bYvu/qpqgvxZa3M7a9H9lss5GrxsiZmtu/U8bwU7SYHeQ qQrY2cTiryH6zy28OrcG24JU7JzAfzxwAMxsrKz/I3vdOROr14eK0+td/SkCva7Uf4eUsjWxM/1j 99i5uAFGjo5GHoisrwPGzsUF8GJ7XVJTkPufsw0AstjaOb+aAF6L8wGY2Tki/nGj3FwAoNgfR38R NwAo/pt4AUDJ38QHAEr9QzysAKD0b2IDAGV+EwcAKPubXn0q/CYeAFDxN71GUPqHeF99qvymVy/q v4kTANT4h/hevRj9plcvxr/pNU+Tf+iPngJN/4WvmYL+hewAoNlv/IPA/+DrIr2i6291rj/kdi6O /7J/VTH/F75mafEvfG0u+F/42gnLf+Fr2tb/wte8bX4j22vetv/C17zt/kHOV93XH5t/iV8zs/8t fg1k/7pbdr8L53tVcHCxcwaZGltbg8yc/xFwcP4W/LWrv52+iv5VK9trNU7/wtcgv5VfNxvobOEI +lezXgtwdrP7l8Hrpbn8C1/Ld/1X618dePyJ/zXfJi6Ojq/P/j+fQK/D/x/+84cGBHIHmSAuL9iZ CARb1ga331eLEbkx704IzVLvaqbSM3stO3a4/ESFS6KvygzccLwVSxrpRV/dlqK7EV0he/Y6aqmH C2v9pNr26P1kEK82vduGuDSFOzhZcCRWN0CCQMysIbrn/ezg/T7ACqoFskuOOsfBhRdVJQ/r3q1f xr1uoOzbWOjCrupeFbc80lPZDHP0uyjdgOI56lzjrHl8ClhnZhJ4Bsxzd7S5m9tZzOzJFzK5eEZE n+NojkIv7e/sMQ/znmvlGuxO3QRUBNr4JFA3mGPTNF7iB8lyeIteJUWxcqGRRWYk+gtNwh1oRg6c NeRdy4Jlo5Obo/g/DAIkytC2p0NjTVS3qMVHflirZ3fiY+bU8RTPPLMZ5CM6qXbuP+N6RlbMZToG iQxaOZ+osIel9SVYkwbf0InAdh6EHSQvci8R5A0O8PnO0NYnvPiJxi+xG+c66/R4GbmjnHS+4bm/ 8JmclLGF/6xzrfHhRFhjCoJ4hNLoK5sFebYArOHJZ83hwUrMFythSKLS6Ok38pZjNRtyXZ6oL31K n49MgkRW9AX3HkSR33xfdNDYdQleMi3TAU3HpFvyy/XYQo/KP0ZQsqvnVeskHg1kiRtjioyVK0IG 2gXX8lSVGBkVJIfn0RI2h5lfF9+pkPBP1E+XznxE+mFaJW6a4D0tWzZ1GvJW3+2jIlrll0kQpaTk GVvWFw7NT/nd4SxjU1ddciy43IuVcySCqRoEiiPHFA3CwY74dATlSoFDOJ+TkbeCKDOEm7Qte24t 8kYaGFPf/XKrernYRnyqp4rb6Zva0t3UVGKRKKR7P6N7prS3R5zT0dvMgYEKMycmcmltgO7UU6oY qtJ0V4bUGeNhpu5RsrnQZfwdY7zPkM+FxD5m8FZlaziuWsnFcYFeQcjFojRmPnBmWHJiObe1R2Uu Ur/YB7+pmQymu3sOcYe5ZMLzHcP4N1KO8n3f2+23hW54QZ67qvxv5kfg2shQFzbHWRKncUzcW8tw 813xU8oI6dIDMQW1BSO4l9K55L/NpddBzjJ6PmHIdOSyQtJ3+zl+KP8Uwb9YhT1ikgeJK45Smssj +34+4BlmYTdH86UlQ00PpSGTrx9GWzuDxMbbBM8uOKwmLmAkUFbXJlQJItTXNi3xcPcE+TaHgFWX 4NlMATnDEXppATqpPUC5/8UXfmVt/CQmo6IbDRJKADKY7ivtyh5ZQEqPWU0QyhOw0EQzJUgydeiZ oIQXMujrT7VBOl20kuDWCaw86P7lma70Jyr7c6xP5FMUJKtXfHVRagAhHu1Ln9LM1Tg9xiNPUAj+ ezidtK8IhLA2DcydDBvb3d85i7CplhPfS2ozVyX3VKbNaa7IJ9s3VCGf5jU35vpquNwVni8oH9Wx 4cSXkz8EEPsaZdDwcBYE7cNE/Vid6RqAYxJqZRwtey+NZ7LHMcBRPUowxGs1gGRxbABzAmvdhrMv Zgsft2GUvtsQ2HSr1D2EMtfFf55brJNGNZ5tyj8FNsFZSLN8lnhUZN8xHONyrFSIveLPx9+pKxj+ +b2ic9z3+y+THMxFRUOEku/hWORmfAbCKodVPvDSBaW3FcIwS3upWEmNqXC78Jyf2kl4PUFv3sEz yuGfwDwQWGEQIAsKM731rdT/Mgz5TrjKFwNz3Zcemg8vWSabGdsgmHFyLgw2hf2+Pc4mHbz3MJHF BuFYk1R6fD0grP3ldGxdZDJmnHdLRRvS5iAOltIGJKlyPZ1e8pG7lSg+8YGvyFXIQqrbQItu0CMu VOk+FY6Oj7B4H8+Ua0Ewb12LswymjQuLPPYcWUsiA7ZailTuNuZLZxO67tKcnlYuNn5rMml9ZcbT sjVD89wYEf8H4gZmYWL7EZ7bpGaF1sf6oSXtz0MSvnVD1jcPN+URmG2maWwF4/zAgJ8Ac3Im7CHE jlWf6FxU1QnGczg+jvdegM4k1WjrhSOZyobGLRQeCHIBsjeQHg6Mm3tfLlBm5OqT/MSOYGAR9lzv LIGOU/MrmUHaJCro9JzVZguMyvZaBh+dfQFcyTqxatpUghY5EutuqR1kF4+cKmJYiW4Wxl1+bx84 wFI+wYENOmiFNseed0lqOPdUJLNPUDpSOZxeWCH99p5qQM2HH6jMCG0BJUYGsjB7DZ9sLD6Vgcwu adub/fd1HDKaiuFZshYkWF2I3+XdkRx30JTtfmMSm0ZwY6Zg5KyjWi8UnmkpSBWirASJyjNovmmX Cq9yhjYYYZbrSBxCMiaZyA/Lv9I2jT0vtQhLjquNjO0Xdlc6Kc1bt9+OdYIqymqRUfBfVi60EL7K nrWLvp49yinJIuyGWJETOJdj1LVBnCAjliN45Mb/2mtFgBWzYBPARwGrhAwwlJzp0ek4BxxYcLt5 iaGZCAqtV7r6ZIyhqUBIFG7AzzBEeDuJLaYcm8kPMl5XsJEHVEQXf1K3fSgY2PaljPYzwGAI1SP5 3KALPifHeZAbB2d+dRhNtZEDyNcBzDpyPnowqD9AovrchoNl3nO7GTVB4m1S2dSGugqie2qrEukr zO+zKh2qOaTfkn9j4BlxUVBunxAy9hv3PX0yWJNDeJYIlBUR5LoohEHSQPZ0Kq4/8XFRMExWXPIv 66Tg+SzfOYsEdXq333a5E7rG66EMiPS6vIwr8M5xNaQn7WWRDCUm5shd3kuSp0oiwM4/QEhrBzBx +NWmJ8+sOfVlSeK1fZ9Vewn/VNirgdLoP2ZSA/eF/Tg+iDrgyUNSV7EeQ53v7dK5anT7fJaj2bVM sf6TLhdXWBLl9rIrjk7Ih8l6epx9XPkd9UJ94npf25a0CkVCrUcmXpp8HiOUQVOqXylrSUj6zkBN zIAlj8kuv65+/GL4q3LOZlV5L/ploXGIEE3JdlPhNjtC0OGKdob9chIhOJgZhCaqkqDwYhzrXX5L aQ9Dwd0lH42qQKaiNbrhaYxalK0dajJz5/MScEJPjaewJG+mzI62Vhmw9R1ApzRX/y6uirke9NLh 6JcddemCrPalxSNKw8oaV7iV9nvt25NZ5HW+6UDmRPqHrkeXGsRuF0Zs63rTmx3Ohh/NnqUmCk5h n3DvNeRgdfhhyl0nCJ9C9qJENMOV7iMVcRhaUaqVYMBnpwpmV2bSc8BiEaR8iLFEO/SJMQOjcfKc VZQt8UNnvRvPQN0OvAyY8vVENWCaI2fB0cERruLPkmTceYiMRWk3v+EzEBdtjN6lX712t8o4qf0w ZMziYmORY5XIsEqcUfAI2oTCJ1pmKKIBxEkTHIyyyNU7DhmfQg1Ign3a9JXcWNtuymF94TexMzkL tDnMGc3Q/uv+q9L3nW6KyhpxVZJOcqmjqkSt9T97grzgEB4DOjTWO5uu2SvfE8RXKjltzMQ8XqL5 dVKLB61YRF1+oyJeZNq8Es7eCyO8svshn7JC8TZrsgp1Di0H0PyFfUIcveTdOA+tXA1d41kbyVDG 9GWFCZuK7Frc5h2LdY7qHKTliDS1ZXaJSj0i22mbpzH7SGWNpv+6IqONfFNwy+CQGRPFKpDDN7+/ SuDxNpknD8jTUyE0ase8r0btH7H33TaZWE68kVjUdg/yhVuM9UevRTe2mjRfdbbGArJXHF0eTeo5 1UZInB//6LVFF+Hdy02s4otl9z5cSTbyEtExdZz0G7z3wS8rH2W2hl2bAmFFDNX5vx/xy4Hs1QbH GkfQ7v22+DK+QETljK4JfrjTsR1Oaay4Hlp/E3MEjz37vXNzIYj3kOpnFyUFCH5KCHPeigSJ5MJs 3Sw4Nzk2KnsUqRKzNLAeWcWNSKqBr/hmkkqpLXhFMO+55OUjhaUWTJuWE9SzoiPh0qFUVf1x1rzC E8UB+TGDGT5jBjnkO7Hw+Mmvb5CZRDgwu7fdeZIvuCmJul6MyK8ihD5uUw5jPT4jjSSNuBTiRRrn NQM/bch01a1SAt6q3OmW6hoab27PU906CnXSj4vEAYsI7/hiBDPssMTfoR8eLfNUsuRwlW7jizdu q8kC0uoK+Ywo8N+TY23TkLhjsJnvFR+VO6RaJQklhx5HMQO02dpDo6va197qnFWm70gbQbWqu4rO 5IN/NLrtz3hQPpumv7zpHT0LRad2+rrhm6ELBXoIv5/Vo4s5W0Hk3700Ym8tEJnGWl31CsnZ3ZbJ XvgML1rLP5hrXf5zwMloj/dguVOmdeZ9hFnbtRxGe+5q51HhmCykoGHLaQyzyOmNngDz1REurIyP OhWpkTfy3aYd8mKjlQvtmVsG+nNVikcAhPRhavC6fV89Oyl2y6d+H1mG9Xo8cYlJYQ+RdGWDEUiY ds4UzCf+raQaY6n9nPd6wWyNM+/PH4yZgijC5O2kWU2N44wIW48JXsay1804ZfTHyq0S6Lz2+8b5 VxHh0xp+resxg+SalowJk0ZNAuZFcGqyn8YsRu6cGDEn8vwifg4aNDhlfB5Ur8XLOiBhMnIu8QBD 1lxMjzn+aqGuv5TmIHgyRy/MLlyPpMlT9Fn6VjLMkS1pNWxhGK2b7ehiKrxxIXAHbhEjMug8DcdB tJM2mE4zsbpRypiLvCezNcz76bqIM0/Pnp70LS1hh4ryBIX7Ydrr1xn805f9rcpGXWnXm3fHmiRI Xf7Wp4jU7jA65vVz6Cchlty+hZgSSHM4qxQJCfpW1hK1Zx5oravUxL/I6KeobRs156UzlYEQ2Iro aR4VfiPYc/u+V9oOHSelK2Rg2byQAKqibIlRjQwG96CyAb3Hsv1k+msycDYkpDOndkbye4g5PZvd 8ID3sxDxK8qPnwL8K2/a6c0hwL9aEC+pmd/IvRvM0TGr1tDimQOyEIaYhOFGBsC/g/7IdGfBc8ar mMc5eUiM4HnAx789Vrdp+cvG0fUmQ+U6hhJFOCEH3tjf/bJt2+CurWhUoa+5PxvJ40to2aeg6utJ LrRpNVGgl9mRqsSFzxfTr93ww4u3vkvbjnU/yKugONues6fjbtqizFTy7ZpO/LsUZxn1aZlSUU+5 DYDfNL0slsZHzkaA4dPph2OW+3XqPjO/TuTCPJGmnUIj5ht3IjYXLPEQStbrfD0zFLwcxHQh1GYs qOgj+T2V1OUU1jyILfU/9ycOJZA0QMYNEpPSz2hM+U7Ja0OJ4fxUYYWJDYSVU8H1pRP25dui6Stc d+3Z0n+hgA+C6aTvCNPi2icSsoBSmgk0UkJfqi2ljmaD2VpwhWVbNXcV2ffn5K7jCXh7G497RP7r qkHp+rnHMm9LmZOkpAN7udN47WW3oCUa0jlX+tLjcq9u9+aC5szryk1Vh1E7Qi4hz6EZkbFlUxTp bNvGlmnDUtFWykwW2QapRE2GpOoddEX9B9Qq3t70e6UQvJL8lyx59Ah+Ir3+AnHysgE8JRF/OZyI s33aRTGWW6tuL0GV4d4UkV0qoHHeNrWJWn1A6KZtOzk0Q9D2ZXouHzY2bwTZqhKai4FG1mE6WyMy Gc0pAwyoQEmfpthF1vvmIZjxZYG1tIECiR8JMs4UTx3uvOJLL3flO7cbZQqpxUc/kqnzyyMh6dg+ wUGyHLRpHYoQd1ZsCIztZ6S0G20vr8Dovut1MpZgfOQT3tEh2kZthPqecQMjanNTFN82vzjj7QN5 zbkvR+9yPEMvIW/zIr8qZYXm3TLJAqZAp2CE/TfWwo1P+xWbJtv5od0wjEBr8ZxpX2woz5A75r0c pE4lmExNHCxNkzbH2CmsHyHheFEifr3vmUz9IXWpBIrKnYk11LqpsRV5eySP/ZqnFLKdPniFS9ZT ciLdB0l6GZXqWgo0UdkWopt+zaYdH5eXQk6mtf+0i2w0S3q7aiCBgQfJtmCKivr09clISgp1lZOi q0hQ53Ck3AeGx5+Heqk1bd/fCsgu22gVTltR7AZlLDgmnoR56z0k8GErHqmyy7pb2mupjAnnNnOv kA8Ri5ZJOMj6iDhf28FBtNoQNxwfWyKy+pQJlZtgxfvyHcFBrQ77UKnXffPY7p7XtUEdccTrcFtn tet0092krkqVb4fEZdLUy5OzsDr/oC8vjEM3O7+Zn7viSYKNsZ62TVd029USzETyHkHpus9tPl93 w2enbPvK7z061c69nrvI+fahunDOyvHCVzYYQiqm0OZ67iLdtSK8ypipVyfdvSwwYHMY/UOIaFPq d0diBbymbtqfZByj/IaFYjwSsb5ajAZ7qZ1qlDI7Yu/7nfUiJNdNUTrryREWWP23GAJcNmC+eV4b 7+BVB6ZWEx19cfwwfVJUqxRRnd1IfTVNYA4PcTlM+E6LxmL8I3fcknLn/tXP73OaE+w1+iv493GQ qJ4yVSdPALBSzszEl+dnD7MOA/UXimXhRxZzw1tCcW6ZfcTn4SPxJNKnYVOt07gG7JuYFi0pH4ia zJOQkD2QJmabwUG2tbVqKRb3MyzYriQ4IEFasC6OkGMu7Ukb6tfalhI17roqrL7bBP5aUGSfVwaN VEUUf56VEAAt/AfRRTqxeElNk6x3YRJitu/8SkaVmXwp2Mq9EBB76tlj09AMQJrl+Llc52tzYoWo ls9c6Ne07dj2tubIKXklfOVJu3i3vIwFWu9Xjx5eniB5I44F/RigzzvDTOctTpn1aijHt4CvOhth noOoA0AAbPFZ0Ifn4rZkpgPT6TDdqy5uu+jUk6q5pe6DFkKpXl4ImXkd9BksVLP6Yg9Rfx4tnRxk 9EhIuB+mmJq03G1yYiWPMjdywDMkKl8Jkhjy+i2fHJRtf2hF+ToksSZt/tscgs4elPh4Ox/ssUpp 1dx1/kj1KNxPTSoW6nC4E3srri3P6e/ywXWQvF/wV87qufX07xLPZcf62ss/BbmghTCkqwn2bFxu NVKc8bNAHflOpN3D01Q8hi43aCAkQigQKfHVyfgKMRpNdBjlAn8uhrbR1SO+IHZBfZ19ceQrs2ie HN4j2cq37ZihRpL3FVwddU/dcvEwUqK8UDXB452yfaBK1jEoCuytaP3lFhr+TqdH0XT/5Ei1M+3X ull9ONxUgxv/2hrlKCqN3oFiq0AGQxN3/M3Tlr5i74zHo4xpy40CcTE3eS01qfy6/fd4pvRtW/se XFIFX01JRm+nr6aaMtyVtR5scMZfsOG0OGHkZQprKftPeab3OeaZHXHR0eM5KWxRJsSS5a6WMYlg kzbJcARYfCFjJeXnIUdX7QK1i6YE8t2M3a/aj3CDyJ+CezWBNkm9X7sGiVKkvrezOZ/hYkvgueoh ckQqzp8z6nPfLU3eWXwe6CSvV/eT/a6g0VyEF/iT3SOQ2QIeC65sroqlDvqjnnGlq6NAxyZX+HXN VqtnbBxR6w/lncyGFhv09qkjjmwHOkZlEnakkuuo4LTFa/qjDD3sx3AnlORIrlRoSDR6BWPrgHRF fR1rnGb/8YSQPqMd3E5oMtj8fh/g3jctPLgRtwfrbSCFeHilDwwSXWKjuqvzXtLlgP57MZuyA6Nd 6sozvaoWZmY9pZBh7IWzxjgUTT578z49G4Ucmufy+hOIwqkLW6fFg6u7JeuPk4eyJrbYYUU19WGE QVo9610cYk0zibJ7zfkwMo8P8JXQVqzsQzWzHwm+f17lIzT+kOX29UixJwMQwsI4tuZ8uLZk/WDw 3FXgledrGJ1v++GyYOQ8DVGEOWXS0ElN+mxaVBy6bcISGLWQIbfBZpUG1PD5wQM8zekU1cwIXzGN jP9cZ4xnFUQfs7VUD7Nq6v2r/Tt/DLf24KbsewfxwCrB4QNG5eDJhP4xssOMMYPFAZVzvsyJZudU 2omkvdU85X19t63wGseqjMdfBevIduz3SO1pSX69eP4+MHqwJ3AF07b04kYAgblWBWYSFNNVxjWl q0zL0JzE7zaJRRutC4L7tqeT6LJMjDKMlnLdH+f8ti8RS30pSWQwvCEXYUB1/Z0emdlvtWi/oHMw /oSBaTgruBn4LBvVM855WkC1UAKkL9I4qZigaqf2TkQbFmAs6Mrs+UAnS2okHSJwKp0Bj4lJ3HpV v8bAKLu9w9FrOCf6jaeKCYUQ6dhNQIlIUWBiGmMR9VJqVDTR8VYjk1yp0XGnpDXvs+IkWzaVgwjc Mv6QIlaf7DW+D3esjya+HzydubYaL7PYuvXyoxrT5ScKmg/fnAwY6Nk+VARXwTMm2WFr/ogNaKc9 F5q4ifStWNySyIPbLG4c4HDZu+2yC5iJsa4ZXbFGWquXw1DdGv1ZOdYF/x1nznQ/b4pnY0H+ooCl JWJtqTdNnTvTrfQjQ3fH1IZCQwSJjYcXOXqPv4WLRK6LJROU663tJ9lJMUZqvRdVNMhsh1XExJl5 Rv/NJz0SOswDe8pr1N5t0uHoJ1lrr5WDYGKpEsOrcaavz4cig4ZGNbYKshmDwP16p26lqohuIeoV la0U1ZRyEYnn9GXJKenOX+8LBpRcRSQpxZh82FQgz7Ctar60sMTOgikZ1nDKfEV/CSq/291cwzqS /HzYNhorPQCrgzFM3wy5hIb72Ao2gN1wF4u/55sobGpZSdYDEMdkpv9y5e/8aSmu1YdPLtEh7aWT vkHCntO+TY6pItsFgSTtuKEkrNYcoDfIXWEjfrrpbn/ItVOtBd/q/9mdi4dGvVCEVQCrgbds3EiC 3K2KbrdWz0VcakKIOIha4lyGd+9NfLiKSqfm9anUVyWkBmJeJcC6Y/44F6Nht0OdXGQ5PvbSQ+Ic OZPHtkqkpFeaifmSPzZnK1AtfS+Mb8eSz4v3/Olb92QJrmQkzhkOkNBQ39xYWd4F7yESZ++upQXH UFivgBkRT2X9XS97qhCI4iuqfh8IiYuOwg5jdzRPcLS437oH7VmjqqXQVb3c+sBlv2ZpkLvUPVYA OJnLfgXVKC8HxVR0ajwlD3vKKHbJO4BV8DjDaoD5DcKTHiui2jZpCRfp4Cb87Qf5QWNTBK9aiIFB 6zX1wTrWxw+M3bZG9gALDOHxjS5WdhpOyHdlLP3FkteeX89pnHvfLmMq2WhczQk7eeEKMn1noJpr mqdwqILHiuBCZT+7x7RkQH5R7U5p/fC0jkitmRotgZO9hI0pO/E+9n3+otOpWU0v9fEp58A2usGq 2FpWbuMR5Xez1F77958TpeabPBeH2nydmqyU4/HkwO6/+GtxcrUUNBTSeVxUt1HZIDJieQsq6uk2 aPifmOPBRvIidbodYaWENogUPt32x0C/wgPIbomnwvWdHpaj3jnZxA4hdtU55HutNxRkwWOejdVs Z8TXb9KeuRSbgUAPAWcp+SNH9ERvWDmOGP7uHOS2EHjPBBuOmQsI4aU0y/29EIyLvsyMad3IFIrm MapoF0HEUqlfnPUTzIpr4PW2KBv+X4oKrJg9iS9zZxNMb5jjlVp8mFZTxX62ErCLgCWyAIhxmOTF ajKYtn5l1hPNAKbC6I6PHiAUrAthEWQVMy0eN9gXLGaOBhvsc91UOA87LE0u0tPYa56nEy+0eNwm 5wER+5hnUZmEG0Cdf0wajnHzkS4IiMq//qlqSD6LkyX0INxUSfjG53OltEHn7QRq67WEM1REN0kZ Ed+216qhZcQae2K4tmr8NgXrKOc05uUzhXTD+nnYjZrHJyppsXqDirwUYgmNUrkfEaVZqxkH63fQ huzt8YbuW/FOiqyy8+AIjXVEuqZDg+MQvMCVx+XRUgE8hrugWhGVTCKmh/dwYJld0jVHR+bgxI/5 03DgiUiSNH1i5zuZkMmL2IdZObFDqn3Ljkk+Yovuc657y1Q9OBLktBgK6tNrRqSsdHe8MZJHoDWo E9hzHTdxFlzUZ6+tjXmJNFK7ro5rCY/S9cyrnnPaytd1YdcGNzYY0K9VjyqslI9xX+FSCBUivBNa lU7P6V7NcSt3veXGGPg1swq/j9+VL312SJFQcB7h6FGO5gmqwOIjxl54P8KnMph9ZtqJtOyDX1aV ON4T8KNSolkKX6LedS3NFH74xfloboU6pAq4/5F8Z3vqVm2rlDkyW0KXFcmFZ8emcTAm7kfQx54V 9gan9HYQNyuRWxyrc1LhCu00CF44/d0w9oG62M3OMkFLhqBXZFL7+IATD67pDxFRClRUbiJ+nVsf MztCNnSlqdnN81T/0BT/DkEGlxef/t7JwXOEUC6f0MJ0VAxK70o6SQUSMtGSu1wRc0eY79vBXD8W OBp1T3CoHWpthp4RrBks6pR7WATp3Bid7u6LtDXHYfwOnKSmPnzYkmnxPPsxpCvHEM1KyqNQGuU3 Gq5nj1SxAF3TWXc54O3idSN15Kd0UfNmMN0+kti7doObiOFBS1YuIhSj3kqU771i7bcTfRJPOvRV Ip/dgJCAKBi+1rtY8xCoUhhtwtWxanZwTs6Tcstmc/DYrvtiYTniqcLMjjKWwNlCBmIBmfFoDI5/ mU8drDL/THuzuczM1pXV8da+oNnO/cT5vDLBuAiDdOvtSZZE3oZhLUZoWJDoxHUNH6GCrk7WOZXC csjJXaSaR24awYbrp6fPISvA8tc/qIm78eMvMypaEfRNBbbEdvWjZB9IfhAOb9BrEMRgESLk9uCj RuIGruO3+k3OP+gZ4yBoK01BekcJCKHlRMMShzifo983A7AoJL3WBx1rirFLj7COd2nvnmmDjXvG 1mNYX7RhuSjtndGOi8zx24zqo1gxK6usOpY/C5nJLoaZwZnzri4oKB4wD5rCVnW3MP/EZq4UrJtn l8iWUgHpJH/jwAlQ6VP0SgrSBY1oIMpc0mqA2sSOBr/QdRNX5c2wOpSIgzTGRr3X4jP1SG7BeH7O 8yqHfY2BYp0I0EyFCVjfPk3ViH7r5W+zriKDLndYEv6JuviTrK30Wo9Ck/bb5hvNPUbGZr06wc9u yLiCjJhZ9m+SUmyUUjGlhZVD8ILVvFLyDU+oKPXfcCqnT1/QTgy5Ylj0tevM9xEPfHjpVDC2r+e6 FPlYRRT9RSWo68W6moT2PoYXlnr1W3mKR0yjHGXNhtGoDV0ELNbmMAGespRAzVeUHWUjVNP0ggbZ mEUhKGEc+5okiNyoZV0M9089SHoVY4vSCWpN6bdblHS7pM4Xjg5Px3mHPEw1skUqtBJSBicdKlR8 Uzca5fSSsAFSfarqzwwSAqrVyjHf57zEWXZYzCc3iWaHEJq71PLCO69L8z9n5Wl50ixw6cEHB71z y1+Rcg64q2RqQcZVdhlf+6GeMHhYWtEMRRB/qZ/JY6jIgbYwKp52tGMRShDbmbAUaVHD7jtpnX9i 6mj+fsjFjKpc56HG7EfcsTj3V/yfIwzhkU8odJN7Ail22/vbo2ypwI5iW/p++CD2khM7nA/lbp+h ODiia1LD58uhdJ/uZ0Ptnvnzml06WU2yZJVi7yips3EQm642x+ijppOYyhPnEvKKX6bJ1MyowTvA eXJRREpzHWIlxPF2GpmHeBuEExJUGxLfMKLBHQ87gkWw+7BhxYrc7RC5FGfnuMAPsbNKxvm8JWc1 RHg8UBU3EvfwbEYyW8VzrTu/fhqZsgr63jwJ2d7dkDV0vbHw2if2Z2bGDh4VSeiOadtfAdjfL7YF dmJcfdu0c0RrZg5q/JhnSBK7jZdvmiZoJa/OZWlSxCQYPb3dKK9kd/haSlNHcRMW4mJIKI06+d6w CNbTH5ISITFL5tuDrbGitn/yPhG9rdCGzaJcfPNpwJ4tKZur7PrJbE17Un66KmMKz04ct1mHwjcQ KgyDhNldFFAlwpJckkHdWgZlK6oXWQNcaoJvlt9vy9XTnE72nZ01q9JV9MrOVTl054TYn3xjNt7r VbYPDzk/SaEjqeLzLcge7ZDOK+3q7iekQEIKtZv0LYGK28t3iCKXbVhz1FT7pTd7rU2mMKi0XwMF +ZQp+GIi3xtzmbzHzB5APK+YrPS90IG9PsnfcJ/VqeVz+pJUdAisZh08rDu5c8bXvEB9vxwDwHL5 5kB/Vvz54CB5lXJQHbrJrilq+arJpmGpsUOl6cy0E/GdgxWVfEJ90yB5MpGjfT665GbDl8Ya8Vxn wm1Lhv7g2of5UT32FpFk/XhP2beH4m/av+NRWqNYDCrKEsdPqLsHfO4cPR01xRpfgUOu+84XiDwA I2a0WKbp0F6cC9L7AqGM3r89Limv2gbX3rngjJ2djOXarV2luzBA9g3Lam2z0TWi5C662HpeN6xG Su+x+FJMKMbz0L7I5EGl8b1Nw13rDOBQntHaoJpG9TuVJ7uYWe07zCfhgOsVa0vQBsSbGAP/znj0 eVNR7xLJXI4rHwmopL5y3Y6Dnvieby7RNS+Nq6YsxvKUet/JUC+FNKNlBD/UTC9+bZ+9ZAhDx4Y0 ZUrS8oc3oDMzirl/pCX+QmDqiidVgyNBeg4V2vd0HhwBuvIkAi90M8UkGV7ceppvUOIVtdGyYIMF F74CPrF+9bvbC113sg7ikFP00/AoeJfYQ/Ri2C9GP4M1EuEmch0+eTC9hlK4nifTXRb3QIcSfofO QfOcts7fnA44zf0Ws+Rmn9QxEzswdQx9hi8kV/ZRqVEAJ5ieNu1e3wb7xpyG8pfcUbUfuAkTse9L mr60mJBFVPt46NmYocpKiT2tLu0JBTy5lex+WV7/pCRzGOxExGRDmaX/dYvl+a8OCIyTKcVUUgnL zEp1yZOrmT6AXu1PtW/MWBlZ68/8H7blGwMsUzIjXYxPhrRu9sc9R3uQWjWlBaI8SdyzIPXmcDAq 9IoyAe+Vz6YZgt4vBeUFouAXDqwbCX7sZA88x7aqXxjfmS63hb85q0duQISEbI4ezIiZ8VKMYi5X vD3ZNLt4FuG9j4bwo+QLV5ZsbBKJR490Sd1gxsqTVYVaeqwyVzb4aq0Oq3x0Tr3+Yrpm9wATWCw5 LNM3N8QT6bn6I9jfMuhIIQs49JH0rZW+sYHkYUK4e+IqxM4jatLg7lW+G1xgDQy+TKlIBLBtj44y SWdc4iuxPY9PXoL8TMqslLWAeCV/jeZj//49tUaSUiYqLeZB73OhCCrK1S3fjkcEuRJd4edTb6Ry ePDTyIl0ktSjMZWc/rwWPeFXzgkNqTdfRipfYCeOJezC9HqBjHKWnNV4mAT4jDbfqqM9kimTR8Di ObA4kY6/0t/uNiCi+JH1rE4zrU7Rfpl3tVUX3xl35XZGkWcEJQlygHnLO38NY/ZyYHcRizRz+l73 ASATEkqXRVa/kEbLDuEPfKOpn4cpf/uiym/g8haVYLgk5zjejLeSN18LkjQxKcZcHxBkEtX0If5Y VaCY4wrcbH1KbvFDd5lWmMY7IhHo2htIshQdPOeAPl39SMWk/ViqijxSlB/12WnQyRgDukVLmdWI snVIvjvtERLVrPsiIuDL3gR8/eJInu5xEu5iPCJyE0sxSOol9rDjXvh1SMII3TDcxXUya7v3ajMy VFcuHSPcSEpjB1zJmcxURzNsNKYTf+bJk5i5JKUWxH6urG0MvLfv5mGRbuE/qGVB9wrxMJAhX42Y Fw7rQ4vj6VFZvBlfx57lkVbvrBxVYvbP2EtFtCX1UkwImxjtNE34eFh3Ub9b0l3DeZimwKNzVv71 k2bKaW2EaOfmuVMhccf29rqJWHomw2er7Hksh8CYVoJYvJcbnAiBMF18YTRlFISxdMIXmLe6PM7B I5Kj91nIeyvK4U51vbzA/IdA4Ff00EyObZIZlxvNmn0BSb4+4jsoTwwLIVV8wm5T8MqARZzNSwdC f3DyVzHoBA2abssDvJz3RiE83cFOg19erIkZRZeuY9D144Js7TODnmvnQS7TQXIWeum5UtXoFLyb P1wth3P52ImHlZtqhQRONOAUmR+Gu+oz5GfSZx7o0Pjxltg/WDCJkXaWcqn5aqMeKVM6b9qENO6c IlU0OSX1NJwcI0Xw8hcxgsnJXZNXwoZ4HIomphxe1j9OKaPSL3wTp/uSIRQHVBjmxpfib9CLJfu4 PP1AY5V78bTKQNAlR+oNAeWLRQVmpuIN4bGy+a42JtGPPjjBgYDmWdIrwJqqBs28p3hvQ6kE+0Yp +f5Z8SJpCulGbg0IY7cFiZ2TUZTxtpeQr0h025acNp+evnRyS7prgxu9E3tvDsx79XxFEjcHeBnX jPwxd16ilVW9RqL7FCj5hGHNu+euHGFImJ1RpfNCT95A42yNKbqVXQWxuV5IbN9+84tv6tu5o7WP 69H9vJXGC/naQh8WN44erPiaJ5qONaa4SjvfT+CsRce65wZsKOxDxsPkevsvmmohvGZLmHP7zJLY pcEjVY7qrAZwQhk0XUoLUOXoXPu+tMTqiSkVNdktJkgw+AJ6gPakooF0RO9eTNUW4wwDvTguiW9m SipR0sWa8mOnYMn9fgLTfSKSpSJK7PxsNJTmYwfoQm7ZhDX+kcgiRZ78Lrx11/oJz5/O2Uy3wfEj QnXIoyNuEGtijq7n11vv3gT9TwUJ+2R4o7KoLkI0l1TCDUbuXolUtdpzG4VkeQOr4znxb6HTTPVX JNrzTwO0VCDI8NIvt1+WJZc2/Lk7+rAfE+8b+HItZc54Jne0W9BwEi7GTK98R+PruIOkQK51bJf2 y1aV+1lI5UCiAXbuDvsYEjPl0mc0apeBMXY3RMNYq1QsYHL9vl+bVfZU1nfySENVU9d2q/dnlAaB 9oLTH6AsZpmKD800zVW98Iejlyv7blAWERvXfaaf5jc+SJArCJ7itXQh/qANkkFok4EGg2pvlfBC qVvQ4O2/H2xLUPfNlWkiG4Ugp9cJW0DzOwakJ9/z0l1AY3CNUx7N0EFXiibNvVwt0RnHMMzutueL BwCDXLPsMgrLrB63sLdI9Qt35S6OkUbdY+UA9k4yXB/mvYfazNUP4csk9ihoIPKhUCrPpTaGPZwI doM1c/dfHsymiljvT5fgUGjqVSn9tRF6DSw8ksaqn/FXj+uPKlW13PPzidEQVL3pZp6HWRg/5AL7 G/MF1oLaVlBzusW7Lum/m4YifHzemydFkjujOXgYTjK78W6WyWQ9lCQsWUNbASnOeIsu1ozLKzdO ealsTgtdFNmrk7uMJ1bJkeuH5TreXyoQL/dCzKeMwe7+bNowLRZkeK+6G9LP/Fh3SNxmF3chwgjF VafgEmoB/5HUJggcIGDLDygf5/uhA5e7bFElMMlyrffJb7Z9p93eZVxN2eeR7+SCeGJy962C1t44 ZEOmU+6FxbWQzUW7t/lNxRMqca/GO4D0zC1urXxIwtNakll3grkdpMRZ2JKPt9NIIdEPlgQLiiwB lk7HuIEDrSJyVhWIow8O6+zlBV3grz0FXfw1ggw/E0sac1OX2x/0djKM+ck+JA9SVCuPfxOFpzg7 qPWH93z5IKXV1dSgfkw5K9QqoABRfZ1LuR8Tu4JqWIRHWiUsclAyPH9wr8jkrSFGpPKZ2jQXhjII FP7DcA6qiwvujVLpy2c0AxAluZE7o2G3vpYgqFkmjAhKsvC98gwCEMhw3IV46J1hj9rlr1JMR7sr 28+s0EDzNCSVeRqiEFDW6l7R1069bC7gXlChBf3m7Xxm02eVbfXeqcEsLEnoKc5hC7ML9xfpDRnt Sbvg7z2a80Wk2lw4A3V8JBqQtbjSGrdtRMp7AHty/1808NNo4+IUDt21eINFwe4pb0qkCev96iI8 ackivOhP5Um34hBEhqSnMT/PW24og2ySu44SBLVdoPOj4W+MK6RLn5vexVuq49UNtgWcIFwnIkFF u80KLywzKaIk7XtG6VtWS+IkSZb/+JhrxBMDOxPohCzJ7KqiHOkiavNWw0H0ORqVTJayLClVDEMX Og+ZrPa+fq0mx68YcLcZ+T6ZhwzzLGeXIeYNfGM66bqfkn9wv5rPBWTCGpq7HM0KjqIkRDS89Ln8 FxQyb5g2wYJMBL7bO0ZZ2EGpZcq1t5Ox6Ntwj7VEbll5T5HUjKZvcs4W2BtSkvFRZ4UQvR/3ZtEb vBrongVNihdSUqe45YpVs6XL0jz1GVZnfkU0oF0HBdQJb2n2iWAvgSuqNz8HUc/pgyOLxYEGB+b0 DBdNsb49jUrOmmCGb2OeO8MTAY+rI9IVbrWHpDJEKHcBHWM6wpG2ONRMSS0nFOOjlP2MBh7O7w5j 3T1oFbLk0Z8vl+AoJm1uiKQnb8F1y5qui3xv3YYjxDw3TRd0P2U7xkAansHNkdQnHLnCCUXdw7Si YE2VJtKl3mOFIhy4PN1JuvUfpZ1+JqCVGJYmIW8c2f58Ol351qGqn6Oj8IGFY+WHamp8aHYbve/8 l7J0MfbXv2R55LR0KySsgxX35Erf18FoHV+GTNPbcXC28OwGH5Z6Ej7L2+PSOB8juNVhp9QpCh6b 7PYdJY6FMjZ0USspvIULcV6hlCzVsRZFWS8YxJBzQmxHBKEELN/BpXBP4Acd2cbhLIqpuYu/G1Ql zsxhBWPfv3Fx7r+H6qMlrZnYG2Qt9VNWunNzU11B0FznwbcEo1a1mY2Y7ZN6F0XB1K1iKJnyX8LD qmsh3i/xNIU/6N4yX039WGjMJF8jQneCuSa2MjIRZXVh2jkkmNNHElOU3kKtNiOpwTnVqCBEGBgZ nhYnVbdVFaeufIo/dcxZ6YWfbuHvCAZPyGHmv+37OZl45yrQiEa8IQeciYD1QXw6awv4uPlBxVH2 2KCq+eX0xc19Iuh8DCWIZj7sPFe3FkfWtLp27hJRxm9cUAAf6h1j3RTeFfIWlfRqGvJRjR2O3Jtk l+qPIiLHmcBbPxQxq3igXeVHfJKCTYil1FylbiGsJDjGXqNhFhLYaxRaemKjD5ByPL/KkuyvIa/u yWRSnj9uzTLhLTnWfUWxGqGRUL+2VHrZ8WxK/EBg/i1FpaGYyCOfo5jqoM/0pwMriaqqle4ieSrj cKVFxsGewnklq2LJY8v+XTOsTj9n1Wf8YUbB0wcRrI5d5KlkRlwOgK64j7y6LBv6k5f7iFjiDJc4 0zteqA3rGM9znL5hkpBMoWnMzopxPHQLPF4nDKh3FbGKSseZkcnxVV4Ujj+esua7mnBR3J3EUqJX SelpMgMzJNEJltiNftgoRKGvcz/0fpoG1LYQ9m68KWLRvcKX8IFIZSIqhPbJe3Ts6Duz/n5r3uig KsNlmSge7V+W54jkpQtqheZY6aAjDXbxObyq1VZS20f1dSWRHQtHYoBMi1U8vYYQwDPcXm1cJA/F v9Q5RgWtOeruPSK5yvZe9svLIi9niMfkPZgY9xNaFNvLfuL5VhIIw0viisog5am9hkzXn4mup6G0 rx11lHu0WQSXh3I8vJF9F8t1CfMzS8YX/yOiEbMT0iG51DTQQUghcw97RvrYHgMaAktri5nZON3T paSwzAu7mHF9GOCctSz0q84zS3wsvXttOXWmCIWGzEHsJOuH4pCPHsxjUFIUwS+ZO5PbYFmor14F xHF1KjQ91BXwWGu+KR19/crn+ephti/Ffd69Wo22LVF9p5YkYYkvcXqGkAfZaYvDiJgHUCrXEm8U 0aFeWBNIp6qXYWpdUdDQki2ql4vJFRl9zsS79LYNWWYTYuwIYh/LdSJHHar7nB/f8SVvig4RbwZy DaEbwIGbNOoWSOU/F2Qq0DFLZty1Pk+rQpP2JkOQLBmWKlAtxPU+4hmiUyVJo+R287ItEM0ZeQ8d Ez1GpAxw4HUc7nG/NaSxxUe1TenXzTLmVRz0+dSTfjmxpIYqH98zxJ3byYvzvFIHHtBLG2/DcE7R LWQrK+drKVBwEgKPDxvFjxxpuht3dh9SGMTgmSZiz/eJzZxpcn4PL+00GH4raIxHFhkQZYuQzBFP NHTIk6vX9n8A4BX6XwplbmRzdHJlYW0KZW5kb2JqCjExOSAwIG9iaiA8PAovVHlwZSAvRm9udERl c2NyaXB0b3IKL0ZvbnROYW1lIC9QWUFDSEcrQ01CWDEyCi9GbGFncyA0Ci9Gb250QkJveCBbLTUz IC0yNTEgMTEzOSA3NTBdCi9Bc2NlbnQgNjk0Ci9DYXBIZWlnaHQgNjg2Ci9EZXNjZW50IC0xOTQK L0l0YWxpY0FuZ2xlIDAKL1N0ZW1WIDEwOQovWEhlaWdodCA0NDQKL0NoYXJTZXQgKC9BL0IvRC9F L0YvRy9JL0wvTS9OL1AvUy9UL2EvYi9jL2QvZS9mL2ZpL2ZpdmUvZm91ci9nL2gvaS9qL2wvbS9u L28vb25lL3AvcGVyaW9kL3F1b3RlZGJsbGVmdC9xdW90ZWRibHJpZ2h0L3Ivcy90L3RocmVlL3R3 by91L3YveSkKL0ZvbnRGaWxlIDExOCAwIFIKPj4gZW5kb2JqCjEyMCAwIG9iaiA8PAovTGVuZ3Ro MSAxOTA3Ci9MZW5ndGgyIDEyNTI0Ci9MZW5ndGgzIDAKL0xlbmd0aCAxMzY5OSAgICAgCi9GaWx0 ZXIgL0ZsYXRlRGVjb2RlCj4+CnN0cmVhbQp42o21BVScaRKojQR3D964uwYI7hbcrYEGGnd3d3cC BHcI7i4huLs7gQS3YJeZ3Z2Z3f8/597T53R/T1W9ZW/V11RkSqpMIqZ2xiBJO1tnJjZmVj6AmIKY qhgbK4CVlYOZlZUdkYpKDexsDfpLgUilAXJ0AtvZ8v3DRMwRBHR+k4kDnd8sFexsAbIu1gA2DgAb Nx8bDx8rK4CdlfXDfwztHPkA4kBXsClAgRkga2cLckKkErOz93AEm1s4vwX6zyOA1oQOwPbhAw/j n8cBIjYgR7AJ0BagAHS2ANm8RTQBWgNU7UzAIGeP/3JBK2Dh7GzPx8Li5ubGDLRxYrZzNBekYwS4 gZ0tACogJ5CjK8gU8EfRAEWgDeg/tTEjUgHULMBO/9Ko2pk5uwEdQYA3gTXYBGTr9HbGxdYU5Ah4 Cw9QlZEHfLIH2f7LWP5fBoyAf3cHwMbM9pe7f5/+wxHY9s/DQBMTOxt7oK0H2NYcYAa2BgE+Scoz O7s7MwKAtqZ/GAKtnezezgNdgWBroPGbwZ+5AwGSIsoA4FuJ/y7QycQRbO/sxOwEtv6jSJY/3Lz1 WcLWVMzOxgZk6+yE+Ed+4mBHkMlb4z1Y/nO/VrZ2brZef6EZ2NbU7I9CTF3sWdRtwQ4uIBnxfxu9 iRD/lpmDnAFcrLw8nKzsAJADAORuYsHyRwg1D3vQn0q2P8RvVfh42dvZA8zeCgH5gM1Abz+IXk5A VxDA2dEF5OP1T8V/EyIbG8AUbOIMMAaZg20R//b+JgaZ/YvfRsAR7A7QZX2bQDYA6x+fv57034bM 1M7W2uNv8z9vmUVNTEFcW4rhPzX/pRUVtXMHeLFxApjYuVgBbKw8PACetwef//bzVwf+U/2fUiUg +N/Z/cOhjK2ZHeCPCH9U8da+/1Ti+u/hoP336tAB/juEot3bTIMAtH+vgB4rF6vJ2xfb//Mi/Hnk /2/+//Dyf1+B/01J0sXa+k8D2n9b/H8MgDZga49/m7wNtYvz24Io2L2tie3/mmqC/rXWCiBTsIvN /2plnIFviyJia279Vy/BTpJgd5CpEtjZxOJfk/Sfq3hzbw22BSnZOYH/ePEAmNhYWf9H97Z6JlZv Lxentwv7lwro9LaHzn/e7R8Metu0/05BwtbEzvSPlWTn4gYAHR2BHohv8/BGXG/j8ba7piD3Pwce wMJsa+f8dgTwVq4PwMzOEfGPW+bmBrCI/iH6F/EAWMT+Jl4Ai/hfxMMKYJH8m9gALFJ/Ee+bTulv 4gCwqP5NnG9T/DdxAVjU/6a3eJp/0Yc3Av5NHwAsJn/RH71iMf0HvsUH/QPZASxmfyHXWwJmYNd/ 6t8k5v/At5ws/oFvSYH/gW89sfwHvqVl9Q9864r1P/AtTZu/8W2DWGz/gW9p2v2FnG+2b/8d/1C/ 5eH4D3zLw+kf+JaH899FvXl2drP7h/otL5d/4Ftern8j+1tkj78jvyk9QY7/Ov1fA2Ti4uj4NmF/ rv3bdP2H/3zBg0DuIBPE5QU7E/5gy7rgjvuvIoRuTPsTAjBn6fda7EwThQbwzoMSM4bbiarZn1fk yiWXB9gkDSy7FUUd7nM2Fq+89upJGzw4b5lIJY/MSY3jF15vIeeSvO6IyBfQWiDyNFNFSfhKHAch lIij0HsQhEzNB1Wp0Lxp6vqXktyosUsleWRDOhS7er6WfpbHJfqgcbqj4txqPKy2z3X9niZ3xyYu aDdaLyJNMZNcTqDrATHRrQZ969vI8BrGAHFvVICsa+D+OWWoM4YE/zCROCdcjcZaPupTbonrSSam l7EUlvCJ6WA5I9a+N4gWQK+xqTH8yKc3IEQXIpHxs/AatJk+0vAuATjDDBHqiVPxzli5JXGDnioV UwJa9nzk0CJEUsf/8NAjk7EF6yZ3uTLK1I23dN+NT0pmCzkPL9iTT4uTFnIEt3n5XZRtk8hZPr4i NL80q4VInV3AcT5qNZgZSEh318/7QwE4TLx5yEk0IMxUtFl8dXXsS9eLC+8JdH1OnqovLCVYtZTs 2m6WdLvF1ed7lZcwRE/+fsD/Le9gvCy+MdtEgBYnV8NzjasjWGjkd4aAl6qa6dn4PcVijA4QvnKq zKruNjqnb7sH/SnYBLvmVCcI60y2ck4g2imq4Xv/1aUAuY5psNeneZbbi1G4SJtA1yj+ElOR9HQL DAnDuGcbzIC4Sk96iBpqKrOCzCCCGGZxRyhsq2j0Y+s7668NcZB8ScMQklrwSuPy7Sua3VzHVGyZ st7QE73ffMkT6MraqD+QScMzTh99ydAPzib4PiWZ7lmnXCHhJ1S0r3GRxziGZiiUwuF13ywwYrF6 TsaF+4yRwXwtzPDNhINQNp1ocehX3zHZ1u7m5saldPo4eopbxAX4qb+n65chFvIk470asKI9KhDX oiyOY23Lf7+Gk/6lAeVsto3WjLPmjjczxxN/zd5DIfkkO/naxeVrqmcw+mioRZy23vCeme9Dyvtd zZt9ZXNVcilLqbOhCtrzNfTzX96fLOyf5c25x3xM1tfswtO3c7FDbcXZYQ/KQC1gTpEICp5Q7YRr y/16Yt6Hmygp3gvflyypi5b2vjZOQcsDttamKs4dVRYTbeforDW23M+m+XfGjKwIljecPSVzZysp 0wdLoRtiHt+ZwqiWtAo0FevY4dyUV1VePMQ+SQQbK7gaRpnomma4O+dXnSFThMLzf+7G7kAIreBF G0EXnowqTF8clgCUP+9PhHmik6s16Ns7axky7Pdb7cklBiSkdR8xFEY45a4mrmt+S51cl6lRuIjH 0dRqVWk9KunIoedXQXYGBsKUjo3PHmsacRpv/d7bfD/9ZPgy142t1H4PyWLUOOx9eoR1TXJX7Zr2 BPHK2QIP4lQn0iAhscjjVaHxXhgAPYRRShBEowpYjH7ZYc2KjR4CZBDhu6cGCGHm2bsp4C9+aFit mQ//EQgOlZgU/ZbmwomGTUbKd3gQBzOQkr7+K9gwjQvht3pX6IniHQH2zkfZbGSlaiQm9gvzMs6g Q/x1p+XAgyfP2JAfcDRnd9P5vgqcNXMJrhK800pDh2ryaiWyHyGrPYikdRu62xQsLceN5SPvAhwf EYI2hcJIU9HVqcIG8akKHo0Sovh3WSUHADbZz4kP65kcyyMJZbMoUVL6t2WGjjFNhrs/SeJ/Nuje uKIgxdHcvFaNgj12UNS9iQGxomSiE9eYorboeHEzPIzeSHqpl0lik6/vY4YQFnPHMBiGaaGRnwHZ qpGX52eEidz+s/KWZJGjoXl3U30Ioe3pBVrXiGJJO65QQQGchYTpttpJMDsekGfSBNJeU7T3/EIK GwQF1kzvl9jNq6aDs17lbU1boijfg2+ruvLJhn8U446cS3E7+hhrTLRQ/1bfyR90JNevXojBhClU WB65NQojNOxPEYCHminS2v75kyeDBLZHpaLbh0V2ijPPfjNsWCFV0sQ1WlZkuw/z182yEG68unzQ 0HxJcszhgsRhNbPpHIY5sX3+U1BatpMoaE8lPnYlkVvb+sKwMoq6NNHM6ifavlWIUXVETUhWZv34 5NT3diEj79mkOZYAvq1ybzuLl8pF2PQGWzF3aF8zAdeGfbjQ87jdxTxZbjvkyWFDG1Vt20kn7oYB 5RH+D1b4t5DHHVuTZLeFSRy5gbOjK/qGP5GYRuGPvjffZy5iqSiiVfhf2J+GldFzu66yai1fOLm+ mxyb7dNpt3Uly5UqCksYLUJ8khMp/8RQZVRIzPThafgMEwkN3ExrqQJTSc84eZOA95G0AdJZmGA8 B46yoxq10InEohTsijRPmSbihwi4ONf0VmxiGXjnOgjv89qOR7oT8MWsd1dUH0ddih5ATKVqYS6Z L1eH6ILXvENEPJ5LiVgZp9pJaFEwLViNRX1KB0tKhSLNPnX9W1H/yF5qoe3qa7OND8kZRd/d1ND1 kle4cx5lAnIj6qvnUl7IxwxdaO/rHf7uFUiyiVIg9iGT+M5njmUUDc/HjY5doqOV8DB5TnjMpU5f ko17v4O0Qtos+fB5Wr5CK3JpPKUCt3IuTVctZPx6MplFbgLuRbwKN5bLGWtcc0s8ADCe4XOwOh9q 3ztOw9pbCbHTsNeeBH6N997fCh9My9ssRedTF9pN5XS7P5qhFBbRaLuUbGhO3zIaccvDIWIUKPHR lH+1fAep0IOl/1wjZI7luTVrmYpckjvT/lAE/ZaJAvpWbv9PFkXlbzt0pviCJN7O6uq2WkAZFw/n tCFv5XcBzZsxj0L6GGTS1FafquzHrC9phRXnFM2dOfxzdQWmWx9DAQPYhJumlzKoWIL1X6djHKlG ttQunl9OrrHIvkdRmh3hRz3Vl7E7xIa4PA67DQ3c13QQ17vAEPaqRMLRz66BMOK5jXBuIumRH8K/ KQbbQMCL1lV4mFW5Ml+5MPzaXSZ/LMD1Bggxa42T51nT/BglzJn31d+DwGdxDJi7D83W12exR4NP RzGaanhcs8c0GpVlIPe5SblrpF9IIyIuOZCO5aRCekTWMN7eyqFcd3voSC2Gm1oIUsGbR7wRa2hK dcRTt6hQF7h+v92ni9dYDM75dijDKryB/sUmkp63Mqh71NDsieUihkMBe38akZ5UvrrChDt9oLPa x3a+jlKVOCUwWiJB/fxWbgzZcMUOezqUfGb3/c67PkTD+XcQmO8gXknIpuKyprREbmDTTL6Vp4Xe fM5ejwjA/eBAyjseaBWahH4tcfs7WnSCEVX2yMRaGTmvcl09uDzFpM1JrKiiSqIDhg+KDiwijXSh AtdlH0FpmEskxvReKsMBcrcRR/TxJ8wkUPSUhXyVLk4KeccYczOaGesVSbCFhCbwA9Nz/PZE1PMK z+JETuF2sl+EYS7rRxrhC5DBwcSG7aJpTvCGXHjkbtRVrYJiusDRCyGzYVi3EqhGtS2ue16B46b1 fKjJ7uuHTZ0VNnaxTwz4/J2Hhz0VCcciK0tmKRmGwrjGwT5qw8bw2As0Jc+TT2GKYYKnjMrXR19v EN0MR6bHvnFYb5ep0uljC05j5ZGUVzQKtc9YMSKzjByF82NELje4JOvOIbMU3EpGVRRBoi1pyeYO c244jjL0Pd1CWSHseo2cG0NSbSBXGf4Qe/ZbInOzvVtC8wslN3hh4Km1IXKXYle2S1pH5COLrFE1 u5E1q9F27QjczSgC0oaThxQa6fWdNYfWyzBGcgQvqi5dLarHFfb1e6vtT8B3++WPWB2p9mqfWBoH YAXlSAlCdH8HGgzYo83i4cMaNxx/H+X9iIP8YhuKZx7RTd2iUyFeBrj/EVeL5EHoSwwe/0Q6UCoF t4G8hSHtsIbpmsK2VRDrL0dOTP2k31944TQ7HottnnDM2dqvAF0okTfxbLe/sN23x7spoqUvXvuy x24M7VcxW/b7+EGkY/eJsqNSXrdX7pal3eTLlxqGHhKM9Gk3RD/P7PnJ2cWYQagSaLIxXJI5rLDy MpglMOlv/Rz0CXIsCIPoWzK65XTUpa+V7FUSTnZJYVINP3HGHR/yJrk3BSkNoccb2N9aK4xTpF9d DLDASkJYPUwNsDYPf8c6j2Zsp/rcpzy3I10nmCYtLTPOqsUCJptaMcIIDULrMlQKJv4sz3YamXiD jv5JMAZEyUDXP2jYoZnriTWd1K7xxDjhiW46/t3kG2m6uiKHik5+vKq7MGfmBKQyRcewk1fn3dYq fsrXW8alGEGoPtNZOaDTJ2S37Aij7jKuFy8QcoPLQT/ZrB0n6qHPgUNHYJ2FrGEO1gfUFlbt87rq d/auU2bOoA9MWBZA1p8eq8gzyx8qsfKZ/FMKzvQYZ8OEpS+ZV0LjFKkZzY3r+foV30WJMF4XxmOJ KrsPk0GXnDnnfm6LzRfsJi/XmUXGGL2if4yeyaKAd6mFgz89OMZNI262T59wCetQFLViru7hfWkM /6j4XbxsKn/ig3mWHZx6ymKVeYgNcDDArhcDtpqQGBvpRcyWIHb3sMYcE9Q48J2s0up5xjqsT9q5 uYT5oKteyrkTTZYhpI6bUuymVxFYJOxAvi+puDanl/7TvDwJdjf193rDw13YEAvYTzjekSUgLHGz 7HTXuCG8bmxNaOg5vER9GlZrxvakM44RUfi7Ws6uQqKKW2cqBhGqUxdMLshiseVri5VgD7nMQ9aq 9B5zRn7e9p6BzD1DkHStueokfOC8z+CvjZClYfcqVO2kqN13MDgMVSe6rhIDHlMTy+J0VNadyNin d3sJhaSRBmIy6wwHlRCvLRe+pp0sXjNdGxvqKBH6mqEnbMqTSp9pxb4c3mNCksMdKo9xldoOXg3M LjaUT8vbfDUqoAYeLBVbInzPCtM7LNnPevYpfGpqOEKDOgT/dA9y+g3Zqg7Lb6xWVReFrHh/kMC1 nBoXds0wUuaVJKjgDeEPd2VCcCN5cVHGIFrlfdibBpWIQazaROW3Ykp69duu5xiJ65SUnjwxz2nR tbXS5qcimIvIAdHGLnF1YzLb0jW1IeYhcOOV3Bt2QP4nLFkQ0hfoL6mHWzyYFTrsUANQBxFN/Spq iXHHGj/VyePi+KnuhByi8AhvZvcRj2FwTJP3G7a3hO5MDpv3HGSwYw9EFUcl/K1TIMSiBVK5IW2b bVm//w7F+I6kJhuH0Ts4UCrJ6ONfua8m4jA1sDz6LHI5D1OzUa+8B8UVIcVtoi+KtXQXn+FZ3C/a lGP84Ssp10WK412jXl2eIMtjm3z5uNJjFnzdg+gzVdOMasW4+otFtnf8sQHtzJapbD4LH/QqsfAD eKLtQXFOl1uKhb+HHXp3cZ59VEOa88R8KXLdpE+Q1mVZhRKUi5kUVr7Hl9nl/3hTxRFEHB/QRODq g/X5kS6K5dY+sGPSCmu60LE68jyiE0Mg6stAyNjwAbmVfGW/09iO7u8w4nkSmQI07KZV4dAaD9BJ xFNDIX9ia1U5rDPS10RmnFHMaQBz3SRIJfGYQ5oghSunH10G9RomUPTwZV9fglm+vAT4Mx+xYhuA tvK0sXf2sfaj/3ni0NI8kGEAy7FL38sFtoj6A5bu4nVZ96gA0nUgt+7yPC8PoDJKAQ0mBC75lv0j mt7FTsU4jlnQuj+P3slG8ClghKDmxL4tzY1XdfDCWUTFEKXwqowdUVvZIfB3Sb1sdePKZLlwAy+k KGVWiFGtiPewgIp+KjU0R2n30sUMeduOZLguCK2jZRh7sblc90VtLUbVEnEUgc4D32zHdOtVm5Sv cy7b6g6HaeIQ4VO5W0Fa4KHW1xMv85+ueanFzzGiA2cVBWSGy/HOjrA9lWnGkBY2qq4VrWM3G4dz OaPO7tfcD6xqvd3THxN8DNNoP4J3BaPo8hASkL9J8amN6qt6hUje1yIuuMPWkvzgFi4J9fgBPZRH YOrVWIxSTX7SXEn/hKbRDsz/6dV711GRqTZATaJ91Mw2oC7CqJIb+vCM+4NGTcskQAfb9TPCIWBN QLMfqmTwtR6IFdH463ib0XMAg5cCmNkEo/DwaVpVKGgft+Sy3th8sGPJvD8movuo/ylEKHkEUjm+ BpEtmab4OjF/WRwOF68w+53srFW5dnWaoHv+IxwXolc+BJAiHWdGlD32k/x9piA3j716XC87BT+/ XsSPjzst6AnwU1CNmi0sxsX+HdRtAtg+Ff5QSJfM6xg04qnKuAasrncG+10/6VgxlpLqZHA2rXdW ajkgt/ujc0XJd7Db3I/muzEJmyKMvxPoKE/UQkyua8fy6LU+ZrwfqQpRlHzMq+jOTFUaPvFyCesK afcyT0BzQ85m/o5tZxRq9ts+y39fEMo9aTkbNj+F0vJ2VoMsOK1VGUAFW016OtEbgs9otPGxYdVh Gq3gORt/+JtANy36wxAbseAdofXmlxy9mMgO4AsteYMR77WWBQr04f2EtauP2HbDqFdlQsc9SsdE jhLwUecospzOWyB6MrWXpV/m5asX9xTBT5G0OasTG8uZyUUxZRU+Mu1zhPxCAEe19wsbJ3Gj1nTP 1kHZnVHvPGdgxMHM4zzSlbvRIznk7KgaTOZ6OrUco7wy0jRW8J0EmcsYhmdoPiYsAyuP9pecQ9xB OZWy28bmLzaeFOqmA1o/sYDERiy6FP4Jp2qMOtIWOilTOQqxup/ctWO9RM3uj5vff04Sc8VZq1cz 1ba+RiiY75/hiKL5VshwZm4uhgUe17V2XSh72eDn/33JZl3u6SRwp81UZvbjOZZdImExZrigEUor yy7u5eIBQkuox66nKrRD6t3wDeyVYP4lF5RvENddFbln0gPd3ZxDabRSBWxE76dmLfkja36Ej1Ah tuluMjOmabeBnx9LdyrDCtbhhfuT+wzWppVv6y0VC6W+fNdL9jbH1SvRp7hsVh4yJ0yS1W+CudY/ fT9zK5V9wV3c+Gn3N/PH3JvDhN0oqunqs82ZL5Dc41yLn2Xsv43aPtFQ8mFsYMEb25CW9nbB9nXO Rpz+CCtzQ5vyReIjLk4dalpMMqW48rWG9pN0jhBoH8zH3bobnEu/8xmcdnNrNawCp9BbBN8P5Seb 0F2U6dF/682Qynh5Yr0QNoQijZv21Yuv72Ob5LdiXeA3ozWl/5KCkOWYXem63oZmKxJsktoffWWK ospe3FBk253c1cKUWRrHqi5MKSoPZkI41xzNyC0hWEtdqE3ZLCk0yWvD6GUzxIZbjnRvsqmBqRwr bomjrbEqiKROmHf8Ie1vz0TemFA+EYvdiw9lzG6zRCCZnrTl7Mtpz6GR4woG/KbFMbvTo+Sj9v6m Df8IP+Zp14MXhM3Kqi0Se8k8IgUh8uGrFcOcNnaRZJadYHPb7j68fqE102ei4nxfpMJ5wx1ekoM0 vHqxgCeIO+wg+M7rmQmusm/kcvvq4mvDJfeEp3MFaTHirtXcMJoaiM3lfWkIjX2uOe03WB6zDyEf fmMQ+UbfLLWq2IaKzAi+uo8aJAy184Wh2GgKZ92RT6VD0cbPn7m2Ln1n+0pn7ttMIYfAY2Xxm8qt 07L4w02Po/j06U4A/GcaVqHihX5xmiWxw4t9wNb5DTLt6xXflWab+GzCzyMV35BcMtxU0aRJ/TiL YFtB1dA8zRAtehvl4VTtpYra3DJpcQal+2Zyue8L5Mh43dmfJbgXifeCfd9nZT5Jw5LeiKOaOtsT el+TrHjVd9MdwctLJKncY3++Ci6d2N+Z8tWT/d58I31KScBCotR0eU0O4NyxFa2elqUzayuanRPW 2vX75pEBkWYz9FvjhvAFH3wgRJ2tGn/99rfbabFucDgcqFpJvIODU0LCCIiW7pTFep97hCsSxEQ6 u1JC5+PmG4m+mkh0Tq2vrGmTvCI4OYCYT8ESmP5ZkUta8NdnaolQPWiN2cm+lsPxFZlQCue4Yl4Y kV7WQLl9EJfp+qW7SA3bcb3gJW92jQW9SC0uwQD89rAgqrSCmDEJ2AdD9blNrZ2INZ3AXdpgJQAv dvfCJVmh9CyEvqMOJfB7R+tQkiGiHxN7dcUyJpAIj/RGapJDVPS26/fxr9EM1MEva9JT7wq+eR0F +Q4M4/eY6Q/TPWOzpUQQdSzu27/QSpN12X5krit9GYGDiZRXN3pEHTk5qNeTYlTemMU22v3ttmG4 c2XVWT0vo+JsUVZ65YQ5r+Bl7hoZ0BgNm35tUSb+sQ6FrLmM5ucHFiRXncGUaavJsQQjTmhRSbF1 qvyJ7kjUseSmhQVqhqw+jy+8pwb0Ia9m4Yeuxlk1YxeBLm0VJjmzOilzL2GdBNDwpu7Vw9vs/NHO fBLinCOd/AqAXWJKZpFYYUBNVVby4qx1duVdIr4GjXkbwsAqak6NZnLxVsGCK9UmDedj85DiTO7A 5PUO+Gq35+pZR8yukf9gb74PlZJXMTKwjX1cSzTcWx4gB8lO8/08EQMdXW8H5jMGf/vLsgNmmjST qP0V3M/3g1/84VSkGppGp3h1ON3vJdgOf6WzvDRQIAQ+dqaH0V7rQ5h8RqObsEocOMJnQNO3eAcj QsQ4oq5dG0/cYY1OAsSDH0Rgglz106berNjyoAAUm2WMtXd4mQpLSony9f9mh98yHYlzlsmnxNtB zTyUC4ao5vIsbCeBEi7MF4ntsBGC9d/zHBnRPm57x4CJYtRMfmt3BWwLg4KgZlBQKqMDuw+Ur59G tMtaHeZbhewlomY4PsDzJruHIjYJBUr1nUhatV3T1+3Urn3MBAVyHYRWJhWhymSxC1jLR3gp9j/U johq2n77KRm9dkEDTT0tPLz55aDvBDeSaWgqYkbCDQ8N5AMTtXK/dvnA6a9aFTYUGt0xqlCKQTYy C8HW2G2UF4xWNubm5oBKTMrYWeOp1OyKHxLnD6v8NO5Tj0J4eDwZVg8W5nOOpNvosK16sMxVHwnJ t/9AsQ/8nPuNV1lmwPNh4lQrzAHBAe5uEvf9tyO7yBiNSh488Y9kvzGcCzw2bEqotb7pyaT7bggf d2pOAXlvThu0Kb4snFoNjtMsoWUMKKXm0Pa5hNic7Oa/lj4Ul1mT0Wd2pjXwZyRLkh2TtbVczDMZ BsGGwjSuSqZSNlLACH9W1aFXYqA+kXfU+sJQdJZgZclc06SMqwtakUxfEJa9aDx86M4cBsS0Kyiz Hf8GyUlHLIKVZ9DqO1aFkq/yWc01KGD12bYQTTe71sn4/fydbpZ5ZK8bSNMdiQ2id7FC9LJy+PRT 0JzzGGd0YF2L8b1fTDR0leSx8rT2BNou1lbS43uw4T9iEC8VM2FHFjIUGv5cNQlq2Liu+7GbXYo5 GxK/1e85TJNJOUP3/fMw90yk6sRcCPEKPu5mbgLERr/aChzwSIuRdhcpO4OY/GRBaeLbxYz8FVoL mfL8wS8TtmJ6j+3Q8r21nbyIywP7GavIwHZK2iklbvgzADsSQ/bkCleUTQgxdSdD4EAz+9gkD9ZS sJ1/YvgH0k2oiOlIVq10El9XDChBaioNzY77glgT3SpZcWxfHt3VYWj+qtbtcDP4dDaZGltvSXWJ xo2lxFpo+aqLTXweGkcsjWsR1UvkZcG4rcKrMQCMErIseYqCoPZZz9Es3scrnuf+vXhrme9JvmWv NdFwDWT+GR18S5EjwjjpYlQeNcrquZZWHrjcPxhjnKjlrt7nu+4izY4Uzd8FqDvs108xkshFYXj8 Fhp92lDx4O6lThcDbekQzuJT/bDio15QzJ6RvXE3+qUIXLCMGEFj3Kg6H2qHisY7zBslN5F4NEgb eE89xPQSTX7Z20wgAHfhq27Twmp+YqmUdU9Lun8WdOg3LsBy63Rgjsp+tinl+WMApwianzpuorUY 1qDck20oSu9+x5hjlyJHVMXzq/HF6DxIv8LhkWT1AUvwUZNVzJ2vOXIPZWdIJF6wh8PxJxn8/IcH LyhuClq1cqPR4q5kcIeTXYdF/CCHFPUTrFtzfgH7xEzKTqXJMftuEemwlImwakYyqGy7/N3Rtmbm zmU3Vc2sY7Y6/SPLoE3Ax81XrorlbW3Gj5DxjMgErVXgXoB+XuXYS10ptgztiZcQt9YEQ397IMYk H1lnmRFT9kJ10wbP5KfxVqNev1+0RpuknF69bkSPECs/oiW+FOXE7+yemyiKQbDjOFVKeKFc/BTz s4YA3HClCYoROQq2vzIYvZBqJQuf70Wkb7Fgv6cTihFizvkIPpWzHAor5macMdBv50+R3eYyailc T510P0c9b4xI/0QAYcqsO3xVWqATGPtcXQNC6b04eaX5IRYLc6JTUuwtZk3puJCipx0hYiafiWET jvuZ8pHM8XedhRfVwXkb8z7/2VAFRmbXYA35p9q+0+/Crx/1n6cQNHLxjfcDoBWbLZu4vCPTaRGh Sp/To3xk8sTWXJ2t3JTblkv8RSLdGkvoKm4uj946VYhSzRzTZfxzOqZSqhFH9wbCY7pyRwW7YhE/ joSbZEKMwAqP9MOtclreu/15N/4U1xI9Jt+Lh9JVXE9inTi7z8ZUvI5dx+KrztktOLTXzetmQ+RW 0PNfN+R0UQCiKhwrJ19bCuJKcF7oum19u2PsNBFiQ9/D4Hgd0zzPO/JUnOs38GLHgFhpjvBY27Cz woLkCoJcGlufFb4ch3WqCivetSvYdxSop4NRgs+c7Ktmu4/pnPSw+Yf4KmA2eCnt27OkdHmpSF+D lG+uxGdQvGbIMbWwX8QyKiai716K/aO+2vQqeHgVyYdT+OXDW3zdz2D4LBVsOP06ZHvTJX6SJDbS OaLx0TD80bcFca6PqTkNKOEr9VR3q7Pnyjfz7nLf7zvgotHupfZUGYaEM6v9RRjGXW2MgYt9Wu8V zaHUtm7KQucmL3klqnv2qe24AAJjYfvsRM5QkvdD1kukPUYmoZwX4jninQgVUmjqSRc9V8pEWh7s Z/j7KFELbey8kLGY7nohDsc7UcdoY/jLX/rU5E/+kLP4t5IO+QQ1FL90cNZNrIK+IgHdCXiyzUyZ cp78Fcg1wgLprDo1pa3V9xy0xqC8e39QTNLC140sm8skOPjPO//WHIePDmcJhN5RYqa4Ib1UBgtV TbwIOiHv0UuLzckwfaCeBzBwhn8w8MDnCyNkgxQmpZCxplkYDcAxzZQaYzv+fqNmAL386q9+SGTV gfuT0PnzJPk2irHJAT1zo0FGrxQOQM+hDy2Nh6DkdLel9mTvmxXuh1+TvUgOXtcjg+8y28S6l9DV ZNHZ/QcT+i/L25JQyJMXtPbCkt6VJpwkUiSNSbgi6RoPm1aQwjSpSPenZl7S5XMoInZToDWXmTxO oOurO900JDs/f8dJxGSwqTxH1fBj1cb2WwW37GbZsX/DVJRw94WbG711Eug8gvlt5KjLOUIYNYhC 6AynzW1VXG5zEIlSDHMCbRznxGx4bwS0k+O6qouiHTOILXo/4LWcez1hDIq3ZG0hRZ24l1m9GuAm Mic2ddjykWv1+xxVVl4Y/1t9vh878rbsWyWw3S85O8lVKpvNYnNH7KuXb0aXoniW/OQ7Jlrf5IwF d9d3uBhQEt60AQQLIyZa/HG1pIxVzVhLy4YPJBBbdv3HsxAPA/Jh8t+aIDd9DhjY0AhRl1rLTnjr MY/NoSVvH20L/R3JJaNbvBF59wQTBRNjRTEjx0cTH7Wvi+auX/x7vrTdipyLKg5l/1gEmIh5cBG6 qefKtA496qKtgRt5WRbt7T28F7ZgjKTMHWB+kDLhSzZXPs5mqLfhar7360iOLx6xlXDFBIfPnuRq 98IlTPoef3oBbVuZ4+5tXXFA480Y9biM4/pnB0dk85nFVRSPmzROLU9LuGbG/8ogjahYGXJji4RS q/Pf93iUjFvMACLZr+hXWtRc9hwww16HoJmtfJGmksfPkYEW9FWKoaPhEzY4dglsxS1eh6Qf5crE mI9N7FdjMx/8SlGkSuUs9CoRMhwtf+15AdvU66bJwGtF+QlICLk9d2rrJN4xPktmjZ2EXfXVJiOY ZuWxuigujJOL5dUDCtEFRbV+s2w6arnU9FWcWVEI+F5vbUIOwfqDJSVdHSO7mlxEFD1K1IfzNI32 iAlxl4Dg0q0FtR+J7ZEahXW1CtJdjxZjXPlbK8UJzYtiLGWrynt9s7UMnK7GrMyZL0M34567g8W/ mhVKSufWoiMuavgSyJbO5CYtdQkOKM9N84qotSddnOeQnM825Ixw4ZTSCcOTVouoBF7xQoOSfpnf zsndz52cJrgESrpntp5Tp2zSQGZXFeGui6qMVqNyjQrzMVZs8dqw8XhCTMTX+/Yrsf6KeU3FyajX 2hXFiwrZZvAmOMMjuErA+s3lc66/pRTQzbGk7na0QEFc8iVBpkhdbao0PJ0k6JxW5uug5KrtePKy DfGVQt0LlSjFmklAT1px6lPXcIMnHnAk9zLUnTPwfthdm2i107OZfUgiq62oQ+oWevjsHJrdpmDP NYmifCO85+jQ2UarKFYpQ2Qz8CPDEA43cdMW6IsfKlSQoirw2jpkKwGvZyJuPPyTxeplkKILvS2q tvGvS7B8hOM9XRPO/grjbZ7310rhDBHv1nwRviwuft98rCVLojTkpOxJdI+bOxFk3LWU4Id4aYoy 0otSh9Jllh0h7LMl7uvb9x3VBHwrodobtJA/MurxxLUPu9lizRtcfi5JHGSerXr4WWgG6StvB0lT yfr/gkF2RIKpiaYz0ODz4KEdcTC/x2Aypof9kLLQfVjPs8qLSGyvra/Y2cr2xL07kfXehi3XXRSz r7bkWGxWgPr+hGZRitiUgrP8ZqSZlmSftll/FnlKPhrZlBOaJYSGyChc2EldbmqkZBGMXeJ6RUwE c2J/+0NEoeQwip73x4QDac823CuIFBVq2UfCQLbfTffZwG2g2smE0eI7+IeqlBIgveBRhZS+9aoK uVjJTbexOoBbES4vnyDv0Wp+dWmMHe/RslpHWUUDayZLiuuiheDjKHBufbl7q3u6B+fbWo6eAiPK EqFB6ru2p5NBjyxpOGVqqTX6OfxNoYzU61OGSQlJUWXFphZqxmfggY0VuU6ai0TRENb2LaUHoBBW zzahVwXrfn7YpofZC0VKCjZbyYB4FLOoCdLLWeg3q3eUj2dF2Y+F8h+69cLLSp/GhOVD7YeqVTWh XbUJOtuMW4VX6CBAoAY4xqkb6pPs1K4Afhfo2KTmiocNqZMh1U/HEvBQkNdX3QNmYFYt2CCe1WZ0 ATHmldP3fqYZtd0ZZRQD99yLC/vLai5NqvUpLLtyqAXiz/XiYFFaK/eOs5hT44LvsB8JF0jsb4X8 m354IkT8Cvd6FNO8yJlg+MVkKfZr2EiLfC96IMVOPHLDz12+7GmuI5CJ6evenOBoEEFzRaSUlj2X igW0qLCoUmmTUWf7J0b5PbuVx/0GRoz7KqnhCZFavs7nAMHSZvHBm/H8KhPoQYdF2O8PS+Hkev7X CRyCCEVjOAPdLN3vuO7KM/zS+naakrwHK5EcAk+06P2aTIPdPzrW27BdcMD4RH3CAigaQ7eo+pwL 6+hORrJ1yCGs/p7koZ7a/DR+rVPgJhle0HklQBTZOc+Ue0XJTq7WVXvAbWGID53FfRbw4MU3/+yN dCAQeJKKFTNNmlGkwc1p5E1N1RE1n5SzlEXqYQW3e5Zw11Kxmt2objwPJd/1+K412otLfvj975C5 BKSsQBBFrzpGDBMsWkTiF5sdF5H1q/BfbskXY5/6vzp4pG5IdDxsgQeTx1ICKzbV89WmhN7FbLdQ qfUImzxcaCbyDPaTE/Mnzg2s5Cngr84VfQuf3dLDDGVnCuVpq2eroIS9IMJ7JKst/a3EEyAXU/tk c04W1wnT7dQadb+93d4SUrPESigXn1/2oxhFcFbLcCMsXx+ndF5pP0M/SLeoxxnUelFcEihQ0QeJ dsLP0SLXZlZ80+TZarG644AdpT/itWG5L6816qdPVYj/rlISYVMskjfh6CniUTEwhp038DEeCQ2t x399piDVqBdqaeJG7Q6aa31csVrnAwkQv3lHFKJyTGaXOsjNC7p78nvMIA0CTik0VLdEUe5GhJ7I l92yp5qZh9E0v2iOfQS0RzU4ykeH6mtJkx1yPtvh17Jh98BxsEvNOxLrSpNYpxZiYxPchG2bzpSP E91Y58oSpy4FbhtsNl8X+91yGQbvan4lEDn/svxMUVAmGs6xh3bTqkdYKVrQ6X9CjTiadYp0ydUg B44dXKqmwsL0JxhL41I9O17Nq5EeZ60kGM+FSAIPc0afkoYSLL+spptujcJ3h9wg/7SFsFlYKBWU ucx2dKDFWf6GkOWw93u20Sa4WFlJx++XgMUvEZJFEpxkwoB3nsrfP8dGFI+FRqRVSnqbTrKvN9wJ ozaIMfKSai45G0BUWEUmnms5cE4HmGiVolq1JyXCWvEquvNXtgs+O2s6qdQQbT7orXpt4ttyQT2I HlJrVcen2KB/Po41Q3RHKu6Wtb8TKmM2XPeRi7hLJpO5HLsaWflSmeDG6WXCalc06UkkrUCeVfvI hY9djl7uQzBZkhcnUDVI1erORn+EQsCLNIqlkNfmebPAes2AGTVA8v7YdAcjlM+u3nzwEZYJTmjU i60paE4hAzpaETy22NCOk8mfHTYaFcGEWjKpYyUVdCNq7lJiRRbz5dhj/hgnrbJ05pjTM6mtMWoI mgbdGcFX8mkjuy81wmZjBPWXZDvzk7LDBP5NzkcQG2ogiSVxyyRjSW8avvwFWxUpQRRvRZCL0Ctt CJ8np8enB3xNZJd7J5Q8cY37CKG1As5b2LyQfKqEXucVG9Eovq7PYfSmZ7sK0q28GWukY+ZKa16Y qMBawDaXgvdZm1OJpRvDeu6PDe8Y3bgHFshZCAcffUgRfsb38X5sUu8FExvljnQQbbipDT/Tne9h nTNY0byH/vSI8XjUbSg6sL2p3xY0R1ngitY6/1Oi8cevg6M7oKa5MmJVIbe8X1rYw8/OoWTXECG8 T7ObXWPmJAIeGaV8PWwJWauUJpmG0a9BxLN3DYtSVEvqNs14eGhsMA/qqeqySoTS0/uxXQrcrmVG hFqXTcv90yieJTgV0JcPBqdSYo/vyYrHoOo2Q/YFktmP1es/vtQ1WfTxpDVx4aJuZS0QfcDiVjSE d1w5GPlpOQ/YF/9CT+c2e19LePh4iD6MjljzCjkoH3TkgPcDq7lnfS5M1YfRXfXbYlrHsPtZOi9J XCxH0azVuNzXMc/pr2hrqYmoq8xu7IcvsMMU8B9eUmIiGnt3fvlwMVwMF63W0o4Wldb2Ke/a78UN LyUXXt2SAIb6/KB9coOx0yYgMvi0gss0yJsEIypZMBdqflmT3tAM1T94Wa6PFITsvbLPInqbRHE1 WpGr6SBAoF7HNLE4vfPSOCZy7g43HaNdLLdS6+UcLgzFssFOq0dfqdodJonnobVflsytfD/IKr0t ugYFOSUjFm7ADorlaJKLZYA/cMZq7KrqFmVSQ5e/2RHbwbqGv+qDlztpMIOYodHoVkioMw3/puFL saA3t6me6oilFemBRcnzLrgxSzPCZl3mqB/IbmAP0z1w/DXKLVYGtoaJJd8yal0w6Co/khoKSh77 V1iPf9d1hSQHI/5zHL/gNRJ67lJpnWj0Wo2qGJATDdkipF0GKcoRqVLMGZKLDftwY8AjkfQrskmq kq2uUcKigeYSTCtnx5HJu0UhCEEzWm2bUIQ6Z1PulsVnUmZec5QLRHUVgRK3WE+Y7clZT6DUMbxA F0fzhAlkciqzCM0UAoYPx9fFOBduVERRoleGxiV1dg1t1QwO62/+LoTFCMq6BOuk72dzJNfhQH66 E9fGRAorBqqwrL925rekLby0J39jtDeC8puJr9WtVdX6i7yRgCM1MJVMgp0WtI4qP15m6cTceRSb GerdPjWKV0Yyo8+0Qbc4/DRQpnx7h2xw7X+OKkjSpfAHc8qceyps+UAfZEn6zve9ushuya9Y0nMe kTYzeO4R0nw3OlN+aZ264E28wXXKVKI+PTVMXfdjhgwM4v8qAvxuM5lshCnl03tQJduwcMVHfok6 WMY3zRZTMQNeHHyM2w2kpZX3/XLEzLgeFmugqFZG7aSUL3WdSNhp76z7ZPZ6alRpH0u3jXS0akLV YBeuA8GyPZ/OEp4zH9Dj7+6XokrF+qUY8mwht0zASToZjkQ2WNyY5s59EMsVXzy6zRWmtCY942Z9 GgSvyF2e1Zk3KIVSJ+GopoAs9mkIupK8q9hp09rUXI1TK1kfldR0JsQxp+HTWDaDwuJ5W19idpmq egSRKE+HSproRHajAYteFAOzIaHkb3WOrq5VzmCiI9iAl/AeljEkz0cVIYeVne4iuYrR5neFGd1J RlHSGyzvYWMNXHPqZQOH7bNndmWvurc6x5T5c5/eO83L7kZb+2NOy5iWZLRRup1WBFoVjqwbZyEH h9MFBcQatWBIXiXEwjmWQROpWAWyrl3Daos3IaVtsMV9GDVrVU0jLhXJYhrJI+NobIL8VSQ/WmHb dsOcu4rG3UxpCq05GHBodSVOPeXBgroadN0mZ1GhFFEAw+35OBh3I8vX8MsXzVbUt8T6W6Noyjkp D+nPFcSJnpqPSzbeY18GiCFYvmezDi+I10lRKSM+lzqhUuoxKSmwkiUSnowCqrQfr1D7B7fnCWec Zrff68HNSnWWovbtOiVIF9YW5+afQAefhO/ZqFkpKAralJufp8LonsTC+B/2pCTUNvFH0Wj69Iy1 rKMa5C5/NPTsWKVx3j3bMmUQCyRQVKeYJ0rZrh9K7+6yGDJt5vHfw5UnZUqdHIgpSjWJDnDAGplC ohGjLslhMXavJ7Vv9+lQMn73Yx9llhSftniZWFBGflw9f3s+hgO+2lPnWrLSAOZHI/Xjk6N4tJ4S RJh0G+hyA8mTf3uTBjuzKfnYjcZHN6yba5XN0qblIPkjVCILXIPJ/wFwRs1SCmVuZHN0cmVhbQpl bmRvYmoKMTIxIDAgb2JqIDw8Ci9UeXBlIC9Gb250RGVzY3JpcHRvcgovRm9udE5hbWUgL1RDTURZ RytDTUNTQzEwCi9GbGFncyA0Ci9Gb250QkJveCBbMTQgLTI1MCAxMDc3IDc1MF0KL0FzY2VudCA1 MTQKL0NhcEhlaWdodCA2ODMKL0Rlc2NlbnQgMAovSXRhbGljQW5nbGUgMAovU3RlbVYgNzIKL1hI ZWlnaHQgNDMxCi9DaGFyU2V0ICgvQi9DL0QvRi9HL1AvUy9UL1UvVy9hL2MvZC9lL2YvZml2ZS9n L2gvaS9qL2svbC9tL24vby9vbmUvci9zL3QvdHdvL3Uvdi95L3plcm8pCi9Gb250RmlsZSAxMjAg MCBSCj4+IGVuZG9iagoxMjIgMCBvYmogPDwKL0xlbmd0aDEgMjUyOQovTGVuZ3RoMiAxODkwNAov TGVuZ3RoMyAwCi9MZW5ndGggMjAzNTQgICAgIAovRmlsdGVyIC9GbGF0ZURlY29kZQo+PgpzdHJl YW0KeNqM9wVQm+3WBgrjFHcpFAgUd3d3Ke6uwd29uBUp7l6kuLs7FHd3LcXdT17Zu93f/8+cM5lJ ci2/1lr3/SRkxArKdEImdkZAcTtbZzomekZugIisEhMzgJGRhZ6RkRmejEzFwtka+K8YnkwN6Ohk YWfL/YeBiCPQ0BkkEzV0BtnJ2tkCpF2sAUwsACZ2biYObkZGADMjI9d/DO0cuQGihq4WJgBZeoC0 nS3QCZ5MxM7ew9HCzNwZlOY/XwGUxlQAJi4uDtq/3QFCNkBHC2NDW4CsobM50AaU0djQGqBsZ2wB dPb4nxCUvObOzvbcDAxubm70hjZO9HaOZvxUtAA3C2dzgBLQCejoCjQB/EUYIGdoA/yHGT08GUDF 3MLpH7mynamzm6EjEAASWFsYA22dQB4utiZARwAoOUBZ6hNA3h5o+4/xp38MaAH/9gbARM/033D/ ev8VyML2b2dDY2M7G3tDWw8LWzOAqYU1ECAv/one2d2ZFmBoa/KXoaG1kx3I39DV0MLa0Ahk8Hfl hgBxIUWAIYjgv/ScjB0t7J2d6J0srP+iyPBXGFCXxWxNROxsbIC2zk7wf9UnauEINAa13YPhn8la 2dq52Xr9C0wtbE1M/yJh4mLPoGpr4eAClBL91wQkgv8tMwM6A9gYGRk5uFgBQAcA0N3YnOGv8Coe 9sC/lUx/iUEMfLzs7ewBpiASQB8LUyDoA97LydAVCHB2dAH6eP2p+F8Ez8QEMLEwdgYYAc0sbOF/ RweJgab/YNDwHS3cAdqMoN1jAjD+9frvN13QepnY2Vp7/Db/e74M6qIKsuKiNP8w/q9OWNjOHeBF x8IKoGNmYwJwcXICONgYAT7/G+W//P/D/W+pgqHFv7Ux/g4oZWtqB+D6hwKod/+h4frvVlD+e2Ko AP+bQc4OtMpAAOXvzddhZGM0Br0x/X/e/79d/v+t/V9R/t82//8WJO5ibf23mvJv/f+P2tDGwtrj XwPQJrs4g06FrB3obNj+X1N14D8nWRZoYuFi83+1Us6GoNMhZGtm/d82WjiJW7gDTRQsnI3N/1mh /0wBFN7awhaoYOdk8dddA6BjYmT8PzrQeTO2At0nTqBZ/a0Cgo7T/6YUszW2M/nr3DGzsQMMHR0N PeAZQevFzMYG8GICHVAToPvfmw1goLe1cwa5AED0fACmdo7wf02UnQ3AIPSX6B/EDmAQ/o04AAwi vxEngEH0N+ICMIj9F3EwAhjEfyMmAIPEb8QMYJD8jVgADFK/ESuAQfo3AmX/9BuBssv+RqDscr8R KLv8fxEnKLvCbwTKp/QbgfIp/0agfCq/EYi76m8Eyq72G4Gyq/8XcYGQ4W/uoCiGzr+VoNKMfiNQ acb/RWwgnbGdNWi6/5Gwsv4lsbH5He6vsTOY/AFB7QP+jgAq65+d+4+EBUQJNFFrQ5s/fECkTX9D UATTPyDrX9DiN/7L+DdkY/kLuv6R8i+9nYvjH+FBJmZ/QFBA89+MQG0097A3B9r+YQGS/ZGQEUTC 8g8IaqfVHxDUJOs/IKiDf1BjAnXnd2Q2kKst6GT8oQeRtftdDMjZ7n/UIDL2v9WgYPagp6CtNdD0 d0dZmf6VOv5Po1lBVduDrii738NhBTXC3trF6Y/4IInD7+mDsjm42DkDTYys/ycF6E7+r+J/s7Bw /av5XzHTX1P7YwRMoI7+Tsv2FwK6/tFyNpC5E+g58t9KQY1ysjZ0Mv8jBIjP7wSgu5jB2dwR+Mfc Qc1xdrP7wwEUw+UPCJqT6x8QVLjbH0sF8nb/A4LCe/wBQY3x/F0cKJIn0PGfVP9znxm7OILm4Pz3 Ewd02f0H//2jAgh0BxrDL83bGfMEWdYEtd1XCeG70e2N882Q7amnUNF5LTm2uzwiwyZSVWYEbDje CiUO96Cu7ohR3gguE714HTfXwYa2xCu2Pnk/68cqTe21wi9OYg9MfDsWqu0ngPtApyK47/3i4K3m bwXZDN4pTZbj4MKJrJCHce/WJ+Fe21+yMhoyv6e4X8kug/BcMk0XpRqp4180S5ZrlDmHSwLjTEfw jhr93B1l9uZ2Bj174o1IOpYG3udXFEuBl9Ymc/TDnOdamQqzU9d70vdauASQN+ijU+RewodJ0jgL XsUFG46jyXPY7WNbNglM1oeUnntySveOR5d65GQT3JTvcRkgMaW+RG60SMbZFBHHwbBmnq9Kb7hL XXI0GznRysdZvMlbc9SyKSFw7XU/Ztd6HsJSYtVoUIcqmWMl1hLZ9rKFEoBn15Ijh+XvmQ0+R+Gs 6wwar42sYdY3cSEbiBE0t/ttm2XUuPkfrakQD6+6D0nNXkh7Toe5Hu1QW2OeXxun2w107o85mdi8 1wNPJKuvD+A7rGpJzBZFoMH/gl2bSeByy9WMD40X9TExIy2nmaVeAWFQX7dm7xiBKr3ysLXfDf4+ rP/nDmaJasmbUGV5DmcG2jc4SdiY6IdGiNCtMRdsicktbtF19NAt6bDoV8qQvCv18p6rsY6rM918 OkZe+hIzZWEURDnrBAytLPmvn85mcqPZ8PW46noxX7mLZFqoob1PxeEDgt68esMBy6F+JfNitBvv s54c2iBw+NstGfWS3Y+TahT7Xyk0eo9OPh8Fh7dZvEcc5nnvoTWscOPcBLWTB4HN/1a9uGp/QMjk r6xWBe4TM8hd2LVeYnhlwmjet+D1+TOvDcwcs9vJaHbSYwhWsBnbgC9kQNLQGRj8Q+AHfC6JAHO/ SN+Jk6tZKl4W5SlcpGCTzlII6V6weZxPhA7UX246yfCDd3jdKJhynPyv9OREXfMbIrNQM+gl+uqU bdvctc5Ov8Vhe86kP9FnKn+Jl70mogpZ4R1ujDx1fVGYMff7yMsk9VFef89p7Vtr35D4nZlLVe/O CbbRpuUKjTNMyP1bfFzu+RNSto1CKZU7xeNODDkzu5eBvem8OUo4zt3EMypslAXUi9kFgWzphE6r HGzZFfIFYbWAWlrxNI77dLoG8NQY396icEQGsTsV5SiZY7VHQDGFHDudFaJPq2+WKvn1Gw1euI1i HwUGF/vWohQHTJ4h5x6FaAbEFAmRBmP+xUhT2gPNzwmTNV3hoyTMaSSBwTizIkOPhjOhwpopvd3G EgOYmBsu7DQ2+iOLRJrAyA5paPdKRO7u/P3ViTNNH6WDCbPBPS7fyskj5yNr1UcsT+ISXYpQR4En AWpToo17769zt1OUeWLXEt0DnYLQrYhotlbpvp6HwJPliP58+Vv+3vUd6azIWqFsqqzxlNJQ7T44 mNYyzDQC1fBevhdoj7MoXLYbLPuE8yalbNW7pksM8kHGtpZijK8BFEl0m+nfazgxEqecpd6E3KlH 80m3iFfNiJQgcSX9lME+lPIodauBnahu+v0SILYujJJdKptiNsOQP1yhxwxW6eOIx31scJuI/0wl 0AxZIPJuIydJzQiTQ2/bKj3r+400T5kxErxcVwL1FdJSlMSHrsJvm0ve6XGzvYRhqt/ejMcqNvIO kYeLrWc/b8WzT5unsUNrwzEvbdRPZn8lJNLadFW9Ttu2blPwJqNSG2xCtjlvWSjNJEBH5yNvM4Q/ KMO+lo7udUoJ0ifIH62JS4X2iP68hOftpRSQF8XeV5c1kiycHMNhKXWHB+uPE+o3lMKcjPdSa4gZ /lxEhviWIsjBiiD4Bob1k5Uy7kMbPR/4KWCMyCBr2pzCvmx6h8Y90XVMwyekRuDLKJrEPAuaKAF3 coBeQ0x97kz2V2ryZtYL+0XHbfeo5y02ICRhToyeOUq6leCFsv5o9B6RiLWeHsyoz3IDnQOXYMvU BvBtHU6wuvW1EDan0TvCTTbCMd4Yk2tDAYtyN4vXirMVtpInsYRjUJIhPr+RNVlpWRK3la2jfMYP Zn5PtR5rgVb8g/cY9WpgZBAkRlmEIdGefTgybqCcMuKkNjF5OTTG1w97/aSfvefubMgg+FXaJ6yq tt4dOboEV2kKytpGNLqMqxuQCeChKHISXLtJsJgR+L3ytCGYFMA0rVqP2MGN4TUzvyDGq25d3bh0 sW+niMx/IcPr11S5oVC6jhzEs5wGy7abZyKQLbH1+fQaQwN2Hs8t/1rPMGpwJxh4MJM9E72HBYY1 zNoUXVcqDY8cJIz4ZXSItFTTcK5KEgtOKPazGSO2lGLC+yvedR6Y5CdXb94TdthZiaVdQecoB42Q SP6t4Rh7KZxysipKQo1r18SRfQ4fpjoucOtSoUqZ5Qo/cCL2ArR3SUm68d2oNGmPm9XWsdf67FhX FEnvjydZVg2sqFUQGygrvutm9aKwsykt0ml3U4qV/xKSJmZx4oYGUgvl/VzNH0WE1notGRJzqpOb irSpM54suaCnX3ngRwztCZu5zOBX7Awlb8ZPGnSv2Dsebfa0TUHuDAuxEimuMnVQp1tKfAlkdwi3 g6VOsaehvzWztNbKcSQ0d8aPGRdg4WYucF66WPIgxxdPJXbjvXUgOkM+qWoVp5edQfbC/b7FkybE aPtoFFch4NqHoSk3TiOdPLBTidHPTp5D+iv96Ssvggf6z4/2KHcTVZC2eIHXllX0KruJCGA3obiH 5KVooZx7s9me1f2ZEnZhXF2jsyR6OOUPfkqo3zDcQ721iPX6UTrcxAwXTKunT4cbLWzeMqJYWfEZ jCrk3n5R4KXeQsqs9LDstaNdXB5V+5fKm09ICw2mptW28esVoXO5Zs989aDGh6WQsnFumqngNpH7 eJYckJ/waQZ/31DJPaeXOPjZhLhdCQE8If1zWFo99UUOjH/nLtd+Do5yRb+oCYbVYi9enBFp1aPa xWBIiZ5zEquPORswdl1SLsyxT7JcqoBWvhGWnWKz4P3WBJp1Xi1RSDp80FuInPxP3OWP7oZgcCPy Klnv1PboWikbiaXrG7U6PDQ24NqLiXUrPRdWBRfoG/y+P2Ahsp2Eg03QxcSLfqpj03dcktw7Kncw CyiqZCB0PCgKQD00G6tUIIv/4g4oM/nZmdTlnjMr9GJ83J8y0jKRm/esabJGGUTNnXVvF2ViEJaX JyT7qGAUFBlCZ/stNvSo0hkqTvdjfUH4gLUZ+fAMsDi5HOUwyYjNBGOKWl4Mju0rUrWqivnj+zVr 106uVnx9KvCH2fejpPYi6WkDxwSoNKOUTbLRcHNtywpPal/i5O1zRwadjUUMWuZythnTo/tq/Co4 2co0hSj7KY7iwpik0xCoEPEqMRJm+EZsBcTso6O3DB3Q8iJ0Kh9+UlD90GddIujwkdHMraSpN5nn 1zNeV/vCPoDOKKY14Ew79IKJ8Z6qzOIWEvgZIE9UWz6TzWaLVohvwSnM9kq9PxCU7L5p2y5I1C32 BCGq8MbfkgMkxZS6Wfml12S71GWIoFnCj3Wfm2f1cbPCTaDbvner7m0svvdly7k+E5tgpYtYnwnQ NtNn4ljjVfBsghi3Qxs/LXipS7PYENseIbKC4tl8Y4vmBXXEqPRLtaJT5iLyAy35nP/yET5RlQ7e QPnX1d3JPKZPwlWlE+gfpXdrwOvPu65eey+aCOxxcBixxgCYYOMawdLoaLEyHFqfREkM8xOvTfel mIo/CMxaOvdSDHNyM1BGSmVJ5GYvw8KAjdSzRwxCvQ9NQeINpMmt+e54C7jPcM9atPT/wRdW97my latzz7NXqoMYqnCgwRENKCHXqSFnLxSexAqj6qRrxGMWC04YRfeJ+yDyjgGrbLli1B7DcDHpHQr3 ziST+8NgwuXQrkEsjFpX4gW5mBhEtr9AgzEqLvL8PSaiyEryYCoHRw2dweASKVRluazrBuxPgp/Y 3rXB5E59keOIFewaxH1mH21NJkK1493QdoP8bSmDTJaik+1m7YjuivSDfyXYMSPv8K61lsBBhFV/ GETqRoEgDHWX/SrVqQfffl2uTWoiqvJTax71AXG617KvcTMg3sJbnv5AjMvNSBmixalfGNbz69Z8 2lAdcX0as9k5xLf5Z0z2YtTNlTEjWGPHFKLmy5VYdTwm5M9CJBu2GnmjF0z14K2swbax/jIE3fIi dNhyQt+3+LhwzNj4TtT13MdGKaHd/ev63pazxD4TVnUvO3PioPrIZV/PS0WLjgQlxKWEd5amFRB1 Evt/xR2GWIf2I9H1AW8Qx+zs74pdElPBYB6yUbLghSvscWQxxGYcPO4nJjTdjhs8hG6NUM4gV8fy UsOfB6uAWdF/QypB3uwrqmdITlU3hvdH2mOKX2TL9ojlrMVnQGyZh1Fr2aksul4qau699A/IRKXW E/fBI/oogKiAvx/d5EcH83zmm4Rn0+treZ6QEfdO3PWD2gCAKFzmfJEeFVaHQjPvdiD6WisgIhsj iiA8bN57H52LHDajvR953dSRA2WrwU+xzde2kQ7wiS1cEX3vhEE7AqL9A652fICfvJILsAjnmX9P om8CocAT7LHSUuFHJa2l+oF/ArbScoULR50D0q/We0sba/iQiDSWbOLV8/dcjnFBx3hRFnBHqAYu fZzENWykeLXGUMUPqqqpz/IjPL4+K8UoTWj3JvxBEUN470aQR5MekIjMWjpgocGtVZ/tXAteIrJp 0GpJFMk8+fpIi1XBwF14G7Gk2ZUkkm9H6lZHQ11t1ndrcchuHnpbs2unMXXL+BREfIKeV/WZVwdn rV5uMDHIjwcgEI54gNUSHAtx0AVSSfeR7YyKpt0MLbG3cBFPRO83ztKzgMTK297M0fHikzxR819t sfxYuG8kLL+G3lH5WiWmjS7JUgh9KVpR4hf50LIhq+fzqBti8knYpYnSLWWonxwwJyFNjXI226Ec xeiNeSjiPmDXWWAhF+knIygnmBJ5jJlyZ1rZt7v2EMi8XvWtx2LeMZdjCjqE7EMehnvE1BhrAceX E84fjSdBftWhQ3lsVr5z3K/tmaJu9iSlPJYXVUJQj+Vw4wbKSjoNmlDI1fTTbnKbOC8i1bFWex4Y pb7rLpIfjyg5kSfkOGoBrigbDFI4hAhmPIfxjLKmeF6LcU/iLARw8VGBym6StvY+otWMJ7CPx9ce N7uGpt8C/YozhFgExG6j6ZVQs4WRwTeg9/yToY0nVL3lL4tpmlUJSOscp/iWQsO/VuDbfn2pO2Az GE3Mp93cYpg913PkYRPA4xpZPEfhV8yQW5DQsoANLNu4y4g1Sw6x5/MKmdFHGUplnJUc46f8Ueei 3j5T4OmdrjJat6MmJbtTaUq9kR/E3oMFO2p8ULGsx5ucocQ25krnVkG9J4uyGBP2cJXEFKCWV4ho ESqwmxcgLWwanPM+85hAqRAC1sJrV/JjHas5AmUweA6+Wb9vGzxnB9189MLP4PYxTMF7Sp9f6bjP 4fvYKhkwKgiyyRlBCuXVAQZn9rNhuQaekV6vFOkZYe6I4WdYNGlx8D9weE95nRfiuOTSlrpHN6yl JAly1pwye/nOnIAoVaPbo1A4t5+/MUmPl3ynjlqo6vz+STu1sS/KidrY7EZw0dfGytS1WIRzbWQQ IygECYlLxV5LMEYPLXB0ALIoKMeCw76xlLkhMUDxU2ktR803+y6kRmwTwQMqJd3uN3OBZW2bYHuB j7/c85MZaaHc9gXsPGsv0xtX30ltu8gJS6JQ/AKvPY/G6VdTL8GKJNZgnwWmabReMXz2cbr3GMtM O6Fc8eFiCj9pZ+zadVpKCeuDuXOBDt+/xjmochrMl0DyCNP5hiNyKQ2WQ+VPvtFlIxtOhg+1kzPa CZumAyXU2T05eslhL1KWk/+pCL1AWriZ8eYW7SZdyfch3+aNnScwuhzOVWGbCkOi8nlnTwGXDKXG VJcsRt7Z/3NM894OrrVpm5AV7HYuOv/wfHbwVj2RT4NgmP5EY83OG/T3K3VM8oNiKuiPD+MLflVp kfNPNLk4pMNfmfct9FO/C2d6ukKdGXzQFTka1DWmOfTzn4psCaqDpg4ceaiv2JPFZ2xuC7nLXdAZ G3qFsRsgWrmKvrJpUYP/Tlugot12BZO/NG8YdEbR7xi93LHUOM5Hk0N/5KxfP01L4VfYo0C1fDVZ lOVIPYO2xaoebL9AmoPwQdWLgC6yoXgqQDgc/wqQHRmC57Cck5oHb2ggP+rtsBNukgKdu741znKE BznT20K69ZPhyntaVumxkc4gEn+zEvE7D/u+7KN3f3avme+l350mwKVEu9r6ktKubekNZtbfDC/V Wgzll1t8Y1P3fBCNX+Rn+2uT5VoNa8wYUQ0XIsy8LAmEZkIjZlkIYDnSXjg1cGHhfsNCaUXDnbNZ 6DBZzFYZPVFHCSsXJscjWWxmC8U/vQcytScQ33LiWYWLCfdMIb7IsbJh6VpXxbtvyoUx1XNTXTZC +YCv6ms85JUtPvgaXuC0VfpD+mtf1DvgbBTtYVTf3D2RUP68eOaiVR2Dkc7NzmGNq9Ta3f4wRCMC ZK4i+jr6VSa4aPXsz6snq3unpDxz/oY0g+DYbMOo7wpVlL6WZAoVIU1Dw/wo8PUS6taJn/ynBCBS M0N3688fCm+h1mK/k3dHmLvcgJG3EwqBh5CiRgZEzZI9RnMZ6yggkkhQkqwj9m+XPLzc1Ids/YIe 7urGTJxGBfMMhvoQmZ1MQxK/64ffhLncFFiiti0ZkRp3s/y+PopJYjn7TtI9YlsTpe1IpWZsunQY 3OkrXa+9ciRMabdIbU2OAzd8ACQXlzvQGEZXacbLU6QaMrRFwEO+RXmefp9mU+WjwbMBySLAH27A KjbDilkonX+P/EOgXs3a4Xslzyk+U7TwoySqezNLobV4CvdQI2EFE97KnQK96WjaGWKU3lzXuT2b RQRp4thqat6KagblGoihHgg+2VEuHw0C7RUllq1HIWYauUCTY5JtVzBxdjixmzjtM3qU3PEZLTm5 AFF8gp2eAyJzXKzIfG8i9HOZA3626S/0bJQyW49YM/w0sCGBNKytHnOCxVjr0JnRk8Jzp9qaRE1t CTdwZQR0Bs9okYnhCbEXY7HEBUV03t9vlNRkHdnhTEjRLKPuSt0qgHjZLeVRJ5Rk9Ab0t5dIzl3x 8Aa4F0Ofw7IOb+tibJn2zAEotp5TK0ClzE2/j7YPNruY6/JVEK9LIbhuPW7yLtRyg/fIVmDuxhGZ qW9q+bPdXGWSPQ6cVczD3ybQWC7yxi8vFz7PpV/nBdL7K6fB0AzhCJKjtEfE6vRN6XZlGqlv5BLn ya0X2C9hGfaeMsaZxrRZx6zZts7Ai4lGIdbZwx6KOHJfVF0RoO3N/lJEheLL3wWLPcTcHs1Ur8eH xOb0Ioy1EVFdbORpVHR+xd/J1K6I0NooQUezHEir89aEVGQlGtoX4cXSAk4snuLY7e49igY7Xc1z gk9Ilj5mrkCNyF5LJh82n+ccsICdpFqetZZ7B/fac54Hz/MRvTmgCUy3BWM08omJIQMnIfm32bNk /H1MmS/cDGN1Dfn5LcWHG6uDX+GTs5JLXpzaghsmwZou4SLihmA7ntW+fmGLh2h/+vGtj4fadsyI 9FK5wvXE/PkHnR+nx9iPr9WbDa+nhHznFk4rXZFEjpSZDEhnQBQwTgq1sOZHrfBPjBycqZ+wkAYp wxJCZZdISrrjkwNaPlmqjdd12o6lPn2tVYOucu6If5By/OlL2aH0LbldkninsYENy5uNdiMX1Vv2 hw8M36EGPck0MWaphP1rcBMthKFMj80PFQjxU3wJbc2GH0w1seZtx7lyJTGw0nbztNreNwTegvpD D3xGuxtOnHplCo5pyl/LVZHxJdGAtWSUncxwy0h3GGcSik8dsnxgCrbqVVbEtGC5o2WnSmeUwBD0 x/ETiONCPT5ERlMNxog+GGBRjpzZqdvXydBzU+3Wl7da89CnQagev8AB53qBiaU6PJmLBUxnMGqr MrNlsceuapYs47yNdyqly2sopCgyG7lQgVUZ+ajhU03FSg9puELf5lFk2Q5j7Rv5GQubcnf1X2w3 fhijzaG3d+OS4E55NVqHQQ2PpyGfOhCjUir7DPAAtNRFYsTjopv2FT8VfmnwskFCGV6VugR79xE9 z7CKgu2bIqLlANzcEvbUTdxUAGP81I0BZ6fYuZgo+f4229dMnCgKQYu9PWou2s23VU74HOf3q8RI WSMrzfUao6P5CmOUel+uvJ9kTj4kmcr21bPoezgvYBjM2gIRUdlr1gL7W3XMVaJ39Kar3gRDfiY8 EErjeJTaUTs9TOlH6AHrORcxYS7vSgrTDqVgu6Yp2UngFjMMvxDmfa83YEsoPh3ERl/ly0U2TsKF I08gTNBXdxHIhQjW+gFNIfLzRQuAHNT8McYBBX2StxUzf0839RseLjxkGbMyi4u7hdwK0VZC7oCr CS1JQDiXAwer3g3Ny/x6PqqXABIWznAKp3WErwF4ZRB8OtxOZKObg5dkTjvJ2TMU271b28emwq1U r8tPFpfun0+d7t6RdFi+v6sGRn4zK9jqzWeOToGmNn9RKlFCKbOgDneUG8oLoc2vEIPlQh4i5K2N p2CmZXnWdFXnAN/AweKQH0PJ8h1Sa+9Dn4EsS5iEdqSytgwPJBIA2lq33J+01ES/nE9LfEPhvtU6 ov4aPjCbcnGWxXnpDJ1zPPbK6T33sqjF5uKo6+q0rzJj/wOFi8wsAwVeFtcELk3zYN6tI/j8AOGq wRr7+VBD+CcT8ynlvOMR2BI9Ft47NQoir+PxtjhDcMUl/AMVq0PUITuZBIXsk/1PC5EA8ZSDhFte 96FNrI/I3B2HKXeGIhdKEQCzd5LogcijrRM+KxzVq1KsjvH+S+Y6izEw+aZUCfyZKoEdd0WCce9I +Oc7r0VVqd7k9HSdA2JEPBbHaRe94UQ2Cpt9pdMLtx+yln8y4sKC1wuVMtN9hMPutYymF8/jSyC6 7WKRvHlmCNEl/i49iRQhLRs/++hmDQAmHJB1Z56j+OBBkcP2vzt50Rp15RhmhnskpDjWPuOAMnEg 4MXrMPwGacRokM8arVf9/BF5aBHPh6DYn20TnPnSIahPxJ4fvtNRomgm7uTFX57LWCirUF+oTvrj c9J3IWLHZhZO1d2+arOJdX4YFnNRx2MxnBhnaqiu4xodVNP3nEpso1/DZT4pSdvkPXx470Qu+EHK 0e12uO9Tdjwe22BaV+jeyvsfqjWXWdnUZwZdJHmigfgBzxdby/uLK6/KqWC22V1QpD4LK2q8enKF F4ELdBGsvTac5GLZYYFZzkC0hhUx/AkEY1T5Sw4i7+Zhm29Qx1j7fWkdZ8m2AmC8RgYhkx21lgjh +a/vkiF9x0v7aIMgNpZ8mWlhBQ6eOFHxCrd3i79Vnr237a+TJ8i2cYQudPAtShlxnV2TnnP/eFyP FAHu7X0RbnQIgWS1rKjqZijQ0/OF25kglk2Vr2KuNKHabe8QFxzTIwyXBq8zeSUoch/1FZFgt/e5 QTm41kdiT7gf1v1kRQ+e2P3j7djoIMqd4jNCn1a0aBOxgZgixRbCcS/lKy19bMTlyZIDDcttWKUe O50VLvz2EHxhtRkarGBLNdh7E6d7OEl683Pp59heXSvAN5zd0R6VheJaFXUeT8tsiQFEW+qFG0pO lnJ305YYQ6uAmVLohUzrtbL0+tTvekQftESb4ZOSCkyDp2uvAwmRGp1DdG81h32s6zRct8pZzosJ vuM1GGJOEConN1NFtZHU/RqujrSHuDNU0MPtfXtyUIngI636pCaf/ApHd2XFbZpsPD93r4wpSmAO d0kF9iLZkknMcaurgwe/l+QWSXvHqp/sed+YtYF4IWHUzOtfHxQjQ+NxiK63eYW6GGHVpC+tr0rv +KUp+HQCbeDXoqHnIH1HGFyGjO2c/fc+8JXdyVHK+PI5BAfSkHFCzabkeJjvzeBP1DQB3jKQ8KEp 4Q4H7tP2qhz42avkQ16fGAHPAGWBxrm47YgoUVK9Svz2Mk4RvqwBJRVLrW50k+/YvWtcTJkwPZ4Q nalcUpFdyGa2padmrIO8dlBPL4nbnZUNTlSQpJq0LBy0qsJBL2At4ZQYrrzHNa2OqfL6VHuCqaUB hZd2kt/r7jkCMLz0OvWDmMOQwD6pDO0z79KYXKzHeAVEEvjRrjs4tWJ637W9N8wz3iK3xJbBxoux 2GKG0Ydx58udJofoW65eKYdgR9LzaIO8VnCmeSlO2RIeQlIXxElTYSwtesjinnnBpB8jhEhsBzO+ 9YHIZNFsBG8rg+JXierabzyCJ2oatM+1XQiW0oHcEpYq2u6kidlolYOIGL9uUCX8nBeliaque4EL FxePAhlbownNjd/6MsZ/t+zDqcPNcjXMjtb/47YIAlU8FYsjZdWiAM/HwE2Hvx4DQEBCL/hc+TXQ 6Me93758IxGAbCWwhsD2u+P5iA6G2v5D1wnGm/5mjLysPV1CqfKucAEX0tQr8wm7rXhGgQWWkSC0 CPO0TcGDgftiRL1l38jtGCWJrQbfBAo42I7Oa70hSesmOXwpAMfVzHXb9agKLd5d0hKMoTIZ3adG q4vjuFdhc95cT//gaqqch+AEDky5zBshEspYzvg5o2vLUZ94AGv1NeBs3h1hayNudeJbnH6g8wTe iPgq53uvT3LeL2I+Lcg5QzpYsRv6jyl5pg2VfGcky3CbrdPNjcWTlkIo8tgWksYaiwvgosmkHFst 63D16hK5hOhGH+dSPudgoMqIswSZCcaYaauovpOWylHJEb+kKzTBi9YugtBM6tCSdT29itNTp4Xn N2wPZt3SxeZaybn50gHNOrjUpwMnb7XtivLB57Iey2P1J8tHuFnqtt7m/ZdnFCLyL7mKTJNOPANR pVOkZwNTDHo0W4+8dWiw9ZGRrkVc9g+vnsbCfbHDMCyo83bnZqdk2gdICiOsoibx1/fP9xmKHtCh hwMGub6ft0T60r8HqanmnPnanB4YBV00bIP5ies+gtOHJOAyZuMI9+DzCbQgZbIFy7lJk4VkMoeh vbxXqXqKhOBb/3Ztc8zgkPuqjrO6PZKBpCUlajuDhSzIj1EraDGg34bmVEL+sz4KDFZloyMHrxLP tSHhnEDeTYwfllACLOrLbDSaMTuJ6ofLXc/1r2Hwtq58J3EXaJE/DAvjYcL6Jo8MF/NE1EO7lvfg zKEyGIiCXbIQurMdP9jM2S4qzVq5WM/BoixWl44J2p4ur1D4mnludcgppEgHNqX7zTwfIOXLAnzp WIWH7huHnEokzz8kEd3lWm6N5PkunB3WbvP2IDDToT0diR9FfvKaA9+wSf/1g5BJszhrpJpIt7yT G57naewUjqf1ga8VPO6R0mCvcew6Dv0zH7a2CfOQvcVSnDyTXRGbOLXG2keKkXPwt5D8KgTdQ+wB nbK0KXHSsoinDqJ3di5pF8OUn+dF+k27Qx3HS7TeR9EMmHN/N7mm2VJI+xxRmc5q5rtvRxk8qiSP EZDh6UjvSJZqefFYpb/X8cPhKXmD0KiSSxIjP9mQqNYA8MlmcS8rAl0Yw9uYl9a+H3+sUH2peDMK FjffzAcLgUmd0ip0k1vh+T0UJyE8iXqzQH1r8AmE2AXm8fhQ76/XJRIrjdpC77rPgfepQLpSd153 spX0X1NNVaIWUWJ3x2B0w5OT4JRf+JJvwVw2GX6O9nJ91kQ/qMpTr5TPZFopilatUnFDCZh3hQty WAvcr3s7ZGGUH1zzRl3/QrRWJT6hWkGPoqNUGcMsB0na3jDeCDspmP5B9h178U+zAjz5erLueqwl SheEDsuCcJLWYLwpotg+fStZ7hu7aTX9XcEpnccbR/RrAuiGFf6WMY2IvND3fsYGNaqs11Bg5bRX kXzufHrZpXeCDE/kq7dBuFIadxcLJpYDl4+kqTz1LXsV5qveFJU0cMN28DAvl33rnMR6h9LY/oFH 65hTewMZ+DZ9UuGQ7+p922+4y6yDHHBrHlHWfi18Ns1NfHYkCAnwopPrqNM1eeC/fOiYIBmreOwM e4wrx/aikSogdJtVx5XCHoyFWlT79pqVmTxzUS8nqh5ezDSVH3R8OzxZrs23d4F6ub6sQpuSj3Kr qjTBVs0frodXi4EcYlW4PQbj9TG7cokI+qRjzmL03IYDyW3jjJ0KRjnzM/LPbvQ+GtNfhiEHuMDy iPBWYzSSyPof8LQGLF38T6+PX23LJlwHHgeJ3khsVaEdWoYMe1cdoDd1F3aU6pdROnl7NTgSum7A Sdtfl9R67OltnyzeFas/L6z7oDrxmZIkip7Ctv6iDP7wbMJj4UtzK1zbdDd/vJ2Gs1jUUF4fUF1k UkZMVFAzBZZOmyvqao5QqD6eK52Om+jzUO+SxLOaOjo9vg+8da3HM59LxohxwGQXAR96NfGFEobZ UvOyjIAcYvPmyJuB61ojE79Rerzvc6BbxcTX8NqcG3w+HX+5SLCcxty0XFGFOMOikBdUMs1ZC9pr R89PaNpMM0ITHVTmeHffiLITCWaJaJMUN7EN2/thlVhAT8zk8/tnscDHsQ5t1Afx0dEGy0vvavcx sTxTs1fiF+k0q3sUyw3GoSYD6ORYqXe6x8M3frX3Jksx0hrk3SipoprW0x/EULIzMCrJMfuJX6YX 6yiJG8e1tbc+LL+3w9ZIRQsiBNJsf5EAHxrMVrcZmigwxhtc3zJmMkx8x4QQAn1KZ9uYmGFoY6QT zJam/sFCUIudwVpkVQGjmWkFwp2xfUYMwfdjJXngqQY9ozi5/0RjvVlXP4tFd+E+eoPNOpSdHvqI PL5FwGEs3iuclfsFiakQz3cUvEVSveo9X0WzRAGyZBqiivUQ28GSzPkYP7lfpVZajij5mE+fhzR5 RC2v9f3E5FPwYJd1TfevSvH6IayIFqFL1y5tpGd5KEtiHIINLSgdavfnULPX0wX1dOUzEpvB93Ry 4CMlCA5N0R696Ew0Znqq7gZyKMs58+uiEYOoWgim6sk9orbaCS46EjhFnMaGLvGlpWve1ePlFYZs JO4fptfshXIkI28TZjg/M31J+aYQVgMHCIOqiV6g2shXJlQAmlP4m7hnfj0dGTJbECzOEqh/ZpZC DMBAV/PQLxI/o+0pKZSi8jJT/bEAIVMZXyX97SEDa0sc7xEDX9qKUzM/4JRTnaOi5aWaWHRqgCYh VSV0xMi9J88gMi7gaxunXzYq0oTndPPD9+yo4guRIFIoEjfqhsEZ3Fg2RkJ+t9yLCRQzzaQ175vo +S16rAjM4XaITVh/jHTR3fGjYiPVAJxWfWQWy4YLSStUz0cSpB6I6qVvsSFGTzBBHRVIIqPC+bOc IXLUbFjFLsrMn1mqeWJkhCnt0kVC1qP1JWfPn9e/9Y+KzV9jinObKgcQ3MtCOQdwspxYBeu259Al 39otJ4kQf9UT3A1R7MLIN7QL+CWit/CqeispNa97J04PBVF0WZiEAvMFlU2+aKeaDud0ngJu14oX gAKDid8wnlnMXueOdgtrMJnJcv5N1hf6YtEzaR5TVmuZsArI1l2tXDJOgWqXY9r8ZP2jjFTbaqBN +00+L9PFEJmm1bdjLfkCYYv4QZCPX+Qp3vIITSsqkX2p7ZzZprsk2fVcNYffdVp282eeIXqfJaTX tgsppqWD5I42upuozINYXcCp3RdFAl+inmL+rPc8wkgR6eLcGXKhEpsl4sfFztk1WsJV1ux7izF1 R8BCUgKEsmmu8BVucZSy9FPp0B98Y4RtmW1p8IxPb3Vojbs6rGcbaV+J9CrnNLazzuYtspQhBDTl mq8EB8ErpaPP9cUiDSW9UZIEjKBXuOF2oHAe54bjOtHLNiWhlB241tJ3ZOUsuvzLn3XiKX+wSi4O igwnrVuGZou0NXDh+FTS3MW+s/Yo6MU4DJWxRz4jxaG7dtpFxyF+VVyF+DKx8rMcB+B02upkM1VE Voo0rDpJb7+qkpdnRTDg9e4NSQM/rpNq+uxgnJstZG4yaEH2XcyhsUX6h4P1x7FXeYp5zabpLbPh HXeivXtWGSs4eSw4U0gK1BFCT4lLbwKV+wmi5Zz66qkMJvCvcWTa5p5Cqb2jGIONNcNc2rzTSauB PFLCSw6kynmzH50yVvqr/Ml2/NorxXtOpE/kYWovnqSRuePybv1VEKLgJpeD24j53T5zDujH28F8 FO0UFqOxzOHBdofYd0vyLoRc32AKZH87SbRvZ5H8HtW1PP2kTfJNgIYIY8k8llqmUSA6FY+e1kRH Y0VsfZIBLRGL9rA7f52CCXGVK0Rh40YKgbE4Gxyfws6a/MCpBBNYNbQrXVEh6PqxOhXsM4vDOpnJ FRaxgE9xGlxMCBTT5yvYStK9Quoz30CggCJ+obJSzmzFmlDAkzDbihYWLHepAxHVNJhPddIvk276 INnznBfxnnFG2L5O6m/fc7Hgg5AV6cP9H0fblJmL+xV7vDo0IEjBlPXRkOTZzybRcMeUYCauR5oR Yk9SX7hKSct9lh92hdDhHDHYN4h4Z2fBczkeQqWkxGCAsHucA9ayO4LqkAFe6O+46HLa9+DmvkdA 4nhS0/cIIs5OE06YxtRB2l8aqDeb0+fbY7q7fyX6vHE54J8bWH0e27hm3mDzS2yTGxansQ3gX8uG 7P0Riv/qY2IPH75fk7ytdu7DpOcvDyYOOvgYSMjr53YjOorl5To9W9Z3HIz3yhx8xMWkGGx2Pxcd qN+iiRvF0rTQpH0r6ihOeEduSt/TzEZvocxUei0qTk5tDMgP8+PuiM/9yox6Ami7yeXr1NyfGDtJ 3dVIDBeMTavEWg7NBX2b/9gcSiIYoAxBXtvEvYZtxSrNT+6KkedZzoWNiv5ENzIue/aDumRQXBgm Zo1YXT+JnUxXs+5m0MVrP0UXHj78nNnsq9bGFwHw/lXeaR7+Bul137PEnk/hNQIbezvRuzlDk5A5 O03iW6pIZO2cUo/39TDjlAa+1vHh/tll2YNgIs+Vj6StyvnvMkQlDjzgdpYNJnHT5PLfD1c19I0b fTzaVB19wr6i8VDN52FR7gq2r0EjtHh++On9RZXajQj3nbM6tsWl5fSmhSql01imy4HZTrNaDekj M8/qUu+MQpMYPFsYZiWOQxbkTX7x/tHh8gGtt2yq5yPdW/YCZ1vijy0t7NetI/0XDOkv09WO3zeq yEZ5jdieWd4NprbXisZaO+wnHQg7ORCVwcC1iXdNMmPhyb1ZluXt82lC036Rp0Nge1VaTPaRdm4P oy4kvfoEIaAUcsrBSEehWVtdrInh5kpflftpGIo5I21E4nJDZE9QY07DpJGMBSmV/6feWrfvYNSe ByqGtpBJYtJjz4Cq88tK23jwMbLvlScJ7flh0qtR/Qoe7Nr5dliywFHjhtIwQdxJaGGi9NuOTp6Q 8OwAdZOwcObNfsHqo5lskx408/RPjG2UBsq7JHJ6PZdGthvtiswxwmEseE/6heJkqSBGJb1Cq8Yf WJ3thXyB3zWPpUnfpdWVm9UIw1/m8aWwjQyyi0zAr+WI+kui+U8EFa/Nt9//uIHwzT9IWyttWese s7M1oGIlXZO7wVE4QiqC5jBQlWFqhjbXNJMc/8VtVJLFZF6k0BJ1bFcnc7d9StPKSV8tDm0iR7rL UlNu84EAI4eVP2OIQdoPfG31mG25hThye2zU//Qy4kfzHt+kq9aMweoUS0uiI3BZdlIt7Yf2J1qX 8sr3/HrpjMIxQ/YEoo7jQy6MnBSwNdDOqO9+Gn2wLec3QZj3IUOofcuqw3coozf8Zt2JE+CAShBJ XxHqC5hv/h4US1xN0TdeED9slOgwLcEYF1r7stsMbB1W2Scm6haU3JOpvNU14ze7TypEPwznh7p6 TOz1JdY/Nh/L/FysYMOtpcMo0V1NxLWyFyYqMpfjr5gNv3Zk2PjYZzZY+YEBDi/2EyNhqyF/prno lHGLbn9Gm61smCay2xT567arsu+aP8GUYPdFNq2O8pxQ5DSFPeYh9qivWDk9Va5/aZgh1syeO0KW wpU89uL+1Sv642kRT0gJF6tp1BNBTE9Jn0a5K646tOHX+uGXwH1JmVC6kOmVFomyjGsqt590yAmu Zzgp8rj84MYFYtWHjJT1wCCpracvRvl6KY9k7+ENbfI8oGUYtMV8G5KQ8fUOwFibqRd3NEVhDx7V +m43bNw+OFiaaJkuoFMjPvFwWOt3RletYvPXpHD9igytZ8cuwQHK7q5UFi2h3T/lUTWNENHYIaG5 6Zs1UELcMaGi9zmcfxp40401D3fwCr7FGCWmv2sJZxdhKH+AnDDKI7ibqz1Sc8szbXG4xGKc6ZiO P2fCA96RiitlfUBJjfbJbNfmdq2JP/t+KHPfK6KSHaNZhEy4Pejc/2ETroWBfnO92hay4YkRkYkt NVM5NOaOE5F4Xn2ewJL65gF3q4Mr3Yk+AiwcEcu+rh4/FAH7dIyFxaL9tJW6W1o02DhY/xIg7h3G u67MPT+BFyC9zUEvvFTWQpSaUHmHl+OKwxFT8KQ0N6KYN0NrXhQrU8fC4lUgY8+/nSwjPvFuymdI TGUYFvNoAlIth6CyaQXBHMz44YIAQWid95d86aKkiuOZZWbqr5CkeobJ+E8HAXfps6OkmQava2Mh 88WhYQiuc8Msm7+yndkrDhWwIFMBjiMjOHrkjsC0hPKP1Iqsgdz+IQP9uopluUjU81MGzGjzKQek orL3s+GAbQJXVzUPCbJo+aDQco61ODODa8l9nQVxCebHBEUnXSHrtSozVpifCfkTkCtSejCdm9rV UhrkS6It1Y1sb886DggqifqZ4CK/5qM+b5lfIBLD5Q46rQKpI3HkppvRS+ZaPqoP4Nw2+RhMy4sA uSoJ6OBelMJCCO7gSjxqEsoOZe602CsWl0q+poSIvJeGI0RTb3G9+9iA5e12oQQR8AUbT6aLxJJn +CZ+n0Oi18L/6rW6pODLTxs077PxsWGqbO4vECdjmjDIjD365rw/mmLKeI1TdRxRFNOzVmCt4aQI 6nX2OxCc9V+Chc3onky+tcXHZThhn5RgrvGWXRfUOs/oLo9kzDwrckQVBOYqwCZy7jhi0hGJUbSA 1w2wX+p5atN5xpkv0vs+qtsRZ/R04DnZdQwZCv/aZBzoyYVystqw8nkQNfyA8qRTVCePbo+IbrhU q+ezffxVyKXr/qraHINr2p6LNUONaT8uWZ2wrJdWfsvTz64zAebm9Vcf0nyj5Gr2Vuo9dMKLcBku GTF/8LShnSC3pI3a+uT+0FSARYmEJ5zy54lH6giTPgYbereTV6sGSOZAKxZJMRx/yClGfR2qe3RB AMchsXfAq0z5g9iESiZGvxyPSyEn86F8JzX08UD3Re5ga8XrhcmInmVSgPiY7nd82FtBmp9anfzj CPEKJdBPATaxtK2lIRoFsiJ+L5Bu1arq4Dort2+jE2H6WkxDH3owpmD672X2zztI+81EU3VSebjG uf0z+AWUktHZeTjCqvFn3grlnCSjXORKEscYmgfYlUrtlagzvD+GofUkJbFnfZ8Hr84dXytXzcnj xnvu+2n+C5vKPgoRfICDOLclzBBqtw1zYXLhhdJc5EesN86nilliJNmS7OVmUkujKzVEsEseSG2S sBZIP71N1W9vOF9LkCzWYSYkMDsC2sIXxI0OiYcutAahdMagOolmdS6QDPzxreB2LjbC8mRxlALN L1tbX7dKCJG7TYaP9u6lkEtRrzPVEDgfcwzWT2Qvz2CmlLPEEEXa9BLS6JCMG8C+jjh4OqZyz7tV 0JbhJK65UYucPcNPIB+GsWNFiZnGnn74QvEM+j9RXNbC6S77DastlPnLD0m20K6V9y48pMoPlMtO MoofaUX4iW/DcoNVy7WAwacSMg/6d+uR53Ik9QuZ0x0r1V6Ovgc4wzP+B9MxYRasyLJmHbXp1VNf LHU6KaO+2wfiJBVC9ojFQMx8VoqRoBxZur0GIrGpGJ3DiM+X0g5TUNGuiRidFVjV1DIuhwyPAKSI v6j588EB2T4ewWvMNZRcPUSwk6NjhHp2Zfkq3A5745Kp1yoIlUj56SocK+QCsQogVw5qm2cFs+sC VhCydzQkSSMoe5ioOIgziCpvA9yX1x4/dYitJO/gWtuZagzQ+4SHX57ELEl6eR9BnXrvBG+glScQ tvI3fG3+lLKpfefQjpK9E9PD+TMaS0Pwl2m9UXJRXNXhwUlQC3dpaCPM2RO2BW7RsUA9A9VV3w9W MK297dZSRfbLRHEkMd5pleSnbHCNAxqyC15d8plPnP0pi0bPhmrxSAZQvg3TkM/GO5RlZBTYOvnx D0QrFl9PPo26PAC/VtHhchxil100CXEuyEqGBLRtSXNV6owPMkdknHEp3G5ETGzjXd+KmNVkEpet 7JL+KCIqlhWoK/0wKi59HDitVpQE8RUa+jzfby58jkrYZmw+x9Yik7n353hqb+QqYPqVUTRltcDk xXYL09AlfQ1CWpy34RXTXfvzQ2EgvgqE7g5gfAiWFYvKo7h496JqUlOntZS1p+e2heft6RtP/Vli 86Z5IlEnKSf35/vnroCfJH5JMGikASaHAQo4kUc7kx197W17t2nBWnJjlkgzZ2gd+8fTRH7e3906 4koJ9e5NVV+YHq+2aSIOIhLw1vz3qJYy7uIuBac1tqlGGmcaj8IKkpwjUpZoUbg6TUyl3/1cONGH +lZ2pNb2fUiAenb/+6Fpy6bKDXoQ0VZt0zqNZWdUdNEU+TeOWvruiYFimflJcRwLfdsp/nw7Ag0q CrjgYory2L5PlBdz/S3mft2TijYM7ZOL16dg8ghsHB6cJ2s/N2UtvZVSH449z8Csrz3GQ68A84e2 QslRz5k4/DilWFQlYChUrgYd2P6VJctbp3j3xZFLK6Ytk7G2ARRVtNHhN8NjXlRTZzOwTntZV2ZL DY6A6ibuiTO9z9YzEXTzgmPvPBCMii3lqC+jFIZECk0hfA9058WzYa1Zk901JRq8DnYIiWaDms3A CmGDS5yT2nxyiZsGuZ84MlxuGcO8IVT2G65Gh1AvVcBOhGK5qDJQiYe1NHKFXRCvCSe4cUnyWVAD HtMviVEfN+x7Y2p2wJIKrUx+1ofUuY7rOlu4s3dXSClmdMyYygV/CZShYgGG/opC5yrCIYojMkfM igqEItH35LETcO7PLonqxnYf030n2Dljl8Iuj6Dwctfz+aCDLNGN59kUcIhZ8WXmoz3Cu3TPAUtB +d6103dhKm+iT4LVAbYCjHC1GwZ6lMmMIyeeR6lEZbO877EkJgrPK476U1DTJdMoeC9nLA88aUNO 2W7bt4VKoRGpJcnooZ2jgndXnc6b2cbWsSPipkpnBbjntKqyWtXG54b7cNE55nO8ObUK+KeqhPmg 5J2GxUKVTrhwRTob85Zf69T1FPaRHCdTtVl4KKBjHevLSzv2VZLx2H4ZK/obr1tWpb5zeOvoO7b5 QYUmwUFvcEfqtCT28f5YoCp0wypGKTywZKjd0+izBMV05+zZazd75+GoZObLVlz+vk6tBhXLUSSG bCo5KoMoQ0g6E86XznUqoxM/HhfBytg2MpOJCf1ZWVHY1KFGwhfRqs/vcHoqf0JatgAieQsSF/Wh 78Nj3/jiYKJwhvhtDzRDVmK466KQd6zDWJ9/3byTVUVKC87H+FDTsypkTTTmrRV5Rq82oG7/VgZl fZatUN36HUNW16OcPJ8CHYdmhMAtCrb4AHsHWy3rGPQwGhvVti7eTYFmyc42dLcZX5dQ01NFFW3V An+JYZTEZBs4Ri7dolY1Qaw5GyqSF3i6Xy0pMRgjAddy1zyUYpFBpcweTplT9sV9xdZlClJIGXDK tLhRkYj+OjUHz1wrnXhLJyxgr8il5do2iRsHM33VfTDNJU/8ulO3IL34xMmK+yFw0nI/cmPV2msS IhBvo+Jx1QAL+rovdMiymuMOe+aG/sFWntHFpYnA5vE2IYR9necHRbfksWJbCMFPpF3JTDduJRdU wFmFl+TlfW9Gk+aX4Uj/5a7+H6lMB9sHUrvnLTAABCIGLEArlh2BtXiDu/G1rl8EXiF7ccTtWTuU BlAo0muv1g/tPW5KFatWQ5jVexLOa7ttmVqARwv+zPWdv7abSFI0z347DsLqiVOKW3ylHfqkxp0Q fwVP16j/pYWisD6n1hnTZU1Ntka/GGk8l2Z0uXIDFWQZtNYtrsEepWmh8DeMtHs0lQ8v+FgFxA9J AKllLcEckXvrpH3Y5OpY2YmdoSu98uSF8HSFImdH1K2Jp/vyYVTpoBJpZqKfljqDr6vNJfziPVKJ KQosseJpCeI/a6CYjrFnRwwSWqBUqj4qARpov5icZm5+QFRHvMaRF+Op3pu54VSPbKLd0PDVI4eW u+z1TyWmxy9o38clGzza37BTqnIx9nScjFvxRQJfevRIRbM6x28sfosyQY7A8zc6yuhT1UnVblMI lNpRia+J+6Tz0anV/fzVwHDL+v6TJUIq8uOMjpBz7av1jErXNl6/khzpLKxfhOo5cLrp5vYifvm7 0i8G+ztWa0AYn62NsRNZO5brS0fK1QxzyfrwphSHANIT3M6GPk1GPjhBkNja5X1dtCzs0mAtmDgb WnjkXahU6rJntVLEEvkNeXT9pzknhnhPxNinVaAdhz+ZsQogtDgK+5cWYuAPwmC3feRZ5Hj6xYvm xVF64k6M/yfGDTnySkeofLX3PG5dbh9Y3LbLLXLulr+mKz0XRx+/rLwO0gTAd6EbeRh3zICrYwBb Z7cKJaPhkE6g9TcrCX0zeZkHk6ULc7xx7WXMd5P1XkCpFa7st1yKjoNSkM2mxWB1h9JRbPe9Dqv+ idRRRqslp2zNp+Nd0FRRA0FhWJMcD+Iq8Tl0QgpIX2xALqzg47ym02ng5zVTPwQokZwkTRo+dppE aypzUYofIWJvBKAB8nnV7DMaxD2/ce3kxKQ1JJI/irePEqVr8bNJ8XKZ+MUhkrCORaNBx6RZkwWY QR713/+p9fkS4zJsQA3Ol9M4SlbuvjPaH2Iqy1H+YOO0dovsTbbp+n7j8QJK36sZrKub7jOaA729 e0I5wH+DDrNaX3HjjMpDw4FQv3wc3yHv7cXzZQ8FH6jWVBMrfQYvuUTsXU4g4A3TPX3Dgip+a8el Vv2jp73RZSXnuvBtyaMHmgL9nb56CkwQcYvRamvm4aohpTUlOGmq2uGgXkQaq0JkOnjf3R9fweKS ANwYhQiYFHhi5se0c8PJ0g8zRE2GNwm21k+sxru8VagUlO72YPGm1giHXpi2DjtMmNyy8TIJTxdh BcnMH2juGQ2U9M/MMeq8cSRoLGJq0HVWFmLPOGqm9phenulm0uzhU1T1dJYqF/I5W2hZ+WBsOhsJ ilq7eEhalmsTbghzIw4cSBUkV+oO+aFKGnvnhDlIxJg+6Xa2r3g+lUM0ms5PjDGXheRpcWmE2QM2 A+f2A4nlmWUMgtHgaFgwfZ8vGLUnbDllELZMvDrojaAQtemyNPvZXNze0a+OeU1NSIxqibRwCV+c 1gXToLM2kKVchFAtNoslBR+0XAr4m9nlyBeTcX9mxFXVwx5JdVndUFduF5moQNZk+cbiyOV596c2 fbSoZO3G8n+COIdNjtux2kLQTLqJ1TP5K0fbUR5q5xarrO4IC9cUyrZa2530BLg2j/wqriSBRqYW Ovrs4hAI6I9CjT7usGqcHQhAO+pGohzRIHDarszPsJzY0z0+fglW5fK7LH+p5h8QU5pby6x6e+K0 1PAcGFgP1jbpdcqB4jsSqI3LKPeoRY4FjRsP0sG00Kl7RiArjGxFe3mg7jN+j2i8gkgowX+9WfWe Sosqq8r+7R7JeY/mWHinDO4W+qDFGspnnq86dnTQwd8Abj0HFRJLKvEQwzuMSYU1BvOL1E65/UDk Uy7SuAOfQOqUOHy4Dx9540PvlD2w6sEthA/jJgyWY/GzHJQvLrrRad0mEUYi8r/riotv06Gt/bki 6+N8MAmN0CMT0hQecKeIsKjY0AtD7b06fK7ftQ8g4ZCc0V2K+jfFZ41pnPIaJx8G/Rjo1EemUOKW C8UO/6Fo3uNOgEKvKZ+pZ76Qj6xvlp0UWjEa7jlapwHbH0z1m9sDhp7Tcxz/pXcCKzwMZ9oyPkAf vQg3vqkL7abgIub9kTPVuKWzvtBztf9o6MZw5TZvaZjfBNCaKzpCBweWE8+YlhpIEPZaICPAuIeN Yacr12yxa9gbetE5E34emSCB5hzDz3Qhf4nXjpb2grTJDwkG2BZtUECf0AdVBqBvBi1MPhgi47TT S6GXe/sYwCH3YWleBu1kxQOyPDRPzrM/Gm6r++6tZwXC9aHiA/VtjJK8TSMSpA4ohLghLQFnEN0p jOiIE1hPifQC1ZvPaWwqkZr41bPy/gvJXlI9U+EZKfIQeKsVI9Y1UYTWgpBW2VQ/KazJ5LuEv4Kq KfklukG8UF3iiz1I0FM6m7mNoJ86ehPo2LbaoLGNlB9SEFTABHLGuTCPoCJXHDYl+qgMMpXU2wcv SPE+BJDTh9bju9iV03JiOWMcY+OLSYw12IrSFspxUPUPMwIF4SoNxVTbQNrEUUtBqDKD1oglYFAc 6brLB2TRNY5XQi3Nn5+t/N8IBRL01CohNNvoKafks5BCc5gpUwhSM3g08TJSvhhuUB8eJoaVwrwp tonu/eXXN695BzzZB4K7YQ8k07GsRt1qdV4+D40LrdO3x25ALgjwoFZ3ioZDsQuXVixoQ7nHm+zy HuAJ5v/wUXWdF7IkRts1iJByviQejgY2DIUsZ8r51DCJVSpmWV9ZsegdYYMvl82yfVIzTo+kHuRo fYzkWyKQyb5ghNwCkVqk23nKVMJEnmamM8/OfVUE+eyTLhvvyuo+wTzs0qHXDzwl25oEjzQZ+ED+ GM5vncc5rOq0ZO/Y+PCAXVzlOtVxYyhTgsNFTA//p8nSe8/7yZkpxK67jEoi+V2+Qxk7QOxsafRy dkTNynz/6u0BR3UCi5M5ZKNqEGpUwLWuzbf0Lic+VsfBIl/8x9g58BQKzYc4PrvdDSr8qCJ32/5X NXGUNhiouFvqBiXclK4+vxagE8cCu8UpjRM92x7UaCWU/D30x9JZY24bTeVPEWeT86myDkGvDzpn 4eYJnGxxDBpQDA3eM298rHyLNNtMu/sYRvV4wpQYSocj1cdKYWiFfpGrqnIFVxZPib5x9B4W0/R2 uwCLzbXSTGt2lCHdVkFvKMq32NCQeImz8RHorwDoXT+IE697lzvQB+IqB1MQZw3bCctWjavsB802 TgvlivVT5OfpiwAkpQsX8VpQCbY0mB8YSSrDGKXOfVBccPs8TFEAi2IBPMmkM5Cp/IQODtLBiwC/ V43e516PC7qcjqkaJ4uTBz2cJ/EIG7nIslqtVFMb8X6AGvXzBhNxg/ev0bOT7sva6EQJkoIF70cl lS/z/KUAwBwkLi0J5Dn35TdkC5pywNTJ4zO4DRFE6r5wTnif8mnM/AqkzBkpAYCW72vjFhiExn+X LwixTbk/hYfnOgtoL05gYzVmckgfx3tcbdhRPjHwiMey/p5HD0TUm2NkInLrKkV8Crow5zvjVKe+ d7BFkmefF6LOofr7WeWjNEuUYVOJGYP1duuK8QUcX5MBvd+qUuXwkTwfU8kopkI6b5p6q9JJekBQ qy0aCPaknQ+oSgcd2n51c4FBW9+9OgAmTebRJqpEO2tz6A0TSzO5to//d13T0LPRt0ZYrIBH2N3P FicMDnvBla54YcWEpWySGIA6lM3HoWkUuY6CrDN7ZASF9QVnP+z2wb3c+kM835D+ByGoUuXBP9lG IAXjnUxGLmCF9HVpLAE0FwifXyGEBD7VOFWAnFscxeKeqcevcUbitPcdw+gnHod4WBLGajN0Vdu2 5yvbT8UIK7zv1IalWxwr075SC/wdtodfI3OdkX5g7MhorJNMnBShKz2nnpoBFMvNXvdtKA/YCWeG NhDBge/NnC66ciNdMv7upjEyVtOexG/INn04ViAgwfTiXtB3NnMUCwsrsJngYGed8OywjsuK6/xC qXrTOlsfZ/S9O/aSyqs5GSmdiMOF5I/sZDFousrZk7NTQ2F6N/CeE63szDr4BTK6Q9NHEWIzh8Zr a1nOic0oaSBAqEc00+wp40yRG3+10pq7qLpHEwyvUXdEx0Sf85d9dwcB/wtvv9Pzf4pZ3LMaaDCR v3Z7WgibYRD2pjox7V5VTskvgdYgUSUYEnk41YOebjCMgEpt4XKb3nSCrCPpPQU2HxOJXTriAGUF 8ZoYjWeVAmNGgZtCrjNmLwEsD+izt31UMlcadd7Z2dqHn7GVwOeULSHJYMI8fQKAZhysLLFBnobK Hq/EHnnvxoSr/QsieqBOu3rdq7YTDaHGJVPsWldEXTdoM+GxDwTY7B0YlPjF+1hTAaJkHwqClsb8 Mi4MMyuZp2bY/ORXI5t+atTHx33ei+4ICS4TE1JHw4VarpG1n/QLRyV532+3PZf4WgQuKjvBk3A9 imy5O+4tN5Ot8xBcck9HSd1Z41PVd45es3aoYcbFylAN3htZuvaXdzYTq9HZZeBNEp7sA2vQ4kH4 dMrnSm7nCXV8BjRU0TQseu/dWIuN+XfHZ+FzmVCkzfoH1yLc2pK4p/O5Gd1HwXXqCEfMthIqyGXX CVpkY+qINXVbStuD2DHWo0R0FtRh+p/Cy6srbKLoAQ9HZd2nsud+agMa92t8errhjDQd04i+jxp5 sZYcOHL3WzdHjvdfgENZ5DCaIJGkYI8MZL8rp+BXwDzGnOpjKshrVp4fnJafPPttJHz/HgwIged0 a7UKxXm1TMzKqIs3g7rPEKOaPQWJL1AVM137r6NZXUQRTxgQPTf8nXnw6abU9gmUJO3xn0p7q3T3 y1VNoXbrFjQNoB+gU4TkgGiwSzQIriUrUfidInpCHQxSU9++hi2+xKyuBM1yeG4Uqnun8B415F0J v4a/OAwo6tqLIqaZcqW/GAo2qiGGFjRvxNAIUcd7Akmv1yBZUM+eKAsVimVsujl2WU6Eryns6UdA hEBHGS+R6JQT0AXOdNLRRpN8+pcKbLksI0ZT2BSB2L+4tc4Vt0V5jjmpr+yEwiFJBFErEzF68Cm2 ZMXbX+I6gVdh+ejgj7WspW7q7BNn6i4o0Qc2wQfIN6sbEtiA7DdGhIExXiocQBMjbLzHio+TYa8H 2jK6lduJOon/bTS+YOBqJgwA3BK2LApx/ASn2dFlJrbxUPJxOPGFJWyGas2GS7Dol20VvdA3iq3Q n8NjZCF/SsUmb7L9EyAFpul98M3PZS+y3huB2VXAAfthTeX1CeylOWsDubYLhUV+86sS8eevHyK9 MclOBfULLBL/J5qNnNa+kWQrdhhTWZfnVP3OyO/xhwQ/Fx2bI837MUxtKAB1K5Y5BnFL9vHBjhwf MujHmMbJh4oKZW5kc3RyZWFtCmVuZG9iagoxMjMgMCBvYmogPDwKL1R5cGUgL0ZvbnREZXNjcmlw dG9yCi9Gb250TmFtZSAvV0RQTUZEK0NNUjEyCi9GbGFncyA0Ci9Gb250QkJveCBbLTM0IC0yNTEg OTg4IDc1MF0KL0FzY2VudCA2OTQKL0NhcEhlaWdodCA2ODMKL0Rlc2NlbnQgLTE5NAovSXRhbGlj QW5nbGUgMAovU3RlbVYgNjUKL1hIZWlnaHQgNDMxCi9DaGFyU2V0ICgvQS9CL0MvRC9FL0YvRy9I L0kvSi9ML00vTi9PL1AvUi9TL1QvVS9WL1cvYS9hdC9iL2MvY29sb24vY29tbWEvZC9lL2VpZ2h0 L2V4Y2xhbS9mL2ZmL2ZmaS9maS9maXZlL2ZvdXIvZy9oL2h5cGhlbi9pL2ovay9sL20vbi9uaW5l L28vb25lL3AvcGFyZW5sZWZ0L3BhcmVucmlnaHQvcGVyaW9kL3BsdXMvcS9xdW90ZWRibGxlZnQv cXVvdGVkYmxyaWdodC9xdW90ZXJpZ2h0L3Ivcy9zZXZlbi9zaXgvc2xhc2gvdC90aHJlZS90d28v dS92L3cveC95L3ovemVybykKL0ZvbnRGaWxlIDEyMiAwIFIKPj4gZW5kb2JqCjEyNCAwIG9iaiA8 PAovTGVuZ3RoMSAxNDEzCi9MZW5ndGgyIDYyODYKL0xlbmd0aDMgMAovTGVuZ3RoIDcyNDIgICAg ICAKL0ZpbHRlciAvRmxhdGVEZWNvZGUKPj4Kc3RyZWFtCnjajXcFVBPu2zadAiIhoMLoho3u7m6U EMYYMIENxoCN7gYBaWkJ6RApSYOUlhIJg0ZCQAGJd/qr/+//fee879k523P3fT33dT/njJPVyFRQ 2RHhANVAwFGCICGgDEBV30QCAASKCgGBImScnGYwlBv0Dy0ZpwUU6QVDwGX+w66KhIJRWJ0aGIV1 00fAATrebgCQKAAkIQOSlAECASJAoPRfjgikDEAN7ANzBOgLAXQQcKgXGacqwgODhDm7oLBV/joC eCC8AJC0tKTA73CAsjsUCYOA4QB9MMoF6o6tCAG7AUwREBgUhflXCh45FxTKQ0ZY2NfXVwjs7iWE QDor8AoAfGEoF4AJ1AuK9IE6An7BBRiA3aG/gQmRcQLMXGBef6hNEU4oXzASCsAq3GAQKNwLG+AN d4QiAdjaAFNtPYChBxT+h7PeHw4CgD+vBgASAv2d7s/oX4lg8N/BYAgE4e4BhmNgcGeAE8wNCjDU 0BNCoVECADDc8Zcj2M0LgY0H+4BhbmAHrMPvxsEADWVjABiL7090XhAkzAPlJeQFc/uFUPhXGuwl q8MdVRHu7lA4yovsV39qMCQUgr11jPDvsbrCEb5w/z/OTjC4o9MvCI7eHsLmcJinN1Rb7U8PrIrs H50zFAUQBwKBklLSAKgnAIqGuAj/Sm6G8YD+NoJ+qbH9B/p7IDwATlgI0ECYExT7Q+bvBfaBAlBI b2ig/38a/i2RgUAARxgEBXCAOsPgZP9kx6qhTn/I2MkjYWiANRBLPBAA+Ovz98kWyy1HBNwN84/7 7+EK3zMyNTYz5f8N+G+TigoCDfAXFAECBEXEgQAQSFoUIIk9BP47y9/4/8L+W2sEhv3ZG/CfjNpw JwRA+g8I2Lv7C4bPn5zg+XNdeAH/rmCAwPIYCuD5h/Y2QHEgBPsF+j+T/3fI/4/zv7L8L7T/7340 vN3cflt5fpn/HyvYHeaG+dOOZbE3CrsR+gjsXsD/29US+scS60MdYd7u/23VRoGxm6EMd3b7+xJh XhowNNTRCIaCuPxBoL9mgE3vBoNDjRBesF/PDEAQBAT+lw27axBX7FPihZ3UbxMUu0r/LqkOhyAc f+2ciLgEAIxEgjFk2MFjJXGAPwi7nI5Q9G9eA4SF4AgUNgSAhRcIcEIgyX7NU0waIIx9rH4pf8vY +QijXJDQ/9AAsRpfxG/5X6Uh3kgkdh1/UwPb11/y792HQtFQCNn8DAIiG/GgMaLjpF75lq/g6qj8 FOeqZRavoP88stP7jJI4nbcuN2wZ+V05ffDl9Q+f1XmOld6zXPhvtzURR7c/Mn7xM+DcLsVkYvUF 2dw4fd9Yybbyszd3SG8LmimtBVx4BliEuuK34XbrcBZ6ektRGhXTnPi+1kQ/e1O58DZqZtV4rU5C l/y8clIw0TzBJrT8HWeRQ940AxsRSvAOCd+NfTTVu+PvUzcKxq5YdFL4yQJ3EkVL/a1WRJJOp/0W q81EvHoYORitGO7gH994O8Hlr7KRoXNz1v9p6TLybeY0fefIR/c0kNsGj9+qgckJcvPbfS7OMRke RgZhfFrt+ITldq1U93LWVCKxvP0POsto7W+SbQ5eAoapsCtDN8lGguqpMtY1B9rbu/T3t5fXJppG BbfUFShFy1GymsQJcquybgcyr0hPSg1N068XfzTlM1R8Jd1ROLY8/3El86j+RsFJAsvEbcHLaMKf FUQnvgCMM4MUjp7XPdRMq3oauadWvQewRGE4mrJ/lWxDwHNlayGuLY3KM7k+ipa02p3aSs1Wcqi9 zdBVUNNu0eaT6TBNPUYz91gv/+giLScq2YBJ6sz58Y6IMVGgzOarkutrAWeqwleFhQVNb5l+vhtM cbs28eh1our+ooh9T0iSnrfoyCLVygt6tVFCEj6qadX+CnHmzonsVa93E3rh5oyuDO5aVdc0PBfM nIasODaO9msmA7I/rLWKvEW7va81jPaeRXelOmtuCAfgEsnscBnAg/c5dAbVjdf2MOfrS+FiV4fv FCo8MNxfmEWzwq8ewBaFFs6nmPrL7hIMSwfFfR4dcQCSe6/lOchqVErxGGncwwuYpVUz5UzFyF9G OcNZjUOtMqLHs/LrKw1hT2Oivq/VPE3WEohaB0vaH2ckrtlZyR3QOOcdWHVfbI+M3GiqzIISLRcU BAdPF4R5pscIBxE94LZWBy2S3hr0dRYY0K0JSTuiRLjPKOhLNcnWIysAcap3afHvNDMnR1tQu9LR Os8JCkdUH22ZKE2yzAe4HTjgqIYKd4z5hdXrFjHbEVTW4sEEuGdAY8YH2ZL3H4m2q7edTFXu7QTH 35HmwO+x+kYz7YixdUMxJyGvF1LLFzDk5AwpJNDQ6oZwvp0EXwd12uzyoKg+0wupM/s9tOSJYQ/T N3ib0izxPohvVl12pvtqXLWPsbgOfB/0AyNevtjN7dKGcSJr8KK+/l5IKnSsXry2BwKj1uQQH3YG 7umauV3ie5zPKs0r2DJeOX2ax7OcXI9YkDR+qGmxnbZSDS+5HjRcIG9LuGmVbJ9AXbz7KIpvDS9I knyDLZ+qkOTh+wTI2twP3zvA1iK88rgLc/97eXChITu0drdRYU35vTJjWyb75ykVBH36BqzjRI/o j3s/3vXdlNlfe2YiG4WqyZljyH3Eb+ucUr9FY/WOdjDykuIC4eVb+WjD2DyUN7dFKr52ABz3Sfk+ uN9HAd2X+HpYK6zDDj40H4QTmh0fyLjHcvUhON4UlTOkmRcnWX1tUreDQTJGpMspr0Y1fw4G1KRz B3krhoXfnBaL5cqw3V5/n1XnOHKlEfLBw32LhrjV/EYdlUtr4VRjQS8jh2+pTZ2OI7EUf49pobp8 ec/d2Ct4y7BFCufUguJk3UyB5pC1WHjCK7Ux3oWvkw+JUgK3wuUSn/PucHpcJatxeiJdJxDfIu7y cBGldPU3Qwji6JLl7r9rNwyX1dDGLylqMVHNXtc9E8RzOn7Qcbyb+cFvlPhFS9HEOPMHQO72zrnL 5wQv/hW+R4zmgrl1LMtUNq8veG73yqtZO/IqnS9xEqBCjjN42Z/Op8bqz2cqy5+KG3A8ZQAZZ7rW ULlUrb2+EXC7LriT12oAsveBbrYjXj4wOX1Mouc+3lvfEm6o9zsnQq06Cg9WtNBRDzGbF+vpjOlT j7lGHwMckoFMjFR60HfOnPzoyHF6Wruv7u/UpPpbY5Cb9YPPXOJ9Lb3tej9PS7y0pm9OlW8mS7Lo t719r6pVRJGzg6ym/c0Wb5ywrklaW3AAVH53YYSwEr8o1gHXLuKqGWi6ETGUId4eFykKmut2H0Sv KWRcm85qMsw7o6ar72G+y2r9OWx+rpAgyf500IfA6VtA4EeTs2RSMbZ4eqIgXmoib7qvPteW1TnM q7K7bOKfW8puJ+QzTeoCa65JvOCYKS1fi0+uMXJVfDOCeYG5ihGLbKeRI/y+uLQ/nOopr6rJAdi/ mdGq5Djf20DYEnQ4qAwylgpTFgxYjnUA2fvVNDzAATQlZyyUnVvZXC8W6kfuVVwrDPrS79Wdzfbc YeumrQkmOzubCzmgBrzuopdjOs27Ve0nob8m/ymT5pKNu83QcCPjsg97/6T+IvzpHNenjgLr5WZX XqR+G7K0/954+u4V6SCoUZ57PzlXYGbRpeLVt3DXvofz26pO6rmh4mcatnHj8ZgMuEvA7beOJDeJ icaFebqDuC7Xe4zqS1Wkn6teoUMSpMlOl7ZIH3IyfPwhIF1AjEs9ZkiiuzP1VY8h6wOBz21aLYEE WoXJpMijhNsCEk1peGz8an2th0gXPTlbYmV+iTWdlzrPpddXF7qTd+mCRX48kGMVdhOpepL58stW tzhTX3pDj/OVBW5qP+nNVT5S2qVcKe183TGIzVHujiHLeV551FRyYKBZxtYPusWfNNJ5t9UmeTaH IXhT0ntFWV7QWcbIPN+vJtFSkousjaP1iBSv1DYGB1TatSCVYoMSI5aT3n2tu84DI8TsKs84R+mY U6ROo0rSYZ8K0trSdo7OnUsXKDTT0lIk0iQvr4My5UzYZhK5neA1giK5ZlDelv61J8qWfQGhonI/ 7gnkpzefqjlc+NbGBo+FiEQCryx2P8xyNljV5pDL32GqvtQeiOwvcj1TPf+6J1UfpqRgHMjsEKlR ZE2wRXNY4XfI7VHIZdqJw2fexMH8bC6D+QlOUhjY3mRCW3463/GrT5ody5wN84JlXsUPlZX24wY7 mk7x5vI3cOvsg2Wub+ZbF813YTX7n1IMXtRBvEe7kfs+N7QzbS20ny7XRtmY7u6yaSfVEq35UAYX eJESpXre88glGjOyvRIPuMIPQBq+j8UNwYVANlWAdIeJuhYBs4XbUWI/loCq4Acen8q2vQallPs2 XuRlrlN+Htl6flV0D09McGdqW69pdf3as5G61/t4MPK9xV0NiwZ/zj40/i11QrtrNbl89uN6hq/M dw7J+tN0u2s0hwbveAUEV7rrPHnJKc64WJc6tlfmqOxDI8f4YKW2K6yRUNCSIP+rzDKVgDydyr2t 6kv3osqVrB0oG1IcRVlDVVscMvj17E15I8HtHQWBFd95tLYd4Bj6zCu2hfrj23l0hgkKp7ip1Bga 20HoInsFELRmytV0upM2GYHLY1/Ed3og/LlDu9Hfd18MlmUIZvt59Tpxz4amYKWN01/whL/ozL5K avGjkuwrR6ui4AOmAWa8W0u30AWHltwMK/vOyrgpKxz4Pj2e45x4NI/k6ZWqyAFGGFo73/LxzfKu EP+46jaPXonc5YmdS0o9vGL9jAv3ZH32CzITssjNe4+DmzpyqyY4xgt3akL2oo4IbvUFlEm0ZQew rGYnDzzUfEirndli/9rJ1fMWOFAJ1UpoxZXHnfJS/guJaGnEOwcC7XLn2+b54iTh0JCagb6LCuXO OtwvhYE3BM7Lv/Gu08vAGEn6IPsuVjJu3Zt1k2HX5DRyJN8RCPNLHdZ12xRt91oIPct9iLBxvVtC DMXtKHxVS8B/40nfCJOEb6lKM0y118PwMXnDDSqNod5MA7Sa7UiBHPPNl/yk96yAc+jR7DsDWwTR 1lZBM2Fp5V8iu7I9FE8OHVokpJNDVX5oPBVg0EySBO/e7LUgujfzIfXnNuo2xUv8mO/6wT4QBRF6 S/anotXA9W2kPkcNmPSKQpve3efx+Lczw60utqcCZDThMjyFnKrb7Ut7FfqwFqW9EZvjPOsGoiOy D8eJsaj0yRU6+Y1Ec/0DuLWKHd2ASIYdfiKlEtw7S1FvUoBOzdKc9u1lwjt0Kw6jvHdgOrt7c7rE nSm0MsUd9H1CY6uf/Jy0pA+3Y/n8b47FTbcgASZapNZ89zCZ7cBVWH8NEOxAeHHLSLJiEDPTFevh w+xabvb+7ayZCT/nG0mc3XMSlsYPrLEN7xmcTpo0qADOmksrOHlGPfEiiqacNAHNDpz1ny8vMot3 DLhQfTIKDOqGQ/d0dPZ0CK9KUq2SHlVuXqloCbLiBGE6Jhk9pt/GTVbjfi6MiFmZqXP8SkHtJ7xV crYsWrLygC6lhK+H544OrtHjNxzcdSU1yl/MTppbbg1Xotma3xw5ZIkFidnepZxuX0vt+nEnS7pW 2kyu813B0GPdLfwnkQMF0vTW2v5ZRmcK4Uzh4pQm3hAunt1K+u9TwA+VPnnUjQeFBbL7/puiF2iC wBd2nEJdBA4FUMvCXerKYhGoKXpHDTdMivlJ1S3ETvbGR/+FFz2bLRu5gSRE3b5wAUuj/ULaWr/H eBkJubIz/CYxqEiLkbsHn+VHyG0mvcvaVdKJ33w9NOdR9enD91XUmiV9gLoKwlErKGFAo76eTYW3 n8WpG6VaD6CpcKqmhBMbI4pEWsJ4ayf4V5pasqkKKCeGGgbRkW/Fklt+VFAmS2hrOdcj1Ge4o4vn 4I0Oa7de27jVvNVLdYHgbZ87CFNW6ga6blnkXC6pkXoXUkqw8y9sMkgjiAmrd1+OwJ7R3+Bxxoi7 mcYfRLf+9O/qJya36hxlMY6lSGgB5j13PPaIt3QW1Et2KC3w+PGTEWOz4EfE42xSLkGyAt3rH+2p 1k18fql5s0xe2lxksXztYzj9szbV+osy5sPkaS3yT70IjyQk9+l7Ag3T7UF+CvPOweEPHCfN/aPt NIzkbS6y86Tj2rhFQcShoxklYTLMCLWTAfPOGgb1qlONVgxCFvj+MuTOvuLSwuSCu9G4AhPbMHt/ et9ZPQVtkpm4Disf00nw6s1T05tK5fprkuxSXDM7gmXlp4WYUPjZa+nHS0enjybFW+o1IxrMlqbf DUZYBBMf3SD7UZX5ld8Sp8HUmslwy12ULsXI3TF2U4C6ij/NNGScofei/MBspyjK9QRzfN1EjoOc buTLYgjpR5ErMAnX5PdW+oLQOflS1XeF3/fqPxqMUkjTSzazVzZ+Chrq048/Ps79YNbQo9ik4kf3 bdZkYqStsNiYttGvqmLW9gaNV3yl2fb+0ZSRklFENiVppKZg/xLARFq/O2TVr2b1MGSVV5QqOWdX v+M7calWTT4x+tbLKwtu4tRSIZ4pr/oz2ZXhgGU7IkOzgul84rvbk9L2zJqEEdfOOYws50ug9/bC DxYm5J3UOd+VGDIxJnk4qpUuuh/5DRBWzLy2/dhGF9X4fKaU0K8nWjeNnwCtTWfhSuZL3CBzdKTf TlzoPjErNnxeec+ii73o9FN7a06b/nnxvm3ZCV9/8ewT6c/UMQSoUwZxdqT7+0/c6rNQSgTHaE5x efriJL+fFFxYXhH+YOIZQ/4ETw3x7OthHgH6nDLDUBFf8k05iSINOKQQxLqq3OZ3y2Lc5NYZ3Wvt 7Bdbc438Zq49iJvStVkLMlTLg5Y2pzE+yULX8TAhyuJ0aWt9L52/4TjZKj6Q4L3qQz+uxCywGjwU L453FpiSEcmLz4nZ1UjsoOZc955Wt9+t/YozpbKBDvb9QcKzWDcYlcPVRHk/53AkLC+MNt6g7Nhi bKNhIb9kTHlAhcKVnwvvpdIg99hEVpTXy/Co9ig72Xbi7OoW2jjPYx67iovWPJmvHULzMry7YfTM 1hOkTIoEDpeoN0IG+2VEAT3vjObd+wJf9gjHeNetsEsqs5lXveJtjjCzmbVCb2tRIBGBCJGFuwUT r4fBwqc+B52haqKu5ulH1j2Ts/xMAttVjTOvmBkYBIbxSQzJv4pxLkdzENJocZ+/aVAloha+0XTZ HJ9CcC3hvLh3OkCg2Ee+bS9wMyqXRTWmMjcXYy9xj9KIr3Xt6CH+I+RSjxQxJ60/QgySBlprNzQ9 UGlmdVQmJpHYrrG15HTugayaFX7LXSpornAkuFLQjxHpbbtv8CB4l7LX2mBaKb2Qz+r59ztrsuwu Lh90JdfZRWTrVBebi8N+NohQ+KreeFiXlMI2xdJn3YRRq9IZEGAfcQh+GWsVEWPBJwlAUy9n6X6R zY9PGk/DDPuEpEtLaPNqs5FysKa8nhZ+evpcqusYkcpjVIwx2eptog43N31iT0klx3JKy+rPGJen WTbeC+3gDwtf41ENe+atXP/Sk4Bx3G3DDOJtnUF+kv5gRv6heg7RwXQBo8iJqhjn8GFTcHU9nQi/ I9J4t32SvphWWLY3pTD5fcppoOrvP6FKwQFXXJCIN917iaCf2gN4tw0D23HFP+5yUYSxyiH17R7V H7ipTsw/XqcKYRokpJs1oGIv609OeiZGmabgI8Cvv3QA2lrv8fCItftcl7RTA9yHGYVcjqSn7yWE dVQfRCe3Opd2WYFiHsqERAatwXhS3RLiPsu7etaC66r3hXpAVF9y55a4ZWR5Z5ZsxVj2BbYlbnqj Iet688oN4PD5+6wVR3qUOI2xqWyF3vT5LI8/kQ3mBsmXjqmOi0ZYdwGEQFIXuB8eaJ+emRnOGVOQ GYdf0bHkp2E+8JSq0/H1/FToruZejl4OF4+hYHahCI6wCMlctBfYOPZBiprdSTKb8l++6hSlM+xa uNjLs2tOTJLbHo2YUtvhJy2/wl1yoVtiim0UZaYZCqOR+7gKE9ra99VikqmfGuXoIBFR4wltuSiT r6rZrZpi1kVU57ly5A3cKO/mk401GB95BGC31CDnIqJlqvjBPvnz2QY1A75YImVgsR/SSnHw5iD7 ITvk4LjwxsHYY5YBwCoheHUtNcHjR1C8e86RX8rzHV2eRgZ16BFnlNO86aqv46lPQKZOV17otQpo 9COac0OE6suQvXMBoDEqGYbDkRA4E8YWwOb82vCxdi2ZcEO0XHajulElH8vsMtean8vaE6qWHLUO nZhFOqHk70PhEEYmtX6rg09kEW1XVrDro1sNP5YfvXHJ+VloLcds1gdtDABEvE804x1lCyVU3uqc +wiy5NNcCUCRW3y1YbYteJKeh9tVqvgiT//2Cjjm+jER84n36ltD78ha6VjG94VfNjfPq75dbkhN bOkFcSc+WPi+ULx72UfwWKHEw/Qi+LoF+6NebxmFTif7MUo2O5U8BDyN1degK5N8DCOsKv5kDOmg +B7noeTPUaUnivyRFuLx08xj27fmamuKyFQSIiXah75/Rdx7deAZ/Pp+jXialQegBSnwQSX15v1P MvgGb3QIaHuaskqvgIeSymNxy+XIlOnyQHWxwkE/5ZqNJUrLh8i3XzjtZ7TmPazaf3jID/oXIIJf CfX4r7NzVx4/89an3kgya1j59ir9DYF1mPOhtv8rRz1lg9S+YNJAB/zb0J0oYQ1FCuVWWwfvvBmf wd7QqhLPxUfJ1qJyY2L6KmYldB7glhWkQad1DTBy+dwwhEbMZqEEb6MuSgXR1/r2c45GNfnDp3g4 eRcL+wqpw9xviB1r4egOg8xkdtj1Qlc75aXHzdPPwuuLvdhlO977DnwfScHFFSmduR7Xc/NpGE5q Y7ltDheCA7L+FKeYzt4xGpP2QCeEalHq7rTXbrbSWunC9LYp68trn0xTLS6bTi67jnKq87c1bUXw KsGRZdRRxAH4h6pAge9uJmMdtQwtX2UhQWU9kASG8einHZc3Ph+02gGsk6uJJHayDc+4DubyLCHZ wYVq90fOvLqJ8SR/ROcdIjMujerO3BmMrwgL5kKlW8f3su5sPKnxWUlyyE0Uek6Whictcs47J2DO GGeYfdFVNnC5q3fAknbddSkjsoE+83V5XpeDMLs/rvLYlVS/qWZ6S5xe6pLdl5q0AxlDeIKl/0wX m0kZW8IhS8EXNp6ZkdfbBq5FOdwQuodeC4n4XhcnMWNO2S2HjpCXqvrlTyIexa4ZOx4lfwl2wnVI vGkDXd85DLuTRd0DeXGxKW7QoNuhuzcwMQ0Ntb8plhvcUDwG7wqvCZznWDp9r1hOaHyVG+cyKry0 pjPUHN6ynXCh72laVoc7x0xwdVaTVLOCifqu9XP1MsUQn9p0/hEY2N056IEJhlBemZ52+rB96l+a Un6bOTtDEd8I0t18qtVs9g6sl717l7hW6WOK5E+LkCGhyuvf8jcyGT62DH9qeeqWFiAuu76+ISS1 SZl4gWk8CfiOtNYC5bGzxrhWkdN932hjjvy2ywwTkPO7/Ki1XIxW6GTmkX8+/nhXydun1xQkzIoo UI2UFywefibZUH0k7/1gk4aMQ2ZYbXGch0TPMoB9SEmqzwNnUcEZoh4/f2/Qzy55Ntu8iOpWuu7W CeooqT6PJJSnUc8Qr2CRD93rL3e3k6cLBLbCISEiE/4fKYYElAplbmRzdHJlYW0KZW5kb2JqCjEy NSAwIG9iaiA8PAovVHlwZSAvRm9udERlc2NyaXB0b3IKL0ZvbnROYW1lIC9ZUFNRVFMrQ01SNgov RmxhZ3MgNAovRm9udEJCb3ggWy0yMCAtMjUwIDExOTMgNzUwXQovQXNjZW50IDY5NAovQ2FwSGVp Z2h0IDY4MwovRGVzY2VudCAtMTk0Ci9JdGFsaWNBbmdsZSAwCi9TdGVtViA4MwovWEhlaWdodCA0 MzEKL0NoYXJTZXQgKC9vbmUvdGhyZWUvdHdvKQovRm9udEZpbGUgMTI0IDAgUgo+PiBlbmRvYmoK MTI2IDAgb2JqIDw8Ci9MZW5ndGgxIDE3NzgKL0xlbmd0aDIgMTA1MTEKL0xlbmd0aDMgMAovTGVu Z3RoIDExNjQ4ICAgICAKL0ZpbHRlciAvRmxhdGVEZWNvZGUKPj4Kc3RyZWFtCnjajbUFUJzbEi0c CO5OsDC4u7u7W3BngEFmcHd3gofgLsECBIK7B3cNHtwlaB45cs+59/+r3qupmvlW92pZe3d/Q0Wm qsEkZgExA0pDwC5MbMys/AAJJU05NnYAKysHMysrOxIVlSbIxQ74tx2J6h3QyRkEAfP/iyHhBDR1 ebFJmrq8EJUgYIC8qx2AjQPAxs3PxsPPygpgZ2Xl+5sIceIHSJq6gSwASswAeQgY6IxEJQFx8HQC WVm7vNT5+xFAa04HYOPj42H8IxwgZg90ApmbggFKpi7WQPuXiuamdgANiDkI6OL5XyloBa1dXBz4 WVjc3d2ZTe2dmSFOVsJ0jAB3kIs1QB3oDHRyA1oAfksGKJvaA/+SxoxEBdC0Bjn/6dCAWLq4mzoB AS8GO5A5EOz8EuIKtgA6AV6qAzTkFAEqDkDwn2TFPwmMgL8OB8DGzPafdH9F/04EAv8RbGpuDrF3 MAV7gsBWAEuQHRCgIq3I7OLhwggwBVv8JpraOUNe4k3dTEF2pmYvhD9aNwVIi6kBTF8U/qXP2dwJ 5ODizOwMsvutkeV3mpdjlgJbSEDs7YFgF2ek3/1JgpyA5i/n7sny1+XagiHuYO+/kSUIbGH5W4aF qwOLFhjk6AqUk/yL82JC+sdmBXQBcLGysvKy8wGAjgCgh7k1y+8Cmp4OwD+cbL/NLxp8vR0gDgDL FxlAX5Al8OUHydvZ1A0IcHFyBfp6/9vx3wiJjQ1gATJ3AZgBrUBgpH+yv5iBln/il/t3AnkA9Flf xo8NwPr7858nw5cJs4CA7Tz/of9xxSxqijK6mjIMf0n+j1NcHOIB8Gbi4AYwsXOxAdjYWDkAPFys AN//zvOfE/hb/R9WVVPQX92x/pNRDmwJAfD9KeLl9P4W4vbXZND+tTZ0gP+uoAx5mWcggPaf8Tdg 5WI1f/li+39egj9C/v9m/3eW/+v4/29H0q52dn/4af8k/H/8pvYgO8+/GC/z7OryshtKkJcNAf8v VRv450IrAS1Arvb/65VzMX3ZETGw1cucM7FxMrNy/mkHOUuDPIAWqiAXc+s/Z+nvy3ipYQcCA1Uh zqDf752XKFbW//G9rJ657cu7xfnlyv5wAV8267/rSoHNIRa/V5Cdixtg6uRk6onE+jJn7FxcAG+2 l121AHr8MeIAFmYwxOUlBPCi0RdgCXFC+n2x3FwAFrHfpj8QDyeARf4fxANgUfoP4ntBpv9BXLwA FnOI3Uvbf1vYXoaXBfgPgQPAYgly+5eB/cUAcXX6V8ALxepf8KW49X8g50tj1p4O1sB/V3gpavcP ZGMFsPzL+7J0LJB/4vleEBj4L/dLfYd/3Nwv6GUKIRb/Yrw08E97XC/I+WVD/3G/hLj8436p5mLt BPyXwJd2XNwh/wp4OTHXfyD7S4TnPw28aPECOv1J/68rNXd1cnp56/2xey/3/Tf+4xULBHoAzZEW 5yDmAiE2tSGtP2vEiNyZdsbYOVH7L6KuELS3pQjcfqgFSQjeXXIURC+Idc7sRt+tYx2yl/5qtNXB Y7U1+u6THs2Pt1jfCmtptsXoM4EldH/orOjKvqvQ0aFnwgG2sMI7/ao+6H4lOk2Hz+h4vjObuS8j SP216rbS+hSepuGIGQV6LDYa/g0vLwcKdEibWGZblfhnsemZdjlK5wvRseMkEun9u/S+EXauzf6l ckYo0EY60cEBqoPCVeaB8Ie9bm5CBZMJwDTVdrfUm4f3tEFXrSGBtGbO3zQEv6qrxzMrJvlbtBy4 +23LvQ9ykOXgZYj5kXae+FM/eNlUYnMst52umsvhSpBt6g6LxjOpdipfT4ns4DLD6uzVJ4W5Gmu4 9MzGw03nfcZAxiJ62aWvIcrc5RltIbcsyk1rIvNrdoMz8fxd52K/XFdPG+33aRHTtiIPeDu+4Jjn 491IHmtw070Kr5aGPuft0M8KAXLnW6F2TJX0kuVLfgz4Ef1Rnz8VdrY9dhlW044xdwddPNaxUHp9 OwrT9oMEfIFcpcaUiPUHZ5j0T6kpd4F71mP7SwKI+fWasm1HKvDvOKNWvKftRcmKFwrkQykPxABw 5OrC0JICu1tQu3uvtRCKJ5P8sb6Qw5Fj1fCTBT/AmxOpVSgRO/LTBipNqczRNfpQIhD6kZR5EJLF DIjxuMJ50A60VMo9aYQcyPjkyxG8JkS4VJuJPx19qxtf3iOUjZCDwUe+3j0FCjpfYSP92Sa+Bp+c 1/LIQiJy7gGd2CzHyym777QHXbwydp6x4iEdyeT/CirWF19D5NkKuKlj3Xi7ZXYwQaNMk87NawQJ iCU8Rv2ekBOsuRpiU0B+awyzQFiP7FrDgadRql+avVPNUWUkE+PdltWGoB39pjW2OOLO2ZpcOeMI JnhoG5dU++TznW09KYWc2nUYX/aiBVHZSXadx7pDKcGQLop8S1KKfDM+lArWQ2EVbMPIDe2PQwQt JdwAUwXJeYeGEuxYVXghHwrLxV20TsXtd4JPBPIUkAs6DL2Da+bOV5Ra5UG0GKU6hULhhE0yTIIz ikbOAOailYltkS0KJwyQpw7hNF73R3Vs3tVr+maHgjEc5IvdJvVCCSLXYhqbdZGJFbm0o89fJOFM JxLZqHI4E6nfjlC2noDVWTJnSbH68BfFqv0IQN3UTGUVVitvzwbO0GEDjEsMzmZEeoyrGxM+tSPk BrTZOV15zg9JGUIRswmwGSYSpGFPhjUqWY/DGeUOF3YS5qCtpC9rxDf/eICupxAhJEFmICIx3xIN sYxFg955EzcP1anUWW+hT0NGBXzEbfym8xQqrwWH8SBSixH0ZMmHIw3kIaaijEZcacWEnjgeCxqM BJsh86dunX2SPPzJW5WvhCwlucMJZcT6yMUJnx9MxNnoWFQ6ulUo93YgxLAytl4gR08/uutDQ+Hq rpw9Sq035uKMPtFYm/+wneOZIL1s36u7OXLLalx5c9dfJqgZFbyYSD8mAzN/Tb8JGjKyc+gvxF2L 2mrBFyfZGTWEJghHQjQjwQALNaThJRTE/Trzj+LyDY6eYahGEY3gCmLc0X8yw4XyN82rR/No+4ki LG/7o2aHyli2VOZCiRSVaosz7Hv9OiHSlWSMmk2J7vI0W49QYdXHA4WI4OESWb2i+D64bxnE7iqD wQWSMOlbiyHcmJjpEklkvNGGBw1WjOZvxBa7iTMh6yLxJ1PZLe02XXciAoH1chX4uWIype83dVU7 FBRwh0oWfzX17IlfOBtvB0xq83WIhVdLsstJfT960r5RUM75TM8BEqNyCQpCK0OGQf7U1MMM8UDZ 6KEYpIbSaLaWfEV++YnLOxRM2B99UxpJJeDNLMQfuVjtayiWFB+yf99YqQVC+SkUfvNUyMwfuDnn 7gZ7+osbGVowMBIXfLeF4yfybnqCIeBgKCImrWKJN+GqRh15ywm6OtDe2QmUGs/57RWrTpLVe/kt 2HFCvwaoh18fYpL5MM8qZGsj6cplvnFoio1zmAsK2JJuAAweyTP6z5r9cZFqyfnCcmdhWmC0xXnK vXX1XEOEvxj2C6VYLzcEfckiqKboWJn/kjhyqlx6NSA0xktgZdt0oWG0pzcWNGqBmVvr0QrzIwAh Tu/y7RnGAQpyV54wgvg0C9jqY71sGSUbjTQMFZmgW1uA/Qg0pHbcRcnoZCyHI5SEX24xx7jIznDR bB+33g+xquTNdkB3jQZpM4Qw3uLtTzerpICCR98upNa5lY7VYjg/PpNvwwe587ZjIUeI9Wi7z4eR ot0tOckCWGlaHTXP0DZsVPHjDYLKfhtB0CH54HEoW0LLd6CwtKFnFJ3K935jeiEN+hiEdm+/HAu6 leXPs9wna8kxcuCiZCQTA4J6Tsb1hPDReiwG0mfG5hpZu5mVrmKrLDjAT4icRgfyCzqOy97XfZWs /Zbr5CUD7n0/iW0H4LtV41KAoZdxl3dS+AJya1OXb1ACvER33x4tE4SpNzQViW1Vzm23UeQkl3/T xjoh5DjdOML0jeZsxU+qQUewidTAjSOwbu4YW7Smee2Nybmn735ZMvQjWLjI0aD0V/2D83U6kRUy o+7Y9ZjCE57OxivCiYt5QyuI+pm7XwKU3BDIobRVh9MY6UbskJuuN8D4cwLWDtSqOLV97Tut69pl IfmSjzhRfE1eti4oPDEw81evwGiRLMoMrrg0G2bBTpSas5xwPNXMyzPYBrFVcqzy1sJrdB/Vgmdq 9BMMyT+/HnxAWXiPhroEp1QiEUxMOr7Vl/JjeTQs+bru9K3W0YMD6ePYdeAYmmv/CHKzbeesdzkM eBQ9Yk/5NY0VnrW1M6DEV5kwNemn1SP9M3adS2vOvEEJS7gNEeeBQ0MkTWcN3ih7+3XOIy4pS5Ge 9/TsijeXbVHN1tRqzv1lKGmZQ8Uv/zdhT6vZqlQoIhIT325wPxAj8q36d4m9u3LwyrncGJLB6pEL oNLTMa+SRzqsdCLx6Mc+ebDSVc57dLKsN3cn3PZLq2wLFi8XQRDPK4WjpnaZLFoAG339lBPtMHV/ WI9dKPkK/UmaWm7m24XHKAnw/iPVkTnuKAh36xRpM/QT1yRMcPKZ4G16d2y+BKsEdvU1zcfUcwSs sM7UIoMh6U9zeXDtz9teLf36S+qfNlCXOWENQ98KVQWHwa/Q7UmZ5KeMjYxm/4gt1ZUjk78tsgTz v9EuMJztS7MdA8tRdEfbkO9HiYeJuwW6LS71TLS6St/KKAswYWHM2H1TGAtIAdE6T9v55svKNEDY tocVvXYGsny2zzQYPMxgg7acCemBHLnC/N/mX+PkxmBIK6O19UdqFeU3hMHbI9X9uq7GkaAhOMIx fFwZop6+NEmPCtcOQh54l4h8cTmCb1thT70etRvQKL4VI5obMDaeMUo9pz2eQMjCiaxrpPIQ3YPR cYZiAoZuLxTeAqunShnH2nFDk2BIH93qguvAjU5UDjzTcBcpK75tVD6dWpKx24UPyU56syz7jJ9C eihglYvZoLdWZLuX45nDJeLmlcLBdIU3309eB/NA16rxkFmGX5Y7DX2wnbc4syRVfn8yd9hz22Bw CxlhSuXe7pAbLr2iUHc3IIWWGMg42HQ8vSe12Dh7uA2j96P7ZHiqF1/zii+pTjOxr3xhSSAvRzJ3 /aM/MDAoWCiKQr2/glO7t/CyY0y4QDHM8DF4qe/qsmgcacIg6kvi/lurnz8OKNnTcykXHqDt9/A7 kYOQ4h0b8E+rvr5eLRBztAgucO37kT7CVSV+ftyH2fkjInN04JO86SQd3lTHd0ShbX2aUEvUMnXf rF3Dn2GC0Mqbbbl4XvETgbPzdBTHrI6NI2ctkTIc7NGDNDZhgBAraqulpzbg9dxy9w2UWMx9NRjP tChPGEbuIc1A/XX5l+GCjAHeu5grPLSs75VJ6BmuH0PWOWlcOqI7AwFbtZSOIPK0B6vrGakqAUK5 ifVvDTm0Rb+6Mjf6K9F4Ix18USJqx4d76trojMidDkqnjofvJ100jLHfcSjAWvOlkGceURZH5b03 TaaxzyppYz71HQrle3u/pfu5cH85dRza7wMTSPzxPir5mp7h0KJ9IyBQB189ZIRYDlUP4SIopmSY Q1q6RHnv2SFQgo6N/u5I/V1JLZKNPYVO+E2dHIkXHwe1wJibMqFmJObQFMjh7QKK4zyeM4MAO/mC ecegboNwBj5NtTIfhnPWgBXm6GzJZOE7hf4dqFBErCQb4654qFd9PVI/l2IxQhc+5Ka4UH5lmZ4+ wBk4UVxKnDFjg0cPaGjJvsl30JKhpoI9qdN1T/28W53tg/a1jqMCRVF/8Zfyxqh0m+K4IVAq9MVc /dQMFPN6HUKglD/boLmJM72mJ8ifP/SWsZw5j/KHxrnLmVCpho30gjHBhIPUq81dr/E1pyDfKvFb yMCl8le0ARhGVE8jk4sFeo4tpeQWIKg4HbbHI7fASjxk3AFTAuW9qD0qMs7PttnlIAGBxG1o1/2d SlYmnHYC4lY1R44WCRHygZFqZ9U+cbdQ7PPl78CceDOj++sQv7lPdOmpOXrA5btUxpFE/3b9eCRP +mpYkfHVukSpqewvLMZDeso/GiySiqXMyaMz973LmjegXTleyYNPJ0/zU19dccVAddu+db0+JlQs FND3S7eApSV3UCIkGlcoLVZIscFEj/yQ8d71amRYzEyeu7Dysn7E7fSNC9PaQyS9ZXWD6zX1PtXX 7QQdDHVCrkiFo7WZMtJf8Hh+bSSdUcGO4vlxma9i5ivrC7sIaaX4iYOo75FUGoavn8aPEnaZH97p JImqmM8tm+Tml1In9BuLw1USujxmR6vEW6k/gxOYzs0SnQ/oGBmGvPe07JcCaT6AiyXoIv3t1hl7 q6nfVChG5WQhayHIQuefucKmxyEoanXRyNtxu/BbNxt2tkaJCb82VW/SnwzC5x3paF1R9j5NFFQL zC3BRuimXng1xgqZWfZSyvItecLojXHfNpOcpcBcPl/BUjn6WMfV+CS08IZ2EmU5AG6BmHtTAh4n vle6C/YyAuFC/vHyskk+KWgO2SRFrKekBJAyxvqANT0kXmdNPLjHidUOWtRhvfqMxUzhlVOpdTr/ 6EaROLAYsbsoYb0Vh5NguQRJdOvfwVMfMFINLE8MGeC2Z1RwWTT6Jt7/Ydj4kQNkgtyIUdz5Ke9g zPXJSXbqaa5Ghqa4MelcLZOa4rGyTbLXtcNZcLIW50G9XnvL7hKBqCHNvBzu0UMc9lJPRBH0uBdp 23anooN8HAbSQ9eV3UbYl7lnyjoiDm9+rsP8Xua/y07XVvjAZuJV1XtJN9D/Rvgu2VIErQBlExiO IYSve85ycBvifrLwiWKg8XR6EsH96DZAIK4o7gJjgBg3KxACz61mJ+XMxR6Q7rCl35GHYn5KeWQG Wx6rOlrXF13I0e58x3e0xjZz7Et54Ui+oToVObaHkWGW9hbCMPt+9Fk6Jt7ZjWl6S/TAgHP5V6Wy vSodgs6P2Jsawu+r+R3+Lf6+o8ZYnNYQa2kCVL4kCUMdmgiHAE4fwsGRhQuEcn5TcR4KBZ6okVpM 8DbU7UEK+SFvXcDgCFtO9l74ChOcd67y1tkFXmqVgIsFah29KXnMqRe6/ysbdlMl3GeVzc90cOKK FO0oX1G84ZZtxDXjSDhq+bTiMg8rXZqtv3AkF/nJqJt9fctrwsOcKgaA2GnCkyM0EkgyMCrwDkSV 9F68mXzWOIzNRahQGjShUdJpHNzDU+asmShwgw16fvmPpcGeUlShq8uQOOodh/KXhKMr4vMmEpDD khcC+c5cz3xWh2RK3zdGPWNX8eKzzfvddSVqLCZfUUsJ8E2SlDBsjK4mMRZwy59r9NR5s78+f46S xZhf0kbzapZz0N5jduXBhgmdcOBrZldLSV8tr4VKUN3Cqk1lL4TKJ0GHaTUSGzt0dxodLVDOO/s4 c00kFTO0Fu8583Nh/tyHZHN3+cLKMaXQXltrpKjJOY0exZKbWVEBjgjaEJUjWjhUCaJ1vQLrXI+J SiC/xcPT/9xtBr16PY7JWHSZvAsDRXk7c79y4hmKM7fut4+FzHCFGhYUVkeYFM9YaD2e2tb+3tFr 9sDlDBNGMiFlTRrN9tTlWQ3tggx3iFjtUNwOnbZ1mDNpR3iNP9g6cNWt0vG+61ZurZ+oU5L+kUe7 dHmbK9CrRKZg6cK5bOHc8dcOgoEOj5t9gf3rVDpl2xz0VzWyB4TXBxg8GDDf+O81Da4apaGJMcEb Rbyv72b71yMOO9Bi0MgiggduieRDkIRVlPVyI5CQSkx5K9LJ97igwABEveXFCXkLMpiNhPXlvYFH sxhuE6GBmbi3SKP2/Bx3Swm6qQp0GJ7WlnVWRaPWy0W3qmLoQavL5jStGqRa+EwwCT5pMnVdvINJ Piz95G9bOpVIFCpjbbvkANDJ6YSdTiCjgElvG1JJwVUBh6D6htAer2cbsSGKBI+SGoGDPZ/KM3QK k8g+ms+u21kR+WJSZ7Ay6bXE2WBkzh3PEOvQ77ljo76cMzaFoJ2J+YgsOnExu+7jS1Ee8Z8lWdZv PaMhr8T7zCz7N5sMdg/e8+KnyWigZ3MkdmVWjsTHBETeKK99sdBhrPcm4hQNptYqYy4bYMO1o85W bVNAXp6rUPO2n6d5DDf/CRNUTbBB5epM6ISPzT3eTFFzSYabkx80q+WGxLzPJBrdoh1Kcj5aK2Xh 2GwI9a44zVq9ktnSYvVzHMTKicODwG455jH69r4x7sgRz7mvYX2FwtRzHyrlvhx0vMCJ8NneXVEy qJJaF+rbDKfoDltcz6k7fZ/K88pURx88o1AzQyvGSrvrcKAW9vrinWFBFLu1WzSmn6CFZaGsSial nGH7isaHGWUqZ6MBNDfdL/2yFYPk0KOkoIHWDxTQgmvYPPYJmDidA+qPoZMYMBGENnSFTbY3PRST LV4z7IOCX4vLRRW/jXDtxUr58EkELbfihTHewPBw7GebVV7p8DeSonbFnZMYw33oehtDooLhU4uR 0WJwQzvRgywH+Txrt01d4JZjT+WsgH5JxrrvrWkBvbXl4sj6QdMP71Gr3VUg8r0PPykFq93m2Ddv GZzF1vT+uE1OwRqKiSb9payKpYjv34wUBrxl9E+vPxph0Vt8a0AVDKFcrTeWf6C3d7juyBmR8/Iy FByj/UXtozk9YvChmrLMpCyQ7DXd9bwaMqtHXDqrygSYCnslgTqAJR66eMS4pIH+yYhHs5VXnVOu gI4yyDomfg+vVXfgl8CtbxSzjK6wYgo8GxN+HFImVuOemq/l5qGJlZStiSa2LdWPmFqOWvrEzMj1 T/EJ8riGcgbTjTyTrPH1xabSc10tT/GRNnnEFYlJbdR2nl4fAr+ZYVG7/5AhnDWMqNpsd/juzAl2 ayo1pmQ3+liU3L+LuPO5LcKY9NON62yRgK/klvFrLMzgIReErNnoDwAkRESvkZsOosKFXJuIAIi0 aU357RSzM9O97ff6LlVfUukt+gkPtYXnjOHk3S+0p0uvLohWwg1tvF8Jc8GKCz91E5fTLt8WTBZa +gzlTXHPTAUNGo3pD8h87+fGFzLMvurVR/DuUY7HeTzmKZ0/lElP8jQatobPGGBLG75u+NHlzTZH joUKLlTlNYM0q30VzSy+z6CtusYil1M1Ncx/naVc9zkmvJVsAaFQNwolU05Srj9DTD+Kta+1q9i1 XGLULw7/8Wv9sfZ8PxGKQxpL2I0zt7Mel1TZWOj60m0R7bC0gav5UPK6p5IjgHJCvIQnlIcpKPte f+kSt8iEh6ZxnHxRdi3I88Q3lxQlPbjMA2nEtaSwqUMTZocbYqXK7omxYRUw6hvPsPCxupt3oPoi c4xgs8Rcwlp0uZBHJxUuCBdNDRx2rhhoKXHaNeREVnPxPm2wYuJ6g69h1d3GIGSocLXXmu1LOQ3/ WmNyy7gjeq6s3cb5142AUO7T8Eev1lOXdvGYra2TapOCHLQ6b7cDL2XufkG3xrGx4yIhnf60Xnip 3l+qKxFOhrbg13vPyxQy5V9Fi1KIWiAzCQ7fybufZpekEfBruTQs8SZH9k6c8DYLC2TLWlZkzpWl svXZhr6uqjuU0Cy2Nhi2iuFTbjGjh1frc1iClFkdaln6Br1TIuUCiyg6Hz5kaRzKF4ckGqdduype bp853eo/Dbu4o47JGVTaQ0pNNUy5h0yRxIz79u1Amse0Z7r1V8/qln4cPzLgbEJIpPQZa1YnT/Yz i5m6eS8YM+rfIrpuJn5hINVdWhS3v5H9LIaYOUacKHk+Zfa+xUw2vNR/raXEdpehsLTNhHfX2gAd AbbmAOYYA71+4EM3UJtly3tMJDfMZ3MBayJ/7hF42O+fiWiyIIZP4t//9RUli/Z35dbZMS2LMV8p WV7WczKbtTBLsc2su1fAN+JoBF5GZ3WHdJ3ZbuiPw8IJpZRRD7RiaShsobeY0aIWpiPCFWhi41K2 whk5Kv5n76AD2mbCGzxxTXSihTb9tvw13lKZvO+NvrijSWfIFUDQCCvw3x+qgikwxNAGOViLnFJA Z1r6HOmVTow4gphXX4knCKA/nfA993lSKj0opxSN0L1HWOfGSY2g/9mO5aP8pvczDfxduElqQQWR SKjOG4dvlBYBuhtWUWwt5xqZVAwXLOrhX0wvQLxELAmJzb2RMoFIWnSxauhuZNsFzyXe9Ezp/tfS LLGF8Q1DNpqjqQ3PXEIECJ5OE/ptlmGEDlNHSW/fIo2/obAx5ntHxwI5J7qwazMWG1okQgzbFXfP IabWXTMUyseNKFjYvjSj+Lj/HYqDeQumjxKnIZSLDhOR1P6k3PzL8X4TzC6mJ3lYCcGiYWHKq9FD QRbprgr1yfjgvJzAR12EAr12ytaboTPZqHJj2jWHhIypD1d5H/myU7xg05OeXKPL3yFXtD847SRf LYIiDmXs3AnZMs6y8InVpj7T2fdcvl/Euaqn1+24t/0la7NOqY6jOsqInOAofvRlFhZe1p3ye7ff UesPhGxDc1zVN6PwBJzts2rUrJoy/ohGvseva32FfsbGf5zuNbYxXPk0GtEovZubGxLCImPSXiMX pOa1SZvag01e9KXHXfByW5o3n501HNZBkzG3Z1GRE0/abqkOej7dhqaDZPz5qM52gHrzppZNZaLU IzM6Egmsnc4EfX1jY/wUi1ebli/tvzhOgRvwoUtzmLhVj+Le2s8TF8NDT2BpOIwLNeURrMUPbyAL Yisi3sNw8mO2cZCSj2NqEF8tHU5+0h5yKUan3untlBY14Fc9dTovqYjsJx+LI18xC8J7D/XmnEPe kU0uEMZVni3L3+GUQbSJ4jQrAAlx5I2tV65rYn+RiSVKBuHSddavjJ3dBuSwiZFoUVxNLGBVkea7 tFYBgm/oGJ7xCfRNnUaV00xtTt0kGlsX3GJr5xYTRRINn6QxLqkISfgm+B8nrauEMuSqB/QNOe2W lA6Rtt3j7hd627Nq5/B63TCZBx1foS/TtF/23C30GvwStiOSCLCOGraa66MCf8bQRiS5cfbphkmo H9O3/FVMLx7WxW1mQhQPmRb3LXQXhfq1OGxQWPZWHwnmEPEgOlFAIrB8ePP8bQlUMEnDynpYUWhq 9M1peAuugDyK0QqZTCQ1rltDQ40AnZkorFDfBzLT2aGrBeW0W8v9WJ96OssoEIRDtkI5OMPu3f7J IIS2NAsjh1mHexz8EBNZo5Bgs7DYO+aEV5/MOSJXG6ZPi5171F57pU9l0B6WskMgsNnfg/6LnGFV 8oqQn+Kj9Tuh6wvtW2J3p61JA7aBN0JigeoHC+uUhLz7SXzMkmMViqTL0devmGXeeYhdMq7hSvIF MC+/N844KS2KuNpRJKWhtRNtos+Lc8DMKJPfREPnpJCMgQB0d4XmdmKtC5O0yGajdUffh55+uggN oawQVViHnkwlGjFUlCBC+u4Ben3GmT6Ck2B6yQ1X2eX/FolaqKDyQg9zlZJqT8Gq9QfBL9wDh7Sc UvyZ9unOSNEd2rHX1GrkGvGbYSruKRrToE2g1r3uIT7HgemETD70DiPGoOtFhv2BKH25HWScdffA kapY9H0FyTdPdwwvyWm/Do5qznc1ZXbYO90M7RslZV2/HnA744bScToe6HocgqiOBpy91LQFMVmO 1snckgOhkedrHj6qlyt6hsgd2xOcIjR9uUuqwhFoNTZK5meVF9ePLm/EyxLH6f96wwLZAd7ljizV F+gXH7vFx6T3yX7Okw3Zfa0kXVgjR7K09W58S0gAtlf1W9fiu5Eh4AU2UeyqlWEFj1JTAdNH4rhk kc7Zs9bKEo1Khcd2NL7MuHjzlQ3uHhQ7u+L6VoPNwWVkabw57kMgzRE5QpoaNCKuXCYC4cBj48Za eK+kpz+jJ3Z3shzpzLUbLIwXk7FmEIuV/lclDtL0/MBdlzrzj0TdpwyvPdJjQUcSknEzHuj17JfP mNkWGdQcsne8Au0DFe0tn7C3m2q6yHLTqBuxNuHqDZW4DUNHqtNzK+BULN2bgRQCrxTCKr2ISqSy 4yU3k58rX8F1NY3NoqHvWGDfrXnyRxlKy15O1xJXEqRKdtq3aPDVqxWt2VzSfM/k17WnjozbaYKN ljL+AK/SdWMkl9Y83B3QmuP2uuFbhO7STBG1yQZ9uk6VZbg+Wu/AMeW1KYVEvE5t5OI9l4oXblFo Iao60M9m4mNzRW/nXMSNt/8ZGgcxAlce+ZTChLxCBKRuT2m5l0IlCkczYHKoxndY4ctKptjaZ9GN OSN+VQOSAaAtcxP7Pi/M4JbUw7VAfQoqBUq21Aw70dF8JMZGxTxajzQH+6kx/fRq3UmN9Fyr8uG1 H9Ej9grZOm1/L1YFLc+RpbBDhKfQmxGq+V+rKqnEeU4jpzq4QOfIOoPSiW4O5VXm+y/3RsawV6Hs ddcz19PoFw9c79Rt1crEhb6kS08KktxHERDA/VgQY1IhK+kbY/3ifA84y0tOKHpP0O/i72CscQsi QiF1x7BccM0wkpUdzHGR1+FezuSdBC3nF8NnX1yJA4uF22IMn2J9aN001m9Pp3TJExjgCOBjvcLV j12avyJrEcbnbozpCi8bujbeLGA0F5FfdGzt+Kuurh8VcTV8nkfkZpa2z3MQQUZjS8XgulFm8qLr i/EaWFvwTTW+PmrJDoXDLkGZDuzXukHiSA+VTIX/Yhhv2vyRzt3YcXXBreLTXD70GP645daqX+Wn 6c5s/92a7sUbjFMSXgTvR3DC425udDPqx8Iubgcb+kqDj1rR8+W25EU4CJc8+TRRG45rIsrAwuv8 7ybvb4LyPe25xa9yD50GILOZGTIqxPezwh7Su7X22k00Ok9P6Nl00QL1QS0G68fM7TxOmBSy6I/o Amw9OQx90+UxH0TLvcvycaMd3OF/+kKtHJOdLfs4V/Pu4EvD4jVgRdTmrTza9OUUv2+4CKouox+I 2wmtvutMystp4sQmSdSrNMzqLTTTved3Fy7jPj3g5yqpfRbbcO5dTkV4s69yLihCusyfefBesfhQ 2SdP5l0mKca+TXSNtsWoAFjgQqJXtjItK/2MYZxuVLpoEJm9NlOINK/xO10Pw4KE6bEAuAdNJ6pa ueeROZ25RPIHwgdsV7enXiiG05OhI9L3CL1R2vCQluOx2iBE6gCb7UFqRGZPTNaj2ITj7Uayh0vX 7IA4z8FJhh844h7a+zBE52iXxgRfK99sbkF0Dcuu3Je9AnO/h3jKcpJmANKWlOLNjhJy6ZZhUWPl qwi2HaAMLjhJDuDgc+MHH3g/cTVT3rdLHHyAy8CJC2vx3kLyOxZNxCZu8dUN0vcNgzb0ZGSvE5QR sIg7YZoO1r9V7T8V6yc+cpHUyriBnryJHZIipCzFMEbH9+diEpbV3wrHs3pNMfz+sD/70IwhARyQ to/d6HuVscDCSGdeUwILNdmMOC3yo5jEdSGAVsIjkHty82QwmCON5ZmC/tyyO64gfCprhowcR13t vgiauNyA89gtuhO+qyjCNRJwU91ml9zbu8rhYtVgPOoc29VcwdNVm85t9CWlpmCKY/cRhmDrneLM ullb6Pj9Q5Jlse2ih/+vapN4JZdybsX2L7sq9ScH4FKha680RvmNyPh3SNAgVizoCmscRH+RK9Dr qQbsUF3WeW5wMpraEPPX86QqJoMoeEZpn3XpLGJGDgeD1Tr4hL4xEQ/ketkqm4yhrLlQpgZNRV2n fZaSAm9ZgzzjRgtiL1PVCGq6hZUdvVt7tZGk6SwN09pwBQLVaNWd0yq9b3xpYLBy42sBTeGvojIp Wdoq1D7xyQXIaTyoNo4b7sE8bW2BCNa/lLeQHHvm5AWOKGl0OfYmNGvs6eY8WKMJNE5WYne6e+qD jxrfjY8lnXREtO0JDWugPDGmPGPmGbCmkgESlY/G196svJ3NGD4hUtkd5p0afDDp2DLTCBOVxSt8 83DYc081tFNVvWsz3XLOPvJN8mDgwNX3E2J6ok3qQaow0msZkCaJtF1IVCL8sjrIaFzJwKG1n5XC m3wcA+OxwVAMNSfjFk/9DcoX7LloKEmu1SkMXZNesf14UrW2m4JeeyRIhieZSEeu6TdBu6WbZS47 A/oqCw+DaX0vCyTcLozcJmO0SOk9y4vpii8dYcWB1ERdMk7niYHnbTKiZwSN0q2UNDQ3fEdZGfiW tJOuOgXLkl9iO2hoGyS+0bAUw/dGCjm7ysU4OeDTDlSwyFBjReXiVIBKAS35KCo5JSdCFnlXfmTx EVgYMoPKpyyFLqamsfoBzdmxD1JH/gpwXvRNZLgSTLNTBIUXFYgEOL+awQV3BC3pU6NP8EW8KXlz Zg3twz2rdxzNRj4CnwoLFWwBiJ/LIPnRl4uj7alIDWtjTBQZLOaw89P4jgO+WfgnRwhVvhL9b74T XclJ8fFbfJ7OamBluTtjFIuoUfp8eU9G3VKoUoQn8ARd08P1inTmxzQeHhHmdqiwcNTXva2foYhJ yrtz7LtbxZSKk8cMTp0/lxd8DApILhsO7+WSyq9wO1gyCS5oj4hpZb19k8z0N2tS0cUy2dxcCSIB dBmXsdowBktvkbSTdVOl/PZFC7DE6rzryBDzNjmMWU3RUUcnsL//IKBDrEyMoTXsX2JUFhM/xfTv YzuV/ZSy3kvxGuUEoa8omycFmm2glw4IeWgoa7auRNnaLMs1Grm/RWtVobGVPAdnMU2EQPmnW14w KK1ZN/MttVJt+sQ0UhDnj+xHzl33coehHDewaiyA0RTNOs5YCYsMRFUER7/2ghKTWSmUOOkN7FhT yu+y3OW3rqLyqdfS9einSA9Uy0EH36kDkjBO1+Y+ZhAk6s/4kOGgzrpL95NFVzZXMySH0z0G8WFB /VzukL04ni0Mvkki7OJ8VorweeIr/5nij9zgmB0v9dwpBv/pa0RDcGsHoj0S6jcN8hzvJG3pWW/n mbZlxzju7u8//YHYjSo5dqRS5UTIVIsHIdYtcwsi4Ax4GmPxT2MoBeHTGJUKoFztBefjcMptnZxb kWiO4YSmV9MoC3fbFFQAZlTsOf9GLfeRs0bEY2Jtums3pfWGHaWOy+pFaJyljg5zlQq0fq06hsiP UmIMihUo8mj7TZPENPWFoTNo27QHFWsKLOzVZweYruDUzjoEMlKyMjrd1Hn5vKu5gbfn6drBH6Wn 77YLgFoYlc6TLtf0zzTDnybc8mvGtJxbVzTEjMnbfHHJhafBwT8Cbe0VyyQO/GnnSOZnimQy3V1Y JfnY2El/9OYnLUSR83bzx+9QBptNngX/Cg8fgsseMO9/g+DIb/izxWkczQRHT/ryVvxETnYAMrrh I1bJ6WeLEdTNLWK8MG0tqcQOYUC3rd1rxSVocJsUvvgRaXi7eFqTuqNdqgVSyWufeg17VrZdWStj hqxeqREaqB8P7fWMrTGTHnPqdpcSJrD3IP2o6KUgAH/ZXVTBMLDQjfpkATJKVIL1IoikXMkkkXuj AxR4ypHdeTvRIaeSX5NiXiabqFV62PbZGquR7mGDlRD62tjjO6HqVHh6crbzgN/CZ67M7uOfTQ5E olZL9A8262g1Ap4XnafiyOKLT+TVOUUr1s0XWYvIXbzUVOYtRzXugU1lj4QfikTec39GkcMSXJo5 5WQ0+D/JOrZoCmVuZHN0cmVhbQplbmRvYmoKMTI3IDAgb2JqIDw8Ci9UeXBlIC9Gb250RGVzY3Jp cHRvcgovRm9udE5hbWUgL1FMR1lURytDTVRJMTIKL0ZsYWdzIDQKL0ZvbnRCQm94IFstMzYgLTI1 MSAxMTAzIDc1MF0KL0FzY2VudCA2OTQKL0NhcEhlaWdodCA2ODMKL0Rlc2NlbnQgLTE5NAovSXRh bGljQW5nbGUgLTE0Ci9TdGVtViA2MwovWEhlaWdodCA0MzEKL0NoYXJTZXQgKC9BL0ovTS9hL2Nv bG9uL2UvZml2ZS9mb3VyL2cvaC9oeXBoZW4vbC9uL28vb25lL3AvcGVyaW9kL3Ivc2l4L3QvdGhy ZWUvdHdvL3UveS96ZXJvKQovRm9udEZpbGUgMTI2IDAgUgo+PiBlbmRvYmoKMTI4IDAgb2JqIDw8 Ci9MZW5ndGgxIDE3NTgKL0xlbmd0aDIgMTA2NjQKL0xlbmd0aDMgMAovTGVuZ3RoIDExNzc5ICAg ICAKL0ZpbHRlciAvRmxhdGVEZWNvZGUKPj4Kc3RyZWFtCnjajbQFVBTqGi5Md3czIN1Dl3S3dEgO AwzCDN0oDRLSjbSkSKdIdyNdUtJISUtc3HH2Puf/17p3zVoz87z9vPEx0mnpcEpbw6zACjCoGyeQ i0cUIKuuqwvkAfDw8HHx8PBiMDLqQtwcwH/LMRj1wS6uEBhU9F8Wsi5gS7cnmZyl25OhOgwKUHF3 AAD5AEBBUaCQKA8PgJeHR+RvQ5iLKEDO0gNiDVDnAqjAoGBXDEZZmJO3C8TWzu0pz99/ASwgVgBQ RESI4w93gLQj2AUCsoQC1C3d7MCOTxlBlg4AHRgIAnbz/q8QLOJ2bm5Ootzcnp6eXJaOrlwwF1sJ Vg6AJ8TNDqANdgW7eICtAb8pAzQsHcF/UePCYATo2kFc/1TowGzcPC1dwIAngQMEBIa6Prm4Q63B LoCn7AAdZTWAphMY+qex2p8GHIC/mgMAcgH/E+4v79+BINA/nC1BIJijkyXUGwK1BdhAHMAATQU1 LjcvNw6AJdT6t6Glgyvsyd/SwxLiYGn1ZPBH6ZYABekXAMsnhn/xcwW5QJzcXLlcIQ6/OXL/DvPU ZnmotSzM0REMdXPF+F2fHMQFDHrquzf3X8N9BYV5Qn3/RjYQqLXNbxrW7k7celCIsztYWe4vmycR xj8yW7AbQICHh0eYjxcAdgaAvUB23L8T6Ho7gf9QAn+Lnzj4+zrBnAA2TzTA/hAb8NMPhq+rpQcY 4ObiDvb3/bfivxEGEAiwhoDcAFZgWwgU45/oT2KwzZ/4af4uEC+ACc/T+gEBPL8///ln+rRh1jCo g/c/5n+MmNtQ1UhFRpP9L8r/UcrIwLwAvpz8AE5ePj6AAJ8QQFBEEOD/31H+w/9v7n9ItSwhf9X2 r3jKUBsYQORPCk+9+5uGx197wfLX0bAC/juDBuxpm8EAln+W/yWPAA/o6Qv4/3wCf7j8/23+7yj/ 1+X/34oU3B0c/tCz/Gnw/9FbOkIcvP+yeNpmd7eny1CHPd0H9H9NDcB/nrM62Bri7vi/WmU3y6cL kYbaOvynkRBXBYgX2FoL4gay+2Nj/h7DU3QHCBSsBXOF/H5vAJxAHp7/0T2dHOjV05vi+jSsP1Tg p4v674zyUBDM+vfp8QoIAixdXCy9MXie9otXQADgC3y6UWuw1x+rDeDmgsLcnlwAT+z8ATYwF4zf IxURAnBb/hb9gfh5n5DrE3mI66v/CEVEANygf0z4n9DTPf7j9Ltsbut/QSCAG/wvKADghvwLCgO4 Hf4Fn4I7/gOftpkb+i/4FAr2L/hUntM/lTzZOj09alAHsI3bP1LgX9I/N+8f36cyXP8DBZ7SuoId ISCYA+zf+QQB3P/2eWqO+z99eIrwx5vpCoK5/Isg8ImRx7/gU2jPfyDvUz3e/4JPFHz+gP81RpC7 y1PJbn9c2tOM/8Z/PKdgsBcYhLEwCwOJhdjXhLReV0lTenJujT2fYtwySGPl9F1w+eJ+i4OazFqZ FbTqcimdPNiFt7wpz3IhtUh773vQXIca3pL44vMvvzvzeO3Jrc8Y8xMkfeOFB9K1vdToVJy6Utt+ 985++oGvEJvh21UYc53dhXG08gmvPXsUvWp7y5ZGwma3XmxXCqpi3pV95YzRi34ZWDzNmGf1foaM HsWNkxqNjeDEC3f64nKKIGf8kVYlnh3D/zCG74Ov8Rrvu5sZn5WPuryuHeQM5MZk1IgXBCOTTL4y uykqpHO+JUXRq8/nsnlYueBWR3pp/Ybb6X6ImqgpE5DyOLgsiCiOMPh/ruUONOzGQs/a0rFwYqtx OsCZI2O38fRUUMzXbCrqMdseCEe/inBht1b3iEXV8PbdXl9AujA6uFA9+FEzbuIzZGKzgY6xQbzc KLMidx/u7E+kjkJ1vi3c4Vdon1nKUO3bbT9+6DTlte2l86WvXP7IHQcDtTA5HRePsDCjQYhxMP6i mpD/JXqlyGcrOPSXnawmg+SLCnpLXM6n2w/8P/JGjNOEVX/5eHpS1Xs7JI0XKQtYETTS2oPifxgT f0TASY/kSBTKKZwdLFOtFr1+JqWyZP1JavKzwiH8qHr4zlWneVneL2v8Z7bsNbcHzxt3JQ4ikaaN 34dbvmlqTDWgitS4IIpUEjDiXtwUN67pK9wtzAvvTYQnoZQf/WAyPMaUG6OM2cW/O3N1iR39HSUF l8zO6ZVNY4SGSUUWI4nM3eDQWaVCl6Stdrpqaa3p29E4cZ0praCmjkfc2I8fnl+UbZiMUHUnLphW PXudNmI81Wk87z3kNVAZXRNe3R9owCRnEj+1B5fJRFC/1TIjRi9C9TlUy//Gvy1+tb170vJi1p7L /H4sJndxWIm2fgW6HWO2f93nGTs4pd44xebl2DRDsm30KZVoROJwGZvLZYvwY5yTjS+PXIefoFT5 Ts7eezHTfe7KNju/8Qmu62JtaaRvg1Wq04iOWQ6XZstMBssBpFEr/b40jfkn944KPRQrx+XU7+tS 2ykXZ9IH+eIUVF8rk4R9EJIi0RWSXlb/pamFE8kl9m4lLlppiB0SZlidLeZfk3CdTbRt32tkeCE/ 515O21og0P/ZaSlORrm1aD0vUgdAcleaK+T04ksMPw8IXil9eob/xN3kENvT6Et3Skb21nveVhO3 ia+DQymLBdF1WZJ25+mRTVZZ6w9sC9QIDAEY1n0fLCIZAFqhWKGZb0UD8eiDck9LWd+V+ulNHveV HkTbkKLclBPK1Re6XeNNW6d2Fec+/5iZJTD6s/UNO2VD4ZG9UCfjAUTqMZVkg3AAn5UcfjJiWJcb bsI5gkJveUJhr/oqrhmY5CY/HvS1cSVfCnsyP4AUSN5dUKNgpi2Zwvk++f1ZV5EzSQiqWRbdr2gE NqCt59UOb0ceP3dRbk4TX7tU+cbLKiGfJUuf/Ukg1R7r20autfSTn4FR8l5BRswtA2VX+wIRCqA3 z6H2bbwbX2ZQZkFEu3GeQKyYu+uRXb4Aqxxl0/CsyE7vphrRdyPfsW1m80qX96MSt/ZPi36V2Nge MrY75f0Sh5zSatKwMPuWxs7UDON4lj/obOuAM5mF5R5PhycpN2p5x8c6tv1prhQqAmRmSmBTMS3x PwjNkII7U4+Mi3JQMA8WHpmT4yrCcOA1fScV0t52W65nGgwE/roeFxNYySrUcpSeHmDP1Z8b8lkZ KkjIBJHQhpNZ8Jnp4mDkaHlgMFFUVW127n+MN73S1s4sk/0gi2RWKXAv8uxc+sc6u2S/zAHDznI/ 675UQo1oMYsB8zYHYjjcg0yvYhkWOadLwohuXljyDIkRJAAJN0SG49V7xJIdJrw3Y+Xy9Jnup58X jNZuSy2OJfAnzSxo4OpYkiummFEc02+pMVWZcVU3prCZJXLws+TXtDQ4RNV3fSN1+BR5tXGkBF/m rElynsyYkRjGMAcs7V68USDhcrZL60a+MqLMOfvlEGRZVuI5sqDxSc9ptNiiouJ8I0NdpBrluzmR iu1XK0+6jVSskVbXnsAh3a9JcBVyHYhWQYOiBs50pm8gfXmenxMtHncLS5FQo5m3uizW7MxQurB0 dWOVy6FKb8EO3xvy8b+MJpy9u+rOtSBYuZzdDEwLuJxbATCX0DhFrOKwPEcqNwI2b1+Wv83Va5/8 WMJ9qHmLrIvBaMF0SOUbwkKm2ivKJXsIGVUVyW/3UurlJPTYkL+b5vcyF6CJ2IrNXWi5i8zo4TjP LrbecBBgthq2WnHmSFG65AJoJlbtCveuo2toQK+MexTPkFGd3UdpGvurZ9WTENrOO21/7U1bcyMi cKHCfrxm6/mSQrDe0oY71A4mcroerscAHtX4w3ADVOEOWVKOfaUpJlzVN6ktJAH4WNn1w/7keH4E Z1kEcAQlZl8Xi1l+LQLbPXLU4Q3v5qXt4+j0kX4GvxJ4vt/HlGmUdwihWfD8GTbsR1pC9j7MQ7Yy Sgvss6mAVP0tZs9CIXhDBVWTrqP8wacvr7+7QJ6LVbhAOsvNgj11DLa/sJ+E9qkCOyDYh8him87v 4U2FJdunUesdPcw4JFRWLjdlKjvn+LmHCBo2HsRc56OAluqASomdEA3ifj30Xr4+2oHYNSHcngAR TbcGEz6/mBMBFx7En81OffMPv+aFUVeTDdKonOXUer+0vvNd/mFRQ5Q4YO2ZROk3F4KtqPLow92Y tavE3GbMEPJ2AS59Xn2sUDimcO1dbJ8x3kjEO/hCNHGhVd9KNadv+N1X+qyMhATru6+GlvmCQw7T dWGFd7JQf36JRuNHl4Eq388upJ16xo8i8mU5s9OKn9bd4/lFFiXWLGDkzm2j3EGF66/3cLw2UmJ1 Qv2ybERo5z5KdhiecrkPyHjZKFrJ33F5Goj9Ck2HcXSgk36QUXv1SrXeF1XEBs3W9o4D3mHPr5yp lTLgKxmO+oAlW8E0w+Vb6naPZ8sNEodCeoegvTej3Q2o5XDMjQhE4dAmDdyJ9DFgdMZOkcG3vvIH 1JKFR/1lrNI0DXcUdpHHRVx+Ifwob0Vclsmuh9VV7JJA7K6OSPoKn6ENHJhml3mS30eQ8QOVj0zh drBUo3GANO1K282alGUB2Ctli9JKMm5Hgn6MOBn+cRDVqCP0+ChrlvTtnm6RrUydAxLhQoyI3r3C vkqlJD2BiZNpF+Yi8pDTmF30fGfuWEdCyWmYHoJwwPhMok/ezrc2tTDJ2oBbhETSO1Ea1SixTxe3 eQaPjCOGSSEXSAzz4aKkVoMiTpt1CmbWGTJjNLMq2izwgiacumw/+OmyYia9jeAsmNXr71kQMSvQ Kn8MqROWa0k/67OJD9Clt6GKpkeSBF+lPhgWMyVot/XI9x9gmpbwqgXsh/XACSJhnUrasQozjRBz y1ji2QRfpiTSkbbwRNSfuOE1qov3RiEa8sD7fAGuMggyEDf6/wo0EF8F2m9BNoQEd+4Z2virwfBV dbWAnVK1LQ6PsTxd9xe7sGbRlbPnp/7U9fJ1wyUsmzpDw6M8XIntle+A31gxrX+RRAU/+zmkjDIq 9jlAy5SU4Mtg/WkziTZ8+0Py3W5yizIiRxL9gGTm+8TAy00rmZzxsYG7OWG/5SXF8exvK8cvxVqU sDCgfesoIN6I1UovWsMxFKJcIJ2k6BrxOn5zyhYeidszmu41EknG5smrZNuQSJ12sIlPRg54FC7U HzC0aseaeit3iIVoz3kbihbSc6gZIuhx4eNeGCdlFYwe1YsO8aoD9a1YuCwRMLl07FBh4ygf7qMQ xJLHLjabS+CjBWIDdfZ1rJHP4ci2jBiOQi1YC7sqLiZ7yOZEkSNJjtqT8U0xr7UJTCcUlD/pvvI6 ybS3tcpWSrRG42IRxejcOC7PouKKnkBIZSx0cm4vO5cgefNRwmzPx4tj0dcsYRRk6pK5IQj71BhW 33yRN/Dto5tSou6y7P34HCBZU2zCeSPQVPiGgci7qVEmAqbmgh3g+9ymo+EjAzNA4brXn+0imPGG LQqH4Lbs1s3aNZ8kUxZewwfU8Dmj+FeQgOLr0TfzXU3ZYMA+bxT1tHU5FQF85KaplasPc/uA6Na4 pYF0XVm6BD8ZO8Y9QpGA++pzFRrds9uKRKUIYnjhEqP1RhJoE+HEuO9dNWcgbuv1LVq22+NXf627 wyS3HuL8YhWv2mThuYEpbaqldS7Sl75DFu4rjybEEZS09cXnjHvNfJ0RsXehQl0vZ1ygQWEaKrT1 +w9R42fp97kKFa8+1mDWIp+1adq3p36h+LS6uM1eh2oAppiZcC1Vn9FPjWItu05L+dm5hlqmcmqk fzqcKVx32AHsvEexfqaiBiSRJFSvG5AkpO/NtCoeedGJQ3Xc1mwDapyYynGp+1KJKW7r8DrmODVe wKhSdfx5oM7WqemBNYZURGGAOarAjJLrWtCvi2Xivn3us42WqJFX+lQr7tu2398k3edFzR7h+TPz OyRPJG1J3VS2m2sqrzYLkWeOc/vGmSH5eHNau4ySeWHbUOZ9GaG9JRBSWZ2cPb0OmZvquKrTeV+6 yZxDQenjS958YFX/YQr0SssK7XVfINw8Bk9edVf3ddGoD+Wm8lQFXQi2bnBTlHYZwxTHB5qgkkPu 3F0ZuMLDjUAm+TQU4c7a9P47dXRPvausQ/7U+cUejqw1SFG4kcVqMxt43tSIPcb5XD2+kK3x/esW yg6f5Rbxb96BetuyE1JhqUYFYZQ5cqYtw6uO7u/DxgR/+Wvukctz6VrVG+PPNL/OwDfni9TPrzsi z0Wpo+TX3qOPpSYprJmo3z9Zx8HBLoWFqaS+M+hkaidu9jMwZrkJ2Z9QUkMflNoOHxOy1uwu2B4K cTaICrOMPyOvXK4gzRZ1JWVE9aBho1cXH0GhSfPem9uwF9FQfomc1icQYzezgsv5MjHy5rsajdU0 cSdHpa0mlFmYePBuoKziEx0phfx0GLMY9UMokAQQvhmWNfSGmfynpXEfwGGHqmTq46XQDX3emFWA 7lepFPoXlEmeTmP2mcSJM3aGwKtaMeaLkgnkOrMiZ1zxcZbjn29uOwUUOzBDwMJ45nGWY2aYlLTm 6Nap7A7luvS+NBDBKnMPF33NJZL8MDT4OBuUlJ2coDK74FVb7nSvz3lVGpM/+qb0eCXPbGWPiF4E lhM9c8LwN1JGVTUJuumN4GibupJOO9bWGqFF1XoFEb8y0IXfSiyMw147sIhwG9OMSNDqy3AYib3U 9WV7yaOLOUrDOCCzuJWrkYtVI4nMk5TqRP7440HtpC7JsK4ae/HbjVySpQbKloRlIIFLK/9c4owU +Y11W1lqZ9KOR6UAUS3CfkuWAJi4R85Gm70R9FwLgvDoWr5kGC9M/ZAxPeHoGBWQsSQmX1Gptt9j mqXkV6GT2FVVHd4IeHYx0I7aXnxyPUxdV7y1U9XxuQ+3zGEH56KG8iZ9EzBwVXIWoiOIQ3ITuX0C LjzqbKfjgw9AkzHStcVLQ0JZRKn2J9HN07VMG4sIisgpH0ewIc6cts0p+jz9cGJYZM7ulgTM42WV nsXxXS25Pt4EBc9V5caXl/hosb0YLk3VmdPesmaj3XHQ+EE6H2tFaD8YXkqE4PnF3by3GJGMimeV jO+Debtj93NE7t3VNy9Oz2vdbc438jFOmZXRNZDyP29IDhI625edprNiDL07yJiqYc9uyJMm6J+k nnDJ3/hcKsikNJEkKzLr6ILs1y3kUkPwU/GGUTT8TMXloUH8G0u5raJZV09d7etKL609FLMA1Y1v 0+SknWtGt0ahcsOJ9NdNSQxEkjaNQjSpixmr/vjNCYbD32G7ETt8d1nMG+y6oWA/oYvXOV7XIgpt CbMoQ4mAZCJ3jJHjtCTIJpodHIaIhhs+OXVCqEjQZ3cIaNt19LtKgCcGyNivoZMImUF2wGSSBhI5 Bz4JxvxKfwX3q2atrU+Mgs9POt5HYd2I/9IAdV6HSmkoTWW9OHuP9fUjEWNWsZVJO76mZPdprxhx jrZn+Gpzv3yFQIEkfbbBll+A2MwnZcGjGIQkQwTml9aRu6o3567+vNHkvi+kWT0pVPbdUA3X3hFD +NyjtQn4EHQznQXirfxoXh/cJojv8f7qECK3kvYPDcxJp/igZjitKbjzixuPGcFClHZ8+vQLQl17 XXwJTQs88amwbbQyPMOFIkfy44i6hbe/4n5RFumQJCsoYRn528+UjEehdVVMOmGZoS/vHOAHuZax EGAXNOMk9TPuER4Yqo0hOpKjPMbiZdStZ+pm2TB5hPXZ4yRsccMIa0uwKrbZ6/61xgB2dDqctjVx R8VmjxhvWx4RieI6ehrgXtGPlxXgsY/KDVg6rviDMsq6jdEz0x5nBAvg3fo8WnTtz688HBj8KiBX hzzOTidfMF54wU26wDaVbl9E1KZ7+JzuZPYGn4AjFk75Yy1K+kfApVZXaS2S5frMlZZeJ8Q0WF1s 66Mk6P158J9fR6lvycqe57/9lHTB85XeJYPSNXNMR/bTNsLzrq/eQ1i/TKZ/TZuJ03EWARmqBNSS vqnj4ytzSjyQzCQpH8EBw62fZRGoiBwxpSvHIoY+qwaM99q+W20AsX2WwYdnL6kgXlYwgFqHqfuP axu5p1tpITYGVXhk7IAkzW3VXwkYw3pgK0TfqUpyaoO/2uSGIb4mXf2h2CUxfTdffzAawZGDl/sx bdP21aApgrwwYaMkYlrOvIVlhAa3hRNKMe7O2MYeLy8EseTWbRidhdukR2rTT0y2aBXO1NJFcHPG 8AI7xKCJA6FVNioSDk6T5PLti8O3RhYlOxYq5jKVazR2xdPOp3uEwlh0YoNcZ+P9lO1sPe1q6lcf +493Mq3ak4rUVclTLX2tYKqZytvQSb0QWBOl54wt9RlfQudMjF+8dpsztST+STnzbSnuNrzO7SxM eVb/ZQ/qkilkVtd2ICrGxHzVuN780D6M6lPBQ5JAYWUJs4awJ656BCMqu7yoRzRG5BB9CGLTVNlx 6Zv1RMsiLxKxcSOsdxa1jlhX1WN94Jiz4+NSeivXNYizBEcPhTF0iLq7JJNvuysaPDHC8qHatm8e 9quIuBXwnOFQ+6bxMaf+YfnjLXJlDjPihoooyL6MryjIhhdDI6FelE9WNk5ME2/Ig178DclArbSk yPP26XDLhCmyAV+i21v14c+RDZdM/mrbUPf3RW/IPvAv5eibjKyxc+Wbh1rtlzC4XZbHAfe5GvwH Z/unOkGTd5W+38vFxTM9Dq+Wm1ZJRnxmz8PFbzFH4U5TBiYKVuBERlxQvpHVB9JnWQW5NaQPpTF4 702UzebpHwJMDVLIuma/TksUrGzyIGbjX7yh1SqUSWg0YKM2lh3sy4zUzN7XCQxAP+Cp3QuYsYwl bQkam+1oQdnSCakWGEu+4vQXehuENt1HssDZmclG2qQs+ynWFzjQHCYdmbZq3Fv7HfUTuYtmPIsi GiYFNI1AcJOWqDU2XwSyKbBqhS93r3FVQ0vD7KnWhY0Nv8VBUGNPn7wHZ+BR660+ZmXjkdlceEPF xDETpoT6KSwYb7KhVdEkyCb0Rw8xU2KpVodXUx2TMgFZRvrjQWEs5DkL7nBq7q6PaJCJPyCDYf9W YhZp0rQub0fZweOTt9b5wp119JQJDWKQG36Lz6Tnd7FJOzjQB5GUyj1DLB3hQwLf9/E4M++NPqrw vyJ2bT5hZnBhz3TLuA/fFmINYV11ZIpSMxh5MY29qSaUyFpRa3sZ0BgYNvczfjS+gZ4roqP10QYQ gSefec+2gYFbRao+dBkkKkRVeWFP4ZiZhohVRlfPXITUTsTX7EXUQylu4gI4+l6OdPQ2l/wAGrdi lu7aMiiaY5aq7tstIp/Y20g0SYUwIjy5IrPpJr5o8Oyg1sB1CZ0vwb8LL3D7tem4hvT9pBddEdYP m6QTxoev6T3911WeXQtU2AGHEQfdw7Uc8+0vZjej6wy6P0aVwQ/nIMeVyxHRfMIhpXSLweE97kao HXkr8y6zqklFzny7AFvECh1ziDpYgiPSs9SjonJ4jWf0Tv/llmX9An+OLjFvoCJ7yc9Hhe+miOej HAsZW6ZvSn1yX1tGAzO3M2UnBzanaFT0KhZdudGZF8s7yOEbFnB27P3S/USWXRQpvYAHAxWNaoVw CRkjr2lzmR+VtwJdc4SJKKOUbnu5t/V4l5ic0BA4qJdxg86oJvrWMleUNhePzzc9MKJwqFGk+z2i G9PQ2X/yiXe8gG5kJFMg8nri0jRG+/ewYm7ZkFfk/TRcc68b6EJqzCj40mq+L5BbaisS18iGev8Q BI/6DC8d5dj6l+eys4qDfmT01XellPlSbsBP4vlvZSHUG1fntoU0c4W4MMVHgGzsya5V4QturUIW rGAzIDiiSAWTaKHoXJm3w6+dvL8TgCqRsqKym8MKn2nN7eWE8ullR8jPzWY7T4LnBnK4Ez3md6ia UaWF0MjqCLHcz+ai5SYlws/UTXNjptNE0cazGtlUHNzraTTSiftg/QE+WPUSsqy60mGnKmBypQCL sYDZQpQWUSezKAL3lnfZQch9xGYVZpK2JrnyJfMZvVGeAAz0PvfvOt56/crQ8ncnGegjbz53wEWf FWPB4XZQFFizLsU5yIp8ZWbZQSTGKebWY+0LqfUyaqDazQkMqbCTedQ/8yl1cq39sWRbi6bvsb8U d3d5W9dIdVzHFzjNo6BJ/MKmRXCj6DZcvivQq0rwYyBKJSCyvEtlgyOuTKh60bY1uESSJMfKUVpr RhNPSzW0g09bfm4eOuCwT/3Fjv6mhHLsxNHjXUZhBGqiTfMmcCKXom+Tr2v6hXmuEpIfJsZNxFYJ UoHAMiILt/3leC6Hrv1LeFYEiP09isbd8fpbMexfb8uocQsew7MFa2Cxje01Nrxmb47GD4NS5Uvq FmO5o/cjRRioRWXeZFJ8hY7qmCl0CqGpBL/8NuKgs4DE1yEpJPZWX8+ebYCKgiyZdl3G7TjfKYjX drnSybBNej+lFf2+GG/PUm9WxD3TjE11dkeZBJAepnzJh+2vPu/h4mG+1FEykNIYMOqV7Ok31G6d YA9Hw65Slfpjqy0zFTZ+dNhimVPUHfOemefNRyKZjKbbDPMPzY48n3Qxpm24QwxCM2NaXoC1GG9J z+VkBRUUVIkqPHNCqHfdPf3YhewA70vInTj9Zh5fn7lSD+NrDNlGN2PQgvHmTzNGESyGL6ny4oZ6 zX4I7e7etcqeMWkC9aZXLKt+Eovmg6TLjLuWN7alEpbnCyX4ndB94W60lz6DOhbp+kk3ldbaQASI DOOK2Ya8/BLP7CCkBiw5bSqyH/X077ZQQ9yuXvUHm6Fd+WMWXBbIH9UOoZejrdpWcGdabf3Qf6FK 881WQqQs4PTQ2ijWcHLOI6XOWxAp7hD97HhH+osx5xnhvLE+UNV9XdzZteWGb00dk7d9ZaDiFbrP I/09NfrI3EqdkMiE2eH6ZDxmmgppPkXQ+/WLuLoKEeKNFq4OmCSu7EznCyE/aPJHMbewXgVVY3A2 4m7w4YtEEhznPa2aNuKN12ckd4fvdj/sNWyUBrz1rnk4jdtVMlwkqFl7eOiO66kiUSQsFrssIMza 01kttXMDvZ3PiHx681KlPmWmaUOGNZp6VyGL8M/cGvwR9nJaFLtQbms/yYq9zIfnRkPuRrwgFDCZ vuMNh3jzDd1edESQgiHpZQwtarqiWzP82y+/5vBUXTILCGWM86Hr9GCrbJhci/PiEp11YhLd8upq vQ2F6lbXdnYdqUaRK/QPHQkZpoPg+Ehr2X7d/hLIGgotrsryUhH+IKZFnuEOv5Sx/W7IVFv/R4ZC TT7y5Ow7AVMl/eL1C1TMZnOcCwRmhWgneOj+0O33ptGLL54WIjm0aDcxRUQYzcsKBWt4CRd+VOsf FFUSeoaDFdGse4LnG0nYsdHVyKEYgkO0E/kv0UHonT5oghvZJ6cEr7YqPAvKecey/J/lU3gNaYqv +FXP+1StuJVWsPinXed8zcsb5sUoeL/8s0LII2thtrYjT9ovC3PVfI44xZ4+0w6D90T2Xh2u5Wpr xPgD8rtddQLD4w5tPDpMd9DUdk7riBEDEdTs2KPgQBqMhLh4J6/CI0yDekMc4tha0xdHQrZfqj4l ivLKbIyvxT3ibuqB5eqbz2su2R08dGaIdJ6Xf1okiyG5/HsFP5u1mnXjhs1GTNvSTp/azeWPcj1c LGHsnJcJbYKaQPE43w27aaU9DD3eOgzkRwSVcTvLQOTcQGsXwf0ZjPHYNI+KHV9WM8jl4oWM9cC2 Skl1rCNy2byW9E4HksV4aEcPvzjdhlsad8/Ue3dbEiY9tlAxwSJIg1L+Ke6LaTZtUqf2O/pNaZr8 KcbO85cZDUeAdQ2vYIfKas8IoaY9i/EyA7+0nDgGsuxeprsoF0B54A7/kdH5vgLLEBpISvbnz04c yQTx8YqA1aNUXA4Wr/T+vReWedvutcP46lmhINimS9hS1UFUgySen5YJFoLa/l7nvlYvfX0hRz5C 6TPxOeREKO2XV9jptrdzBlJHULaiS6RyGRTCAk1meXCFn1KM4f0ERfF92Cn1AJ7XJXA8g6Es4HOf WhggQtApcQjHb7FSUe/+vYES8rhdMBVKNHatGj5vNPzdithcr9/PCNdEw2O1C1+WvBXWSSxftvlB HW6NqjxRjrOqONcAl8my3aKAxm0iiejAdq0QFt/MZdJ5jJt9mbX4lPLoW/KWbnETmzty+6Yj6a3C i60vixIqnFKrX5ecMzDXlqNCsR57a6jYr/HvYFlaw5DLlOq0nPVz4z4pyt5rAQS/Aw/xWDOftXwC SqLzzU6Rg04Nl43bRQds6gVX8kRFw+sDxa4kzc0Qq5z+jY7OF7SzSwbxG/NbYVJvGuRHw+OFS96h /XCSsNBsvQ6gKzbCif1Opv/yxeoY6qfyzVgiHHabCJnAqGe1lPFEauP1VEQsbQu1bAOcNdBQuwlQ 3VDfdvOypJKnpUB0QwiT0sebg+JHJLcpdz0d6jASkdNyLpzK9UC/V3qixtxLb34oFRZ6VhfUXzFj Or66p//hLRTky9XhIzz+QCpTQxh9HWLYxD/6hadmgr1d5WT8ZENDdlcBglpI/lr/XU5Don3wy0s5 GSa4y/iNT0LNS61Sl59f8TmG1olQhmThr2qiFWh90PHwXaWm55K6wHfuQmV4l4nFQDxcx8KduieZ xhxVkOPEsk6na4DsECXrdasb2kEv65WYRHIIkdSTzMDt8JzxhtefuiYUCd+Q/8CkvlVh/yA16otd iLXBfQQgSmmmf1vvdTLY66/sNC0QDpZ3KBZNqB2jAGaSMileJBNalMg5p7PnPtxQyQ1H7SjDbjQs wp/bDm0py6fyOZJ+NkOoP45V1RRfSJDsoA+ouEZ1/CQ/8e3SO477IlvMyNbsJQ7g65bF4FQ/ETsD /JJ1mjmVO5leVbUXz45yoLuFmnSFNFMMc9wbDaRjfVl5zh90FudCaJPc0+fwrD/UabtZTzsJyaWn o9+LuEKivnG0GrJcPEcPuUafRp0svNCjGwgv8RIcj9KZCGjFrlbJHJjm51cgzRQz3NEyqWiQS3pN /JqkZrXS0yAMz2OyrZkO07DTfc2U1EuFbxf6U5qIT/51ZgaZ87dti7tsipVDsX6yFq9FYb4wUTLD X+qiPf2kEV4WDdjpxSDnOsLcyNrVkTDwjRNLzRJeFLO7s9tEqI8zo85aiHji26OKubg7tBego8Ul r+7tMcMFEX+n1TGuxY9VtnStCoUjCupeC5iN9WmMzdKDEVQ4P5CRVe33LBnbYzKVC0gSN69+mt6/ ZYjvMm0VSncQ0A9ZJvlgpaNzls15S6QhDcxINFmr6tImpQ0ZjoVvRkLgeWmcDut65fT8PCtk2mT7 WXRF+slFIo8m5IuJ1hKid/DVWEdSl/w3pGus45HtE/FIIfp4UdUZUXk0vtihaffm5PJ8ivKhHoKU LhhLSGqiQn1wA++ScpnDnugq4v7Nnnp4H6K2jcj7/Eaa3bBU1yPeYz+C3u5v8QQyL8ncnAlWpaqh 8ye1nbld9DgxzS8/NMriAc/4UCRQiQ+HM/f6Hgjt0cyPRWkj5cRvz3txN2dopfxr3AdpXV/XanKx Tjf6vJuPTsFN6v96cC6v4rgmioxheprdZ07rod2xi5d9P2JsXvtRfLxgNJhcUE/dF6M66lTndTYd uNGzeWjbBodpe2LmclIU0x6yk25ZJsdHl6S0lWtiKY6xocXdWiXnRbQmPU/UzthRNquWbaESOAOs Ps+nYVgnjVHtCgE2YpmAThu+PMygLTjCm5anGMJupg+zOCdyGMLluLMD0L9OJqqfvaXGSlbN3NIK bY/gMbfIRXmbclGMJjLLNdYjhIZEIOhD+5h0kvPFDD+7/PBFJR3ZMk79h37ZSdW1UtbOzsgeYtZV hKIN/92+rKjrhBS7GeLXgq7dHpYOQlL6WWKfcQT15wtTe3ob91bueWhxUhwFCnvNBi63W0AVWcmJ jcIJ2HybFbLLC3vwqCDF0wGLWRdqirhGkHjNEjAk1Ww2HbvW3aty6CXOLpG/uJzk8/zZ4x4SAE2u QUW2w9esfaVZSKA3b8vw6YqR4WRqxwZ4E3jwGKeqNgaywyCP964LW0G/HO3reMtV5Gcbn/0zf2Uy iiH1s4c+Gs0M4GfsoF8bBffEVwfNTUDHWk0YqEnxu08rkSZ4cYTbwKkBIW3tw0fsn52SH9iuD6BU LnvfxE3vJZ4fxYTQ5gKTk3BBefs2Bnj6FB+PzA96VnopsFe7uYKMoMglitfE90IHmHM9kiwN6ZJj wA9yVYFcaYaIy1SeZLwOKgNhEgig3p/0C2Ly/NfH5DB549k3nNsth9TDc3Z9eIbR25FzZT9Mh0LM s7PQVw0kJ6vP5N70qjfaZ33JJa84+IQZtFEHtwHLtDJcP3h3OnhnQpW/9C5nlErEwY1p5llDyuxD 3tyqlwhZXCT6xTZPLm0PCTBo+guyyrk4fku4w0k5ms2ZD/I3lTzXPa3M2+7bkWcXvvqHxBNQGkWE r2D9Bj7J6yXDrFeaMr2tTuEEBQmPb6TiH9PLMI2NGHHQc11YnShUGSD4B0zBehQnCKUWhseZc/EJ TNMKFTpWQv4bYp3kdfFKtL0O/TZqGR9ju82Olt+XgrsL6hGhiPT9n96+w3u5TcFu8cYU/Xj4ANtF q/SD7MMokrDA2XtbeiwSDTZFmsiyptXBebfvz2a+iXtJnTkZHsfTskqRdJSb8Angn2JA9zTuvuu0 DZlxcCd1GU5SbHcmPsw1Spjg6GQF9bC2fLl34Uc7YVWYVIsjP/GVTn2dbKyZFi2M9Uq7S05AtoJQ pGZfS+guuyh26FVcVNPm5tX5LAmUsvQGIFGbml46/pA50HgcWnAoDJcSVo/j3hajlMuwO7av8mNX bxvfB7QVB3+iPYNNhlfm621mOoPbGnNAw4WXagiPrX+P9pPwe3g4BlQ2y/Ph5kJnt3oV9gUauHha EBwQL1j+Ia3fUDkvuMk5rmri2CWVIosn6/t9AMOoAnoM58TPkcMO+HjbttXJ6ATX7GyHwYgrcRtb qyGgvZKgbmBhzpe9jhGf52X5YAxYk6p+WmEO+NBB31TGfBL5cjkC1+ANHZ5svlI/rE6iFILi8iZl czIwjq02e6k6gONYNmDPoz6igHjp0JstTFq+9pt23L6zYJIPD2NgNrrLqYDHjRh6DWCnAKvaYHfE pDT54Jvsxl3l9eaE07B08EX3W4V0h+1sPqT8bOaJuVkyFlyHxG/o9HI9vVqOzBZF4aGxvcDQ7Oc+ tVcUr4piEqSOP3zdEGdzfHeGhfqLeLjML1c7p3F0My93SXBaoyyLX6fQlBEQV6GC15Jc4Fo5rIWn Nj751pEZRE/9eoHzLlifX2dXdfstD/bFz8d01oKBvtFnIV2dWcFv2U/zBGDvSZacp0veISohVYST RncMNeXEyQbRmaibznmVdReFh2Q0bTmxMG3XSatQ4u+MF8rSNQ+WBjkkr60WZvvNvpgpDVSn8iuE 2wUomhwRew4cNoEaLqNhjtxRDWFtu/GZzamEO60sx4twZ5TL4T7l0yCu7aeFey8+NRPdanBeuVIY O1mNyPv8eRF2Hr29epRntHBycMvF69DLK9X0G/xQlyCvJq/igpmUXCD06nMhxVHbRHzlMlC+O1UG TrUQCniI+15MVj+u/2A3x3G/PSaRe9t5m/gR/ZCcXe+GKRPTEArmRL0yDsB02j3sbbsXBoq+X/KX LVH/PIpvhB275LgRqSg9SuzEoPQ9jDBCgR+8qmKZifmJ2O7sKx3G19p4lr0a2akWRCKa4LBGuytk 7DHC0CYn0vr/A+0rE30KZW5kc3RyZWFtCmVuZG9iagoxMjkgMCBvYmogPDwKL1R5cGUgL0ZvbnRE ZXNjcmlwdG9yCi9Gb250TmFtZSAvWEtZSkJPK0NNVFQxMAovRmxhZ3MgNAovRm9udEJCb3ggWy00 IC0yMzMgNTM3IDY5Nl0KL0FzY2VudCA2MTEKL0NhcEhlaWdodCA2MTEKL0Rlc2NlbnQgLTIyMgov SXRhbGljQW5nbGUgMAovU3RlbVYgNjkKL1hIZWlnaHQgNDMxCi9DaGFyU2V0ICgvYS9hc3Rlcmlz ay9jL2NvbW1hL2QvZS9pL2wvbS9uL28vcC9wYXJlbmxlZnQvcGFyZW5yaWdodC9zL3NlbWljb2xv bi90L3UvdW5kZXJzY29yZS92L3cveS96KQovRm9udEZpbGUgMTI4IDAgUgo+PiBlbmRvYmoKMTMw IDAgb2JqIDw8Ci9MZW5ndGgxIDE3MDkKL0xlbmd0aDIgNDY0MAovTGVuZ3RoMyAwCi9MZW5ndGgg NTY5NiAgICAgIAovRmlsdGVyIC9GbGF0ZURlY29kZQo+PgpzdHJlYW0KeNqNVAk01O+7lyUZhES2 +GYdssxgLJN9J/tasjRmBiNmmMU+lfVnpywRIknZSnZZskRJyb6myFa2EIWSO9rc3//ec+49c87M fJ7t837e53lekVPmVlKaKJwLWg+HJUpBpSFwQNvE2hoqC0AgctIQiCxIRMQaQ/RE/7GDRGzReAIG h4X/twhtPBpBpNh0EERKoAkOCxiRPAGoHABVgEMV4RAIIAuBKP8JxOHhgA7CF4MCTKQBIxwWTQCJ aOO8A/AYN3cihefPXwCMFAegysqKkj/TAU0vNB6DRGABEwTRHe1FYUQiPAErHBKDJgb8qwRYxZ1I 9IbLyPj5+UkjvAjSOLybmrgk4IchugOWaAIa74tGAfuSAVOEF/q3NGmQCGDtjiH8cljhXIl+CDwa oBg8MUg0lkBJIWFRaDxAYQesDI0BM2809lew8a8ASeD35QBQaejfcr+z9wthsD+TEUgkzssbgQ3A YN0AV4wnGjDTM5Ym+hMlAQQWtR+I8CTgKPkIXwTGE+FCCfh5dASgp2kBICgKf+sjIPEYbyJBmoDx 3Ncos1+Gcs26WJQ2zssLjSUSQPvn08Hg0UjKvQfI/G7uJSzODxv0B7lisCjXfRkokreMDRbjQ0Ib 6vyOoZhABzY3NBGAQSAQJTk5AO0DoP2R7jL7BNYB3uifTui+maKBHOSN8wZcKTLQZIwrmvIDCiIg fNEAEU9Ck4P+u+PfCASFAigMkgi4oN0wWNBBdYoZ7foLU/qPx/gDFyCU8YMCkP3P33+OlAlD4bCe AQfhP1ssY2qvr21offq35L9OLS2cPxAkBQWkZOXkAZisPKCgDAPI/67yV/8f7T+t5gjM77NBDuoZ Yl1xgPIvCZS7+yPD9/dcgH8vjTjwbwZTHGWa0QD4YPgdIDAIkvIF/X+vwM+U/23y96v8n8P/nyfS I3l6/vSDfwX8Dz/CC+MZ8DuCMs0kImUzTHCU/cD+Z6gd+tc6m6BRGJLXf3oNiQjKhmhi3Tz/XiSG oIfxR6PMMUSk+8+J+dMGSnVPDBZtjiNg9t8bQAoKgfyHj7JyyEuUN4VAadZPF5qyUf9m1MUicaj9 1ZOFKQAIPB4RAIJQ5ksWBgOCoJQdRaH9f442ICONxREpKQBFHRlwxeFB+y1VgAEymvumX0gBkNE6 QIqAjPYBUgJkdA6QMiCj+xcpQgAZvQMEBWT0D5AsIGNwgOQAGcMDRGE/e4Ao7MYHiMJucoAo7KYH iMJu9hcpUdjNDxCFz/IAUfisDpA8IGN9gCjsNgeIwm57gCjsdn8RjJJHoGzpH0zZKpmfzycBiaO8 qvvmf3UFScLjKQ/Wz8WhtOwP/vk6otH+aCRodAiHPBPuUR7esFWmyesnNdstK8/csR6zccRuRpfb d94iVFtl+7PcndgRzeaBudjtyWOLsvf3ai6d44RccnoXnBEL5xytbKBzdZmWDO45prqzSDAmyc6d ffLE/qIcFuXG+anW8rnfhka/+AlJn7XZwayP+iqitQ++lrp/oherXpJmou6Oj6XnUlKSY6IOb9TM anyg9Uizf6DJUJiwrtG9nHxS7+N2RnuXLOx9x1iR5CHMVAbvwgKz99mNrAW19A+tCjxnL/YA/SIz rbpc35LAoRsN4SFgF8JL86S78jyu1us1i3dJZoLhIfja5fIZg9fFJbLTfeob1pZLiuESbheuzUqz jBlrHXnw+D1etvpIkjm4QeW8lG5Uz2UttwevvtxVFzpP0Mg1DDr3ZE2Mv3to2KYHLC5Hhoawwy4t Ww4iBaIwd66vbazkldFFxU2t74wE79z21sg20q+onn7B8pnyNKxvrgs1v5jK/fYW3Of/ns7lfDrE 9ZNA4rmveXlvbsvYS5QtFZyrdOjfjlBhi/Hn0uvmWxJFpWYPhfro4AUl0s38VOOcVWIeWh6HPmU0 WPVrndRKTbcQrv08IfEN97mfWb+IPbr5Zraf/LXxtUFy8Qa+y3miNtKppF+ktpS5LvBN6gZNBbH5 RewHBmzCF4vLkqrN7banNp5IM43xPKozWthgbYr6RHPKmK16J/t9Wea5h7z/vOYaZ/D5J8090Mfe 4y57jkOFNukBtVlv3nTCR/fjHQ6TbDiIzPPtjTOuzBIOZYIuGulicys3hn70qpUJaQqfhZofIhbg P0eEu94M92G8OoyMyrN6q/yBBW7/RSyz7l7/0V7ku+MM800m63NVuxv3bPGbb2ZJR4Z4jHOUL4SQ yLwlYAbRBHzNVePEkI1K3zO7U/m1ZMTpEKxfiLUViCqvd0Vc5ta4wXfRYd1KPe4f/3zUQ7mw50im 540/POPv03Fjc8foDpspCAD5XVqqTzktpT8jWK3C8WA7yuyGzHaHd/v73BAHgWezmu2pVz7d4Lne 1ZRd7yr6Dm4My3/UPrfcqEPSuokuiSgqoD8eY8Ux1nGLFTt5ZW8w6YxR6rVrEQYfVzVRASpUetNW 10/4Py+qarfKUwkObVmdF34Xv12iP2POdxRe9ZYBW4xX1oTK63gM4fwFtZOGV6+age++gyRFFA08 C62JgXXHN4HEGUuPRPQM+xmRn39iGXzosYdiUDzPnplcAMPFY7Edvfru3j4sBUuCzXolDuUN7W+o 6NE/OkytO09dqdphk6lLhFg9dmG0ZhlJpwp6Og32PclplK6dOg5YrJZTmcLaCXwj1kscIPYCDFHL 385NbK6Vf7iRI7Q9zqZsF8RawiZWmXy7dJwmkT7A3NKkxj7Mprmy7cv3NwsPlDyXaU4VsdHs7TLp BZA3r44dBZl9siBzk0TmVDkeClS9KF9Omri+wHQCxYV5QR+BQYhtV0wyk+C9bqdzwDqJ1R9P0+jO 9pdepmI4JWHZD2fERzI5IPMqGtXzKiYJ8Jxe97CyfgSKmt//QVS0vHnQ3Bw6rAByI76p+UWl2pBQ yzV/4BgsluVmgH44WoRqzXJbBkzTCfX5csd5cvLGTvNhzmGWcasJdpjYNmK5SbeO1yDUlZSbVk/9 hqlWNZotB6Kyfmgvy2cc/6DVpoff2cJhYBslmfhxzHykrJKcU2iuoe6uw84flrwnFGARVLB1u3LR 4ngFpIqUk/NRSDQfnco/VLE0wxDbFvJYVi4Iotbmc3yBDW4aB+deZn/Np9Prcuh7OEj9vaTl64LE +MX1hYx2Wtah1ruQNWqOvrMa5sRY7hJJsSwnl1IH4QxlugDON6uI5IWYs51a8+YveiOGjSAEWIJ2 BPGteLFTen7lEeccGh/4qMKrH5uRq/5a8pXwgoJH3LfpLu5IS3+ndl8Ozf28p6LVll8yiPCYhaXY pWHP91TkgenSOdhXzrJYGnScuJSJUrv0HU2r6BW+/cRdAZfbZTx7Oo9jwmaKPdPg5Wy+vbFf/a38 zViBezosPBMf3fzLRNbXBLnXOf2rvfxeDPLoD/IcufxI3Nt1vN+oPfkTzuD4rAq4JFMa9OlqQwN7 zF38t396OEBkWvq99lE7Fx3csJQsrz8be/yK8ZmeGefHtZ2CSpfzdPfq1906uYV8BHZ4liCXAy0d fB5kDD5ku5c8zrAZ0990GKL+j0YsRwhPhdTj+BQ8C1YPLq0xUBPq3ChHde8Jwv7Zo4ss6QyFtaLz R8imd6rDXoLf8gwwpyvh22PUvpBTJ06qWnI0CJQm15yVZhjISonMK+4TVKOfdVD1QTA6iabr5voF nZDSK7t6jbOM6YT1BwtHDXrpcIe0Ouf2caSwxZwf4X7pxawpk9YF3qkZzpyd14Yprk8NnfPJZ2hi JOItxHh15V/0RJ2ziUf6k+9cZ6naeD7D86Fby3ni/rsfUq9Wghrrp/J28TRbH0RsDak/9SREioX6 gj8bhFmjD4+dc6nKfCzVYCQCYqpl7+tM4osjyOngcjtfFjZlOR5u+HRl4CmLaYF9MEscQw0Et15w S2ZzeoEwdkpCtVk26cqF3N3LxUchyNq8MfmuoCRLjOidKfhznYw+S77TmHYQLGHt+9gsuugS2Srg MHtFlpuQSHveQoUzn/d1u442ThvFyfzOtiPpwwnyZDLv5dEJKruq7qOP1VP5CrjZBMKshJrwqJIp 0qFdJQyv9/1nK+WQvm5wD6+TntMR+mrBgYGlUybfxo6WsvrOXZavzWNouKn9zYXVw2NF8XYl4N7I A5ufkZpqqm+XftvXdszhkpRYispSDPQTyJa2Ij4AdR4Nx6yd7dtxOZeSg5bCsuygFxnGDxUHRA4H nWG6zOfnCtJJjDOGzK9uB5orz7XU8ny9oDPiv+rhVxmXfb6iwx1r0tTHv31tjTiA2CooLW5xCPKo beZyOBRBcIWoexS1NLDEMU3ZnZ5OPL/BkbucANIPigsCHRMykOXNWZiz5eLxlf/Kq9buiXcYub4y n2y0mCwqKZlml5lx7WLl9ZMKR1q8UL16mvmKpvDy8mK5by03Q7WTzQfEXztwn0F2sVFJxI0ctan+ kmn5XkD8h+l8QglAvpkZo39SNE3t+7DHnfnSSce3fQ9jsm9Ba9XXppy3nHoJrdeaj+h+xxTiUk+p VEDfKa88EB1V2HB9GfzCfCuiNYYZaQWR0Qs9c62cP5vhh+jWMfqrbwTh4o+iHRtjBDUPv/IDR3d5 KRilFxZ1jftxlT0LsZYA0ke10lca1c60vy8y9Vt4TRvkVNl1o9p35gF5AIv8YFMtSmuoOXSRXK2k qR42LTz21HHXmJ9Wwf5r6PN0A7t3jUtHZQW+m8fdLpTsqM+0rT9KQ+/S0Gsb+2BJwXn2JI1rxPYW HeFTW/7dMerCydutnBDOc9rQ54I7k/hLMhceepcwmkwkv9q5YH0W+UIxo8sjfuhjhLrh4wwnggV7 NG76AunbWh+fUA2kLA41dTtYmvqGxqR1CK/anU36WJF0a+rZj/aKLmk8fLZ3zC/QSeQFfjHosn5M KrRvlJi0IOtVFML49XbWT/ApGsFGBUqM8+SFXsk0ibsVBjK0peQ6MsvySd5gb87bYxkyy1tuybYW zE66dbjEq0CDLCoe0hY5ybScDlcrhyn4XZQwad+uZ74stdy1G8HRXVy/FoZ8WJN4sf9GDeuHJc+O mzqYoQ+f+PHz5T0skophr0naz69k8O2MmxtgGhV2kon5Loc+9OmcuEzgT2SZOBRa9ch1BMY+DkZ5 stjcTZzkJkWSa29F1psyl78rqTsX3TIBina6B7bUydbmNH8iZlNf8Zg50D3yfhtJXWcBXfyEOrmR c6fLwX/YPXOmJG8vdKU2CTzcIw5btIHv3aERXIQbfwiLe6qSf+ySMT04bOhWdGZpAuMI3xdVqePX z23VpIRr5xGiQCnMUVeYvngeq4q44sewI+sxml+Q9PH6VoKyArZ14v5FKlbGk7qx7izod2/dHOnV NS3nLqo2rD5bHxAx2pSuUvbi0M0u1UjTcuUFvetRgxfpVDM7dcaq36Vm5e7aaIkWTels3m5vWKtc Vg/UZwVqbeUeZVyZulKd8OzJzVK9hadG2nPzXKtqbU1UD9PlneyG+oXh81SLvVLPB79IpgwUZgbm vjFuwmbZjQts1sXXDm5fVeWbfSblbctdt/SOAzU88DB1UqOd6pmM/8lwnhcV96hVQoTGkJEvPYW+ LuYGjDCGVah8nt5IvTFyV6iINWqiWqeMSla6fPtRz2mY/OpmoaWWWl5iRc716+B0ZuOvAlks5pWt ndo8EmfduhwXJ6uoXSTY4EczrGwYrixHFPsbSdjVjU3kOa64O0lLRW7kLGsTiZK618Okji0e54d0 G47eN/mSEAob5RhfDBL2ZOtrsWm5GviYmPBEvasqyHjucs12whH1CVZGE52djERV3BCkrSSzto73 kNDwnq83/z1H75c2/jDabvtyM7a1K1wLbuK8g98yzNXl7FWfSsRLxB3dOhwFZxP4sffsSIz74GnQ +WvU9dwoR+5J35vfO9uC8K0DChlrL0uVTg2/10WpBqbdY8Z/qRwxSD2lczFzXO2Y3ax0a87bwcDQ GhZmB4IHNTRYsBl6NqptEJJwy9pdZ/RWBEPfyETJjIFwR2KL7+CwznpP6psRZ9aCqvZp8ESL58TJ LVsHrCrxEOLOK0K7sQWVw+GpJv/QU3s7dpsJQ6pNj6m8tt4aLnsYn98Mbb2WEtGZNsc7QY85nS7V ct/YpKsE1ehteodPloEj5Tkj9VKpbtBL9rdWftEONLJI1qNCQeDO4IlKxMmhwSqD9y3CBk4mODNx ZJVsZobpel1hzwtn2iDuQo17XdMkxtj5iBnVw6Hv7Ua7lS7bkoUmvTKJacVUlqorEaXao9YKLaEp StcE+jVoVyo9vPHySAg5yDUisCTN4RzOvAhwM5ZXTxgPAZot2qqdlNs2Ht/EUHc9OrmrfH+mAdpb YidSHbhauWtuDERoTyx89/we5n5bofTeIkyDvybU/XaOu7NA0s3FKH/D4GYFKo386MKh3eKYjAVV 3J3PZ5eYUP3YLMZ5OoaSd95AS/37RLUPGF5AN3iw073v5XgihCuIzs5PwHA6MShydyUOPm70VcJv gJzxEnqXxTAx5/Rnkm8Cn0q97RgTSnIWvSJrl2+2fb32EdsDQpXD2LXjtVXHGCyQm06yb1NPhzru 0KSQl9jmTs2HlzN/Zwh+WKIp3DEpshHc8VQOwN5+OdfafbSsqlgRDf3nRppAjaxPtu/gNvR82Xjj ybGkeHuQUR6Oe0seYL6dsLDTdetC9wXhi3trjuC1S8mstcdfgTEC5agikdSTVw+veE9dE0soCxd7 lQRIPENqhTh3RUtqR/14FWUswdkxtV6fW4EuC6J9c+l+lra5ex+MygmfydxZFRpj5ORzHtV6JB5Q VcIckw/a5tsc5nJzE5GmkR1IV6sZso77Edw+lD3NWcZ+pcJvvYqkk7THKl0Bnk7KApQ2pdp9Hbsd xc987GsLN2Ytzc0ajn+WJnBjNeluR8eiMC2eeOx0GDMsoi4m8Qa3WS3dRJ73P9uj70gXun3exrfv xTZr0xjGRSh+kFMfqrBASppgjcrrNVhGXhzqf8U5uf5cRZ8zEn0+m1qhLu3Ec3XDqDBaswu3Rhnd 871eKucYlLksrz15svRyYfnQqr3WDj76OosgUx073NCj2iyUCsol3egLIQf/EHMT/vrak9+bOfpq fAf5x0liD0ew/ze6rmGBL5fbxSp112LW/PSptKLK3Njjr6jRqYjScTV6ndM3INJB9aWEQt7mZsYW xDDHC5DPssnF/1DPCljg9rIZX+vPG43k9mM8ZVBzk8HFK3bjpEkvPem2nn1rdU2P1H2nHjyw66JR XkqKfVYppHsM4ji4tX46WIM+wM939/Ce0cxUE31sXNUD3+o1sa2O/o7s8CydVeeELPCHbObX6cIB pd/2NIuLSifexH8Onfo2AgNf4Cmi2vrOVPg6y80Dwb8Zfjuz9v05NCLSKgYXHfgqcTdnjH6CRemY g7nt9j9FeRMqTjFwB7IF3/qY+Cjf1WndyTKpx53nBFytYlgMjPRroBk/Dn29xDjIebPWSPZZ9TIn d4oKV4SGV5iBfbES1RBvfB3dcmZbL23IVZIVV2X7nG3mCbnkl0dzJmVsS133ThhJ2Qzh7uE3MDa6 JptFdhMcGifCosm0nq0fQwOPfiDS6jev3Libms2ontbzX7R6HPUKZW5kc3RyZWFtCmVuZG9iagox MzEgMCBvYmogPDwKL1R5cGUgL0ZvbnREZXNjcmlwdG9yCi9Gb250TmFtZSAvTlpHQ0lUK0NNVFQx MgovRmxhZ3MgNAovRm9udEJCb3ggWy0xIC0yMzQgNTI0IDY5NV0KL0FzY2VudCA2MTEKL0NhcEhl aWdodCA2MTEKL0Rlc2NlbnQgLTIyMgovSXRhbGljQW5nbGUgMAovU3RlbVYgNjUKL1hIZWlnaHQg NDMxCi9DaGFyU2V0ICgvQS9CL0MvRC9FL0YvRy9IL0kvSy9ML00vTi9PL1AvUi9TL1QvVS9WL1cv c2l4L3VuZGVyc2NvcmUpCi9Gb250RmlsZSAxMzAgMCBSCj4+IGVuZG9iago0OCAwIG9iaiA8PAov VHlwZSAvRm9udAovU3VidHlwZSAvVHlwZTEKL0Jhc2VGb250IC9QWUFDSEcrQ01CWDEyCi9Gb250 RGVzY3JpcHRvciAxMTkgMCBSCi9GaXJzdENoYXIgMTIKL0xhc3RDaGFyIDEyMQovV2lkdGhzIDEx NiAwIFIKPj4gZW5kb2JqCjQ3IDAgb2JqIDw8Ci9UeXBlIC9Gb250Ci9TdWJ0eXBlIC9UeXBlMQov QmFzZUZvbnQgL1RDTURZRytDTUNTQzEwCi9Gb250RGVzY3JpcHRvciAxMjEgMCBSCi9GaXJzdENo YXIgNDgKL0xhc3RDaGFyIDEyMQovV2lkdGhzIDExNyAwIFIKPj4gZW5kb2JqCjUwIDAgb2JqIDw8 Ci9UeXBlIC9Gb250Ci9TdWJ0eXBlIC9UeXBlMQovQmFzZUZvbnQgL1dEUE1GRCtDTVIxMgovRm9u dERlc2NyaXB0b3IgMTIzIDAgUgovRmlyc3RDaGFyIDExCi9MYXN0Q2hhciAxMjIKL1dpZHRocyAx MTQgMCBSCj4+IGVuZG9iago3NyAwIG9iaiA8PAovVHlwZSAvRm9udAovU3VidHlwZSAvVHlwZTEK L0Jhc2VGb250IC9ZUFNRVFMrQ01SNgovRm9udERlc2NyaXB0b3IgMTI1IDAgUgovRmlyc3RDaGFy IDQ5Ci9MYXN0Q2hhciA1MQovV2lkdGhzIDExMyAwIFIKPj4gZW5kb2JqCjQ5IDAgb2JqIDw8Ci9U eXBlIC9Gb250Ci9TdWJ0eXBlIC9UeXBlMQovQmFzZUZvbnQgL1FMR1lURytDTVRJMTIKL0ZvbnRE ZXNjcmlwdG9yIDEyNyAwIFIKL0ZpcnN0Q2hhciA0NQovTGFzdENoYXIgMTIxCi9XaWR0aHMgMTE1 IDAgUgo+PiBlbmRvYmoKNzggMCBvYmogPDwKL1R5cGUgL0ZvbnQKL1N1YnR5cGUgL1R5cGUxCi9C YXNlRm9udCAvWEtZSkJPK0NNVFQxMAovRm9udERlc2NyaXB0b3IgMTI5IDAgUgovRmlyc3RDaGFy IDQwCi9MYXN0Q2hhciAxMjIKL1dpZHRocyAxMTIgMCBSCj4+IGVuZG9iago4MSAwIG9iaiA8PAov VHlwZSAvRm9udAovU3VidHlwZSAvVHlwZTEKL0Jhc2VGb250IC9OWkdDSVQrQ01UVDEyCi9Gb250 RGVzY3JpcHRvciAxMzEgMCBSCi9GaXJzdENoYXIgNTQKL0xhc3RDaGFyIDk1Ci9XaWR0aHMgMTEx IDAgUgo+PiBlbmRvYmoKNTEgMCBvYmogPDwKL1R5cGUgL1BhZ2VzCi9Db3VudCA2Ci9QYXJlbnQg MTMyIDAgUgovS2lkcyBbNDIgMCBSIDUzIDAgUiA2MCAwIFIgNzAgMCBSIDg0IDAgUiA5NiAwIFJd Cj4+IGVuZG9iagoxMTAgMCBvYmogPDwKL1R5cGUgL1BhZ2VzCi9Db3VudCAxCi9QYXJlbnQgMTMy IDAgUgovS2lkcyBbMTA3IDAgUl0KPj4gZW5kb2JqCjEzMiAwIG9iaiA8PAovVHlwZSAvUGFnZXMK L0NvdW50IDcKL0tpZHMgWzUxIDAgUiAxMTAgMCBSXQo+PiBlbmRvYmoKMTMzIDAgb2JqIDw8Ci9U eXBlIC9PdXRsaW5lcwovRmlyc3QgMyAwIFIKL0xhc3QgMzkgMCBSCi9Db3VudCA1Cj4+IGVuZG9i agozOSAwIG9iaiA8PAovVGl0bGUgNDAgMCBSCi9BIDM3IDAgUgovUGFyZW50IDEzMyAwIFIKL1By ZXYgMzUgMCBSCj4+IGVuZG9iagozNSAwIG9iaiA8PAovVGl0bGUgMzYgMCBSCi9BIDMzIDAgUgov UGFyZW50IDEzMyAwIFIKL1ByZXYgMzEgMCBSCi9OZXh0IDM5IDAgUgo+PiBlbmRvYmoKMzEgMCBv YmogPDwKL1RpdGxlIDMyIDAgUgovQSAyOSAwIFIKL1BhcmVudCAxMzMgMCBSCi9QcmV2IDcgMCBS Ci9OZXh0IDM1IDAgUgo+PiBlbmRvYmoKMjcgMCBvYmogPDwKL1RpdGxlIDI4IDAgUgovQSAyNSAw IFIKL1BhcmVudCA3IDAgUgovUHJldiAyMyAwIFIKPj4gZW5kb2JqCjIzIDAgb2JqIDw8Ci9UaXRs ZSAyNCAwIFIKL0EgMjEgMCBSCi9QYXJlbnQgNyAwIFIKL1ByZXYgMTkgMCBSCi9OZXh0IDI3IDAg Ugo+PiBlbmRvYmoKMTkgMCBvYmogPDwKL1RpdGxlIDIwIDAgUgovQSAxNyAwIFIKL1BhcmVudCA3 IDAgUgovUHJldiAxNSAwIFIKL05leHQgMjMgMCBSCj4+IGVuZG9iagoxNSAwIG9iaiA8PAovVGl0 bGUgMTYgMCBSCi9BIDEzIDAgUgovUGFyZW50IDcgMCBSCi9QcmV2IDExIDAgUgovTmV4dCAxOSAw IFIKPj4gZW5kb2JqCjExIDAgb2JqIDw8Ci9UaXRsZSAxMiAwIFIKL0EgOSAwIFIKL1BhcmVudCA3 IDAgUgovTmV4dCAxNSAwIFIKPj4gZW5kb2JqCjcgMCBvYmogPDwKL1RpdGxlIDggMCBSCi9BIDUg MCBSCi9QYXJlbnQgMTMzIDAgUgovUHJldiAzIDAgUgovTmV4dCAzMSAwIFIKL0ZpcnN0IDExIDAg UgovTGFzdCAyNyAwIFIKL0NvdW50IC01Cj4+IGVuZG9iagozIDAgb2JqIDw8Ci9UaXRsZSA0IDAg UgovQSAxIDAgUgovUGFyZW50IDEzMyAwIFIKL05leHQgNyAwIFIKPj4gZW5kb2JqCjEzNCAwIG9i aiA8PAovTmFtZXMgWyhEb2MtU3RhcnQpIDQ2IDAgUiAoSXRlbS4xKSA1NSAwIFIgKEl0ZW0uMTAp IDY4IDAgUiAoSXRlbS4xMSkgNzMgMCBSIChJdGVtLjEyKSA3NCAwIFIgKEl0ZW0uMTMpIDgyIDAg Ul0KL0xpbWl0cyBbKERvYy1TdGFydCkgKEl0ZW0uMTMpXQo+PiBlbmRvYmoKMTM1IDAgb2JqIDw8 Ci9OYW1lcyBbKEl0ZW0uMTQpIDg3IDAgUiAoSXRlbS4xNSkgODggMCBSIChJdGVtLjE2KSA4OSAw IFIgKEl0ZW0uMTcpIDkwIDAgUiAoSXRlbS4xOCkgOTEgMCBSIChJdGVtLjE5KSA5OSAwIFJdCi9M aW1pdHMgWyhJdGVtLjE0KSAoSXRlbS4xOSldCj4+IGVuZG9iagoxMzYgMCBvYmogPDwKL05hbWVz IFsoSXRlbS4yKSA1NiAwIFIgKEl0ZW0uMjApIDEwMCAwIFIgKEl0ZW0uMjEpIDEwMSAwIFIgKEl0 ZW0uMjIpIDEwMiAwIFIgKEl0ZW0uMjMpIDEwMyAwIFIgKEl0ZW0uMykgNTcgMCBSXQovTGltaXRz IFsoSXRlbS4yKSAoSXRlbS4zKV0KPj4gZW5kb2JqCjEzNyAwIG9iaiA8PAovTmFtZXMgWyhJdGVt LjQpIDU4IDAgUiAoSXRlbS41KSA2MyAwIFIgKEl0ZW0uNikgNjQgMCBSIChJdGVtLjcpIDY1IDAg UiAoSXRlbS44KSA2NiAwIFIgKEl0ZW0uOSkgNjcgMCBSXQovTGltaXRzIFsoSXRlbS40KSAoSXRl bS45KV0KPj4gZW5kb2JqCjEzOCAwIG9iaiA8PAovTmFtZXMgWyhsc3RsaXN0aW5nLi0xKSA3NSAw IFIgKGxzdG51bWJlci4tMS4xKSA3NiAwIFIgKGxzdG51bWJlci4tMS4yKSA3OSAwIFIgKGxzdG51 bWJlci4tMS4zKSA4MCAwIFIgKHBhZ2UuMSkgNDUgMCBSIChwYWdlLjIpIDYyIDAgUl0KL0xpbWl0 cyBbKGxzdGxpc3RpbmcuLTEpIChwYWdlLjIpXQo+PiBlbmRvYmoKMTM5IDAgb2JqIDw8Ci9OYW1l cyBbKHBhZ2UuMykgNzIgMCBSIChwYWdlLjQpIDg2IDAgUiAocGFnZS41KSA5OCAwIFIgKHBhZ2Uu NikgMTA5IDAgUiAoc2VjdGlvbi4xKSAyIDAgUiAoc2VjdGlvbi4yKSA2IDAgUl0KL0xpbWl0cyBb KHBhZ2UuMykgKHNlY3Rpb24uMildCj4+IGVuZG9iagoxNDAgMCBvYmogPDwKL05hbWVzIFsoc2Vj dGlvbi4zKSAzMCAwIFIgKHNlY3Rpb24uNCkgMzQgMCBSIChzZWN0aW9uLjUpIDM4IDAgUiAoc3Vi c2VjdGlvbi4yLjEpIDEwIDAgUiAoc3Vic2VjdGlvbi4yLjIpIDE0IDAgUiAoc3Vic2VjdGlvbi4y LjMpIDE4IDAgUl0KL0xpbWl0cyBbKHNlY3Rpb24uMykgKHN1YnNlY3Rpb24uMi4zKV0KPj4gZW5k b2JqCjE0MSAwIG9iaiA8PAovTmFtZXMgWyhzdWJzZWN0aW9uLjIuNCkgMjIgMCBSIChzdWJzZWN0 aW9uLjIuNSkgMjYgMCBSXQovTGltaXRzIFsoc3Vic2VjdGlvbi4yLjQpIChzdWJzZWN0aW9uLjIu NSldCj4+IGVuZG9iagoxNDIgMCBvYmogPDwKL0tpZHMgWzEzNCAwIFIgMTM1IDAgUiAxMzYgMCBS IDEzNyAwIFIgMTM4IDAgUiAxMzkgMCBSXQovTGltaXRzIFsoRG9jLVN0YXJ0KSAoc2VjdGlvbi4y KV0KPj4gZW5kb2JqCjE0MyAwIG9iaiA8PAovS2lkcyBbMTQwIDAgUiAxNDEgMCBSXQovTGltaXRz IFsoc2VjdGlvbi4zKSAoc3Vic2VjdGlvbi4yLjUpXQo+PiBlbmRvYmoKMTQ0IDAgb2JqIDw8Ci9L aWRzIFsxNDIgMCBSIDE0MyAwIFJdCi9MaW1pdHMgWyhEb2MtU3RhcnQpIChzdWJzZWN0aW9uLjIu NSldCj4+IGVuZG9iagoxNDUgMCBvYmogPDwKL0Rlc3RzIDE0NCAwIFIKPj4gZW5kb2JqCjE0NiAw IG9iaiA8PAovVHlwZSAvQ2F0YWxvZwovUGFnZXMgMTMyIDAgUgovT3V0bGluZXMgMTMzIDAgUgov TmFtZXMgMTQ1IDAgUgovUGFnZU1vZGUvVXNlT3V0bGluZXMvUGFnZUxhYmVsczw8L051bXNbMDw8 L1MvRD4+MTw8L1MvRD4+XT4+Ci9PcGVuQWN0aW9uIDQxIDAgUgo+PiBlbmRvYmoKMTQ3IDAgb2Jq IDw8Ci9BdXRob3IoKS9UaXRsZSgpL1N1YmplY3QoKS9DcmVhdG9yKExhVGVYIHdpdGggaHlwZXJy ZWYgcGFja2FnZSkvUHJvZHVjZXIocGRmVGVYLTEuNDAuMTIpL0tleXdvcmRzKCkKL0NyZWF0aW9u RGF0ZSAoRDoyMDE1MDMyNzIxNDk1NiswOCcwMCcpCi9Nb2REYXRlIChEOjIwMTUwMzI3MjE0OTU2 KzA4JzAwJykKL1RyYXBwZWQgL0ZhbHNlCi9QVEVYLkZ1bGxiYW5uZXIgKFRoaXMgaXMgTWlLVGVY LXBkZlRlWCAyLjkuNDMwNyAoMS40MC4xMikpCj4+IGVuZG9iagp4cmVmCjAgMTQ4CjAwMDAwMDAw MDAgNjU1MzUgZiAKMDAwMDAwMDAxNSAwMDAwMCBuIAowMDAwMDAzNTQ1IDAwMDAwIG4gCjAwMDAx MDk0MTggMDAwMDAgbiAKMDAwMDAwMDA2MCAwMDAwMCBuIAowMDAwMDAwMDkwIDAwMDAwIG4gCjAw MDAwMDM2MDMgMDAwMDAgbiAKMDAwMDEwOTI5NyAwMDAwMCBuIAowMDAwMDAwMTM1IDAwMDAwIG4g CjAwMDAwMDAxNjYgMDAwMDAgbiAKMDAwMDAwMzY2MSAwMDAwMCBuIAowMDAwMTA5MjI1IDAwMDAw IG4gCjAwMDAwMDAyMTYgMDAwMDAgbiAKMDAwMDAwMDI1MCAwMDAwMCBuIAowMDAwMDAzODk3IDAw MDAwIG4gCjAwMDAxMDkxMzkgMDAwMDAgbiAKMDAwMDAwMDMwMSAwMDAwMCBuIAowMDAwMDAwMzM2 IDAwMDAwIG4gCjAwMDAwMDU4MjcgMDAwMDAgbiAKMDAwMDEwOTA1MyAwMDAwMCBuIAowMDAwMDAw Mzg3IDAwMDAwIG4gCjAwMDAwMDA0MTggMDAwMDAgbiAKMDAwMDAwNjEyMiAwMDAwMCBuIAowMDAw MTA4OTY3IDAwMDAwIG4gCjAwMDAwMDA0NjkgMDAwMDAgbiAKMDAwMDAwMDQ5OCAwMDAwMCBuIAow MDAwMDA2MTgxIDAwMDAwIG4gCjAwMDAxMDg4OTQgMDAwMDAgbiAKMDAwMDAwMDU0OSAwMDAwMCBu IAowMDAwMDAwNjAwIDAwMDAwIG4gCjAwMDAwMDg4MTggMDAwMDAgbiAKMDAwMDEwODgwNyAwMDAw MCBuIAowMDAwMDAwNjQ2IDAwMDAwIG4gCjAwMDAwMDA2ODcgMDAwMDAgbiAKMDAwMDAxMTcxNSAw MDAwMCBuIAowMDAwMTA4NzE5IDAwMDAwIG4gCjAwMDAwMDA3MzMgMDAwMDAgbiAKMDAwMDAwMDc2 NSAwMDAwMCBuIAowMDAwMDE1Mzc4IDAwMDAwIG4gCjAwMDAxMDg2NDQgMDAwMDAgbiAKMDAwMDAw MDgxMSAwMDAwMCBuIAowMDAwMDAwODQ5IDAwMDAwIG4gCjAwMDAwMDE0MjAgMDAwMDAgbiAKMDAw MDAwMTY1MyAwMDAwMCBuIAowMDAwMDAwODk3IDAwMDAwIG4gCjAwMDAwMDE1MzUgMDAwMDAgbiAK MDAwMDAwMTU5NCAwMDAwMCBuIAowMDAwMTA3NDYxIDAwMDAwIG4gCjAwMDAxMDczMTggMDAwMDAg biAKMDAwMDEwNzg4NyAwMDAwMCBuIAowMDAwMTA3NjA1IDAwMDAwIG4gCjAwMDAxMDgzMTUgMDAw MDAgbiAKMDAwMDAwNDAxNSAwMDAwMCBuIAowMDAwMDAzNDMwIDAwMDAwIG4gCjAwMDAwMDE3NTkg MDAwMDAgbiAKMDAwMDAwMzcyMCAwMDAwMCBuIAowMDAwMDAzNzc5IDAwMDAwIG4gCjAwMDAwMDM4 MzggMDAwMDAgbiAKMDAwMDAwMzk1NiAwMDAwMCBuIAowMDAwMDA2Mjk5IDAwMDAwIG4gCjAwMDAw MDU1OTQgMDAwMDAgbiAKMDAwMDAwNDA5NyAwMDAwMCBuIAowMDAwMDA1NzA5IDAwMDAwIG4gCjAw MDAwMDU3NjggMDAwMDAgbiAKMDAwMDAwNTg4NiAwMDAwMCBuIAowMDAwMDA1OTQ1IDAwMDAwIG4g CjAwMDAwMDYwMDQgMDAwMDAgbiAKMDAwMDAwNjA2MyAwMDAwMCBuIAowMDAwMDA2MjQwIDAwMDAw IG4gCjAwMDAwMDkxNzIgMDAwMDAgbiAKMDAwMDAwODUyNiAwMDAwMCBuIAowMDAwMDA2MzgxIDAw MDAwIG4gCjAwMDAwMDg2NDEgMDAwMDAgbiAKMDAwMDAwODcwMCAwMDAwMCBuIAowMDAwMDA4NzU5 IDAwMDAwIG4gCjAwMDAwMDg4NzcgMDAwMDAgbiAKMDAwMDAwODkzNiAwMDAwMCBuIAowMDAwMTA3 NzQ3IDAwMDAwIG4gCjAwMDAxMDgwMzAgMDAwMDAgbiAKMDAwMDAwODk5NSAwMDAwMCBuIAowMDAw MDA5MDU0IDAwMDAwIG4gCjAwMDAxMDgxNzMgMDAwMDAgbiAKMDAwMDAwOTExMyAwMDAwMCBuIAow MDAwMDExOTUwIDAwMDAwIG4gCjAwMDAwMTE0MjMgMDAwMDAgbiAKMDAwMDAwOTI5MCAwMDAwMCBu IAowMDAwMDExNTM4IDAwMDAwIG4gCjAwMDAwMTE1OTcgMDAwMDAgbiAKMDAwMDAxMTY1NiAwMDAw MCBuIAowMDAwMDExNzc0IDAwMDAwIG4gCjAwMDAwMTE4MzMgMDAwMDAgbiAKMDAwMDAxMTg5MSAw MDAwMCBuIAowMDAwMDEzOTY2IDAwMDAwIG4gCjAwMDAwMTQ0MDIgMDAwMDAgbiAKMDAwMDAxNDgz MCAwMDAwMCBuIAowMDAwMDE1NDM3IDAwMDAwIG4gCjAwMDAwMTM4MDIgMDAwMDAgbiAKMDAwMDAx MjA1NiAwMDAwMCBuIAowMDAwMDE1MDIwIDAwMDAwIG4gCjAwMDAwMTUwNzkgMDAwMDAgbiAKMDAw MDAxNTEzOCAwMDAwMCBuIAowMDAwMDE1MTk4IDAwMDAwIG4gCjAwMDAwMTUyNTggMDAwMDAgbiAK MDAwMDAxNTMxOCAwMDAwMCBuIAowMDAwMDE0MTgzIDAwMDAwIG4gCjAwMDAwMTQ2MTUgMDAwMDAg biAKMDAwMDAxNjYyOCAwMDAwMCBuIAowMDAwMDE2NDQ4IDAwMDAwIG4gCjAwMDAwMTU1NDMgMDAw MDAgbiAKMDAwMDAxNjU2NyAwMDAwMCBuIAowMDAwMTA4NDI1IDAwMDAwIG4gCjAwMDAwMTY3MjMg MDAwMDAgbiAKMDAwMDAxNjk5NCAwMDAwMCBuIAowMDAwMDE3MzQ1IDAwMDAwIG4gCjAwMDAwMTcz ODIgMDAwMDAgbiAKMDAwMDAxNzk5NCAwMDAwMCBuIAowMDAwMDE4MzYzIDAwMDAwIG4gCjAwMDAw MTkwMDAgMDAwMDAgbiAKMDAwMDAxOTQ1OCAwMDAwMCBuIAowMDAwMDMzOTQ2IDAwMDAwIG4gCjAw MDAwMzQyOTQgMDAwMDAgbiAKMDAwMDA0ODExNCAwMDAwMCBuIAowMDAwMDQ4NDA3IDAwMDAwIG4g CjAwMDAwNjg4ODIgMDAwMDAgbiAKMDAwMDA2OTM1OCAwMDAwMCBuIAowMDAwMDc2NzIwIDAwMDAw IG4gCjAwMDAwNzY5NTAgMDAwMDAgbiAKMDAwMDA4ODcxOSAwMDAwMCBuIAowMDAwMDg5MDIyIDAw MDAwIG4gCjAwMDAxMDA5MjIgMDAwMDAgbiAKMDAwMDEwMTIyOSAwMDAwMCBuIAowMDAwMTA3MDQ1 IDAwMDAwIG4gCjAwMDAxMDg1MDIgMDAwMDAgbiAKMDAwMDEwODU3MCAwMDAwMCBuIAowMDAwMTA5 NDg5IDAwMDAwIG4gCjAwMDAxMDk2NTYgMDAwMDAgbiAKMDAwMDEwOTgyMCAwMDAwMCBuIAowMDAw MTA5OTg0IDAwMDAwIG4gCjAwMDAxMTAxNDAgMDAwMDAgbiAKMDAwMDExMDMzNCAwMDAwMCBuIAow MDAwMTEwNDk4IDAwMDAwIG4gCjAwMDAxMTA2OTggMDAwMDAgbiAKMDAwMDExMDgyMiAwMDAwMCBu IAowMDAwMTEwOTM1IDAwMDAwIG4gCjAwMDAxMTEwMjEgMDAwMDAgbiAKMDAwMDExMTEwNyAwMDAw MCBuIAowMDAwMTExMTQ1IDAwMDAwIG4gCjAwMDAxMTEzMTIgMDAwMDAgbiAKdHJhaWxlcgo8PCAv U2l6ZSAxNDgKL1Jvb3QgMTQ2IDAgUgovSW5mbyAxNDcgMCBSCi9JRCBbPDkwQjA2NTY5N0YxMjJD RkE3QTVEMjc2RjA3NDk4RjE4PiA8OTBCMDY1Njk3RjEyMkNGQTdBNUQyNzZGMDc0OThGMTg+XSA+ PgpzdGFydHhyZWYKMTExNTg3CiUlRU9GCg== --001a1135f0aa79a4e505126d9ec7-- From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 15:03:18 2015 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B151779D for ; Sun, 29 Mar 2015 15:03:18 +0000 (UTC) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5C0D8DE9 for ; Sun, 29 Mar 2015 15:03:18 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id 98ECBB805E; Sun, 29 Mar 2015 17:03:16 +0200 (CEST) Received: by snail.stack.nl (Postfix, from userid 1677) id 88DB328494; Sun, 29 Mar 2015 17:03:16 +0200 (CEST) Date: Sun, 29 Mar 2015 17:03:16 +0200 From: Jilles Tjoelker To: Konstantin Belousov Subject: Re: kevent behavior Message-ID: <20150329150316.GB95224@stack.nl> References: <550A6DA2.1070004@gmail.com> <20150324221541.GA67584@stack.nl> <20150325090041.GY2379@kib.kiev.ua> <20150325223530.GA79065@stack.nl> <20150326214551.GG2379@kib.kiev.ua> <20150326225826.GA97319@stack.nl> <20150327132654.GJ2379@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150327132654.GJ2379@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@FreeBSD.org, Ivan Radovanovic X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 15:03:18 -0000 On Fri, Mar 27, 2015 at 03:26:54PM +0200, Konstantin Belousov wrote: > On Thu, Mar 26, 2015 at 11:58:27PM +0100, Jilles Tjoelker wrote: > > On Thu, Mar 26, 2015 at 11:45:51PM +0200, Konstantin Belousov wrote: > > > On Wed, Mar 25, 2015 at 11:35:30PM +0100, Jilles Tjoelker wrote: > > > > > > POSIX does not say anything about kevent(), including whether it should > > > > be a cancellation point or not. Given that kevent() may block for a long > > > > time, making it a cancellation point seems to make sense. > > > > > But POSIX language is very explicit for its own set of interfaces, and > > > that makes sense. This is why I think that cancellability, after the > > > 15+ years of kqueue existence, should be opt in. > > Hmm, I think most code written is cancel-unsafe. It is unlikely to have > > cancel-safe code using kevent() and relying on it not being a > > cancellation point, but still possible. > Ok, I re-considered my opinion after re-reading your responses. > > This also means we shouldn't wait too long with adding missing > > cancellation points like ppoll(). > > > > The kevent() cancellation point implementation would be much like most > > > > other cancellation points such as pselect(), with the difference that > > > > the call may have committed even if it fails, in the case that > > > > nchanges > nevents and in the case that nchanges > 0 && errno == EINTR. > > > > If cancellation is requested while blocking in the latter case, libthr > > > > will have to return -1/EINTR to indicate to the application that the > > > > changes were applied successfully while allowing the thread to be > > > > cancelled at the next cancellation point, even though there may not be > > > > any signal handler. > I do not quite understand this. > If any error (except things like EFAULT) occured while processing the > changelist, kevent(2) returns error count, and not the -1, regarless of > the length of eventlist. In this case, we do not cancel. kevent(2) has two ways of reporting an error while processing the changelist. If there is enough space in eventlist, it reports the error there and continues processing changelist; if there is no space, it reports the error via errno and stops processing changelist. > Syscall cannot return n/EINTR, where n >= 0. > If -1/EINTR is returned, this means that the call blocked and sleep > was interrupted. The changes from the changelist were applied, > do you suggested that we must not cancel if nchanges > 0 ? Yes, so that the application knows the kqueue's state when the thread is cancelled. > > > > If nevents == 0, kevent() does not block and need not be a cancellation > > > > point. This special case may simplify things slightly for application > > > > programmers. > > > > A kqueue() flag for cancellation does not seem very useful: since such a > > > > flag would be a kernel thing and pthread cancellation is a libthr thing, > > > > requesting the flag from the kernel would slow down all kevent() calls. > > > Yes, I thought about adding some (small) userspace table which would > > > track cancellable kqueues. > > I think the interaction with dup() and similar calls and with exec makes > > this too nasty. > Hm, yes. > > > But if we make the cancellation state per-call, and not a state > > > for kqueue descriptor, we might indicate the cancellability by > > > some event type, which is not passed to the kernel at all. The > > > libthr wrapper for kevent(2) would need to scan the changelist and > > > zero-out the cancellation indicator. > > This seems a bit hackish. However, enabling cancellation per-call seems > > better than per-kqueue. > Patch below always considers kevent(2) as cancellable, unless nevents == 0. > I will correct the call to thr_cancel_leave() if I misunderstood you. > [snip] > +.Sh CANCELLATION BEHAVIOUR > +If > +.Fa nevents > +is non-zero, i.e. the function is potentially blocking, the call > +is the cancellation point. "is a cancellation point." > +Otherwise, i.e. if > +.Fa nevents > +is zero, the call is not cancellable. > +Cancellation can only occur when the call was blocked and no changes > +to the queue were requested. This is not how the code works, but I think the code is correct. The code also allows cancellation before anything happens. As such, if cancellation occurs at kevent(2), the kqueue's state is unchanged (from that call). A new error should be added to the man page: .It Bq Er EINTR A cancellation request was delivered to the thread, but not yet handled. Both [EINTR] errors should have this sentence added to them: All changes in the .Fa changelist have been applied. -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 15:20:26 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7EBBDDC6 for ; Sun, 29 Mar 2015 15:20:26 +0000 (UTC) Received: from mail-ig0-x22e.google.com (mail-ig0-x22e.google.com [IPv6:2607:f8b0:4001:c05::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F12BF14 for ; Sun, 29 Mar 2015 15:20:26 +0000 (UTC) Received: by igbud6 with SMTP id ud6so54974973igb.1 for ; Sun, 29 Mar 2015 08:20:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=zq6KuNpEuI+iuzljCR0Ry9a2ywB2KKHWOxrx8O/U7Ak=; b=c0fASuUTRWpIXMEX+zffY1GrrLWG0xpiTKP5uK9UopWhGeEIirRSZ2gslu1Y7c+qa8 pVsbadMRXXIVAtC4Tl92Zx5m/AJgjnMQfnPzVWAur/QpKylN1/vMIxA9uJTpnK18aZU7 s0nEQC9kvCC9Xzh+YlaldP4xYmz7FqPwqHJbaJLyvdRjO9RZ2HIJoBJtk+ov78Z03LiF MsZ4VAFaBH4gVpqaP3tkutSA7RDQefG01p6itVsMeUVJ1AbBFwhcXTRI1XWxhXy5HliS C9w7Og78zWgoZ75to/lyKIoy/HoZL8kukxciyb4N53IfyZxnjevgRjXPAFeQ5/8DjbTl B80w== MIME-Version: 1.0 X-Received: by 10.107.39.72 with SMTP id n69mr27670147ion.8.1427642425766; Sun, 29 Mar 2015 08:20:25 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Sun, 29 Mar 2015 08:20:25 -0700 (PDT) In-Reply-To: <20150329081902.GN23643@zxy.spb.ru> References: <20150328221621.GG23643@zxy.spb.ru> <20150328224634.GH23643@zxy.spb.ru> <20150328230533.GI23643@zxy.spb.ru> <20150328234116.GJ23643@zxy.spb.ru> <20150329003354.GK23643@zxy.spb.ru> <20150329081902.GN23643@zxy.spb.ru> Date: Sun, 29 Mar 2015 08:20:25 -0700 X-Google-Sender-Auth: 6b7u5b9oxAaCXfoPqIaS2Ko1HM0 Message-ID: Subject: Re: irq cpu binding From: Adrian Chadd To: Slawa Olhovchenkov Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 15:20:26 -0000 On 29 March 2015 at 01:19, Slawa Olhovchenkov wrote: > On Sat, Mar 28, 2015 at 10:46:54PM -0700, Adrian Chadd wrote: > >> >> * It turns out that fragments were being 100% handled out of order >> >> (compared to non-fragments in the same stream) when doing fragment >> >> reassembly, because the current system was assuming direct dispatch >> >> netisr and not checking any packet contents for whether they're on the >> >> wrong CPU. I checked. It's not noticable unless you go digging, but >> >> it's absolutely happening. That's why I spun a lot of cycles looking >> >> at the IP fragment reassembly path and which methods get called on the >> >> frames as they're reinjected. >> > >> > In case of fragmented packet we have first fragment (may be arrived >> > not first) contained L4 information and dispatchet to correct bucket >> > and other fragments, don't contains this information and dispathed >> > anywere. As I understund IP stack gather all packet before processing. >> > All we need -- do processing on CPU arriving first segment. >> >> I'm pretty sure that wasn't what was happening when i went digging. I >> was using UDP and varying the transmit size so I had exact control >> over the fragmentation. >> >> The driver rx path does direct dispatch netisr processing, and for >> fragments it was hashed on only L3 details not L4. Even the first >> frame is hashed on L3 only. So it'd go to a different queue compared >> to L4 hashing, and subsequent fragments would come in on the same >> queue. Once it was completed, it was processed up inline - it wasn't >> going back into netisr and getting re-checked for the right queue. > > Two case: > 1) let this behavior > 2) rewrite fo resheduling. > > I think 1) acceptable -- fragmented packets very rarely, compared to > target data rate (2Mpps and more). > >> > What's problem there? >> > I am don't intersting how NIC do hashing (anyway, hashing for direct >> > and reflex traffic is different -- this is not Tilera). >> > All I need -- distributing flow to CPU, for balance load and reduction >> > lock congenstion. >> >> Right, but you assume all packets in a flow go to the same CPU, and I >> discovered this wasn't the case. >> That's why I went down the path with RSS to make it right. > > Only fragmented packets case or other case? > >> > >> >> * For applications - I'm not sure yet, but at the minimum the librss >> >> API I have vaguely sketched out and coded up in a git branch lets you >> >> pull out the list of buckets and which CPU it's on. I'm going to >> >> extend that a bit more, but it should be enough for things like nginx >> >> to say "ok, start up one nginx process per RSS bucket, and here's the >> >> CPU set for it to bind to." You said it has worker groups - that's >> >> great; I want that to be auto configured. >> > >> > For applications minimum is (per socket) select/kqueut/accept work >> > only for flow, arrived at CPU matched CPU at time select/kqueut/accept >> > (yes, for correct work application must pined to this CPU). >> > >> > And application don't need know anything about buckets and etc. >> > >> > After this, arrived packet activated IRQ handler, ithread, driver >> > interrup thread, TCP stack, select/accept, read, write, tcp_output -- >> > all on same cpu. I can be wrong, this is save L2/L3 cache. >> > >> > Where I missunderstund? >> >> The other half of the network stack - the sending side - also needs to >> be either on the same or nearby CPU, or you still end up with lock >> contention and cache thrashing. > > For incoming connections this will be automatuc -- sending will be > from CPU binding to receiving queue. > > Outgoing connections is more complex case, yes. > Need to transfer FD (with re-binding) and signaling (from kernel to > application) about prefered CPU. Prefered CPU is CPU give SYN-ACK. > And this need assistance from application. But I am currently can't > remember application massive servering outgouing connections. Or you realise you need to rewrite your userland application so it doesn't have to do this, and instead uses an IOCP/libdispatch style IO API to register for IO events and get IO completions to occur in any given completion thread. Then it doesn't have to care about moving descriptors around - it just creates an outbound socket, and then the IO completion callbacks will happen wherever they need to happen. If that needs to shuffle around due to RSS rebalancing then it'll "just happen". And yeah, I know of plenty of applications doing massive outbound connections - anything being an intermediary HTTP proxy. :) -adrian From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 15:59:04 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 31F987F6; Sun, 29 Mar 2015 15:59:04 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D87772E9; Sun, 29 Mar 2015 15:59:03 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1YcFbz-000J0E-5C; Sun, 29 Mar 2015 18:58:55 +0300 Date: Sun, 29 Mar 2015 18:58:55 +0300 From: Slawa Olhovchenkov To: Adrian Chadd Subject: Re: irq cpu binding Message-ID: <20150329155855.GO23643@zxy.spb.ru> References: <20150328224634.GH23643@zxy.spb.ru> <20150328230533.GI23643@zxy.spb.ru> <20150328234116.GJ23643@zxy.spb.ru> <20150329003354.GK23643@zxy.spb.ru> <20150329081902.GN23643@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 15:59:04 -0000 On Sun, Mar 29, 2015 at 08:20:25AM -0700, Adrian Chadd wrote: > >> The other half of the network stack - the sending side - also needs to > >> be either on the same or nearby CPU, or you still end up with lock > >> contention and cache thrashing. > > > > For incoming connections this will be automatuc -- sending will be > > from CPU binding to receiving queue. > > > > Outgoing connections is more complex case, yes. > > Need to transfer FD (with re-binding) and signaling (from kernel to > > application) about prefered CPU. Prefered CPU is CPU give SYN-ACK. > > And this need assistance from application. But I am currently can't > > remember application massive servering outgouing connections. > > Or you realise you need to rewrite your userland application so it > doesn't have to do this, and instead uses an IOCP/libdispatch style IO > API to register for IO events and get IO completions to occur in any > given completion thread. nginx is multi-process application, not multi-thread, for example. > Then it doesn't have to care about moving descriptors around - it just > creates an outbound socket, and then the IO completion callbacks will > happen wherever they need to happen. If that needs to shuffle around > due to RSS rebalancing then it'll "just happen". > > And yeah, I know of plenty of applications doing massive outbound > connections - anything being an intermediary HTTP proxy. :) Hmm, yes and no :) Yes, proxy do outbound connections, but proxy crossover inbound and outbound connections and in general this connections pined to different CPU. Is this perfomance gain?.. From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 16:03:45 2015 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5B7DFBB1 for ; Sun, 29 Mar 2015 16:03:45 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D8A4D3BC for ; Sun, 29 Mar 2015 16:03:44 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t2TG3dC7099724 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 29 Mar 2015 19:03:39 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t2TG3dC7099724 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t2TG3cUo099723; Sun, 29 Mar 2015 19:03:38 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 29 Mar 2015 19:03:38 +0300 From: Konstantin Belousov To: Jilles Tjoelker Subject: Re: kevent behavior Message-ID: <20150329160338.GZ2379@kib.kiev.ua> References: <550A6DA2.1070004@gmail.com> <20150324221541.GA67584@stack.nl> <20150325090041.GY2379@kib.kiev.ua> <20150325223530.GA79065@stack.nl> <20150326214551.GG2379@kib.kiev.ua> <20150326225826.GA97319@stack.nl> <20150327132654.GJ2379@kib.kiev.ua> <20150329150316.GB95224@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150329150316.GB95224@stack.nl> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-hackers@FreeBSD.org, Ivan Radovanovic X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 16:03:45 -0000 On Sun, Mar 29, 2015 at 05:03:16PM +0200, Jilles Tjoelker wrote: > On Fri, Mar 27, 2015 at 03:26:54PM +0200, Konstantin Belousov wrote: > > +Otherwise, i.e. if > > +.Fa nevents > > +is zero, the call is not cancellable. > > > +Cancellation can only occur when the call was blocked and no changes > > +to the queue were requested. > > This is not how the code works, but I think the code is correct. The > code also allows cancellation before anything happens. As such, if > cancellation occurs at kevent(2), the kqueue's state is unchanged (from > that call). Below is only a new diff for the man pages changes. I found our list of the cancellation points in the pthread_testcancel(3), but did not checked it for correctness. > > A new error should be added to the man page: > > .It Bq Er EINTR > A cancellation request was delivered to the thread, but not yet handled. > > Both [EINTR] errors should have this sentence added to them: > > All changes in the > .Fa changelist > have been applied. > I formulated this after the error list. diff --git a/lib/libc/sys/kqueue.2 b/lib/libc/sys/kqueue.2 index 93223f1..c5e8caf 100644 --- a/lib/libc/sys/kqueue.2 +++ b/lib/libc/sys/kqueue.2 @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd July 18, 2014 +.Dd March 29, 2015 .Dt KQUEUE 2 .Os .Sh NAME @@ -41,7 +41,7 @@ .Fn kqueue "void" .Ft int .Fn kevent "int kq" "const struct kevent *changelist" "int nchanges" "struct kevent *eventlist" "int nevents" "const struct timespec *timeout" -.Fn EV_SET "&kev" ident filter flags fflags data udata +.Fn EV_SET "kev" ident filter flags fflags data udata .Sh DESCRIPTION The .Fn kqueue @@ -550,6 +550,16 @@ On return, .Va fflags contains the users defined flags in the lower 24 bits. .El +.Sh CANCELLATION BEHAVIOUR +If +.Fa nevents +is non-zero, i.e. the function is potentially blocking, the call +is a cancellation point. +Otherwise, i.e. if +.Fa nevents +is zero, the call is not cancellable. +Cancellation can only occur before any changes are made to the kqueue, +or when the call was blocked and no changes to the queue were requested. .Sh RETURN VALUES The .Fn kqueue @@ -620,6 +630,8 @@ The specified descriptor is invalid. .It Bq Er EINTR A signal was delivered before the timeout expired and before any events were placed on the kqueue for return. +.It Bq Er EINTR +A cancellation request was delivered to the thread, but not yet handled. .It Bq Er EINVAL The specified time limit or filter is invalid. .It Bq Er ENOENT @@ -634,6 +646,14 @@ sysctl. .It Bq Er ESRCH The specified process to attach to does not exist. .El +.Pp +When +.Fn kevent +call fails with +.Er EINTR +error, all changes in the +.Fa changelist +have been applied. .Sh SEE ALSO .Xr aio_error 2 , .Xr aio_read 2 , @@ -643,6 +663,7 @@ The specified process to attach to does not exist. .Xr select 2 , .Xr sigaction 2 , .Xr write 2 , +.Xr pthread_setcancelstate 3 , .Xr signal 3 .Sh HISTORY The diff --git a/share/man/man3/pthread_testcancel.3 b/share/man/man3/pthread_testcancel.3 index ef6a1c1..4390c92 100644 --- a/share/man/man3/pthread_testcancel.3 +++ b/share/man/man3/pthread_testcancel.3 @@ -1,5 +1,5 @@ .\" $FreeBSD$ -.Dd June 11, 2013 +.Dd March 29, 2015 .Dt PTHREAD_TESTCANCEL 3 .Os .Sh NAME @@ -107,6 +107,7 @@ functions: .Fn close , .Fn creat , .Fn fsync , +.Fn kevent , .Fn mq_receive , .Fn mq_send , .Fn mq_timedreceive , From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 17:01:07 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8174BD7E for ; Sun, 29 Mar 2015 17:01:07 +0000 (UTC) Received: from mail-ie0-x236.google.com (mail-ie0-x236.google.com [IPv6:2607:f8b0:4001:c03::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 419E3B60 for ; Sun, 29 Mar 2015 17:01:07 +0000 (UTC) Received: by ierf6 with SMTP id f6so34333462ier.2 for ; Sun, 29 Mar 2015 10:01:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=z/PTopEW0VdERlcF+jHp4/Hw9oC3jRMc/s5MG9waOnw=; b=s/ONQtRIITdmwiSQEODUaXbBT14ip+MpfPZ8nALy4DBz/3h+CsHhfBTsczx7vpYAgl zIFL0KFue8fKbZ383LSBSUCIhfWim4WJGRAuyMNAjYv3uwaG9MQX1OvWaREImrxNTFDR 3nofot6svEEP7cAkel4NmNlxxKMAb4Wh9byeBs46Xx/1yOl0AsBqVQlF9UxeoS3m6b1L kzdNAzv3iE8PxyUesZXdtWkNi1I0pp/ksFJnOED6LBAcPcmbmybUpnpv6GMQmVBmu2qo Fvm1XjL5xNibksjj58jT3Z1pgYJJtoPG45dMdmigahLZ5WBVvnkgoBfm5kQYpIAZ6KXv Jg1g== MIME-Version: 1.0 X-Received: by 10.50.143.42 with SMTP id sb10mr12411777igb.49.1427648466657; Sun, 29 Mar 2015 10:01:06 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Sun, 29 Mar 2015 10:01:06 -0700 (PDT) In-Reply-To: <20150329155855.GO23643@zxy.spb.ru> References: <20150328224634.GH23643@zxy.spb.ru> <20150328230533.GI23643@zxy.spb.ru> <20150328234116.GJ23643@zxy.spb.ru> <20150329003354.GK23643@zxy.spb.ru> <20150329081902.GN23643@zxy.spb.ru> <20150329155855.GO23643@zxy.spb.ru> Date: Sun, 29 Mar 2015 10:01:06 -0700 X-Google-Sender-Auth: YPs05l5GVU-OxkM3AInIprUMu1s Message-ID: Subject: Re: irq cpu binding From: Adrian Chadd To: Slawa Olhovchenkov Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 17:01:07 -0000 Well, have you seen how fast the windows network apps can get? Not the gui torrent clients, but the really serious HTTP intermediary stuff and message passing stuff? The model works and scales quite well.. :0 -a From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 17:21:14 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B636E335; Sun, 29 Mar 2015 17:21:14 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 68A5EDC4; Sun, 29 Mar 2015 17:21:14 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1YcGtb-000OAk-FX; Sun, 29 Mar 2015 20:21:11 +0300 Date: Sun, 29 Mar 2015 20:21:11 +0300 From: Slawa Olhovchenkov To: Adrian Chadd Subject: Re: irq cpu binding Message-ID: <20150329172111.GP23643@zxy.spb.ru> References: <20150328230533.GI23643@zxy.spb.ru> <20150328234116.GJ23643@zxy.spb.ru> <20150329003354.GK23643@zxy.spb.ru> <20150329081902.GN23643@zxy.spb.ru> <20150329155855.GO23643@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 17:21:14 -0000 On Sun, Mar 29, 2015 at 10:01:06AM -0700, Adrian Chadd wrote: > Well, have you seen how fast the windows network apps can get? Not the > gui torrent clients, but the really serious HTTP intermediary stuff > and message passing stuff? > > The model works and scales quite well.. :0 "how fast"... What is "fast"? How we calculate speed? In what enviroment (dual or single socket)? nginx is acceptable fast. FreeBSD is acceptable fast. But I am need less overhead [for multi-socket system]. I am may be wrong. I am use dual-socket systems. And when I am think about pinning to CPU I am think about elimination transfer context by QPI link. May be for single-socket systems this is will be less impact. May be double of interrupt/network/TCP processing thread gain lock congestion and eliminate all win. You have other opinion? About NUMA. I have expirense with Tilera. Perfomance with NUMA off (interleave memory) will be better. From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 17:22:09 2015 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 452CA424 for ; Sun, 29 Mar 2015 17:22:09 +0000 (UTC) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 05E2FDD1 for ; Sun, 29 Mar 2015 17:22:09 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id 6241BB8058; Sun, 29 Mar 2015 19:22:07 +0200 (CEST) Received: by snail.stack.nl (Postfix, from userid 1677) id 4CEFC28494; Sun, 29 Mar 2015 19:22:07 +0200 (CEST) Date: Sun, 29 Mar 2015 19:22:07 +0200 From: Jilles Tjoelker To: Konstantin Belousov Subject: Re: kevent behavior Message-ID: <20150329172207.GC95224@stack.nl> References: <550A6DA2.1070004@gmail.com> <20150324221541.GA67584@stack.nl> <20150325090041.GY2379@kib.kiev.ua> <20150325223530.GA79065@stack.nl> <20150326214551.GG2379@kib.kiev.ua> <20150326225826.GA97319@stack.nl> <20150327132654.GJ2379@kib.kiev.ua> <20150329150316.GB95224@stack.nl> <20150329160338.GZ2379@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150329160338.GZ2379@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@FreeBSD.org, Ivan Radovanovic X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 17:22:09 -0000 On Sun, Mar 29, 2015 at 07:03:38PM +0300, Konstantin Belousov wrote: > On Sun, Mar 29, 2015 at 05:03:16PM +0200, Jilles Tjoelker wrote: > > On Fri, Mar 27, 2015 at 03:26:54PM +0200, Konstantin Belousov wrote: > > > +Otherwise, i.e. if > > > +.Fa nevents > > > +is zero, the call is not cancellable. > > > > > +Cancellation can only occur when the call was blocked and no changes > > > +to the queue were requested. > > > > This is not how the code works, but I think the code is correct. The > > code also allows cancellation before anything happens. As such, if > > cancellation occurs at kevent(2), the kqueue's state is unchanged (from > > that call). > Below is only a new diff for the man pages changes. I found our list > of the cancellation points in the pthread_testcancel(3), but did not > checked it for correctness. Given that the special case for fcntl() is mentioned in pthread_testcancel(3) (only a cancellation point if cmd is F_SETLKW), the same should be done for kevent(). Looks good to me otherwise. -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 17:53:30 2015 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 20272E10 for ; Sun, 29 Mar 2015 17:53:30 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9B9AFE4 for ; Sun, 29 Mar 2015 17:53:29 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t2THrKsH025066 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 29 Mar 2015 20:53:20 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t2THrKsH025066 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t2THrKBf025065; Sun, 29 Mar 2015 20:53:20 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 29 Mar 2015 20:53:20 +0300 From: Konstantin Belousov To: Jilles Tjoelker Subject: Re: kevent behavior Message-ID: <20150329175320.GA2379@kib.kiev.ua> References: <550A6DA2.1070004@gmail.com> <20150324221541.GA67584@stack.nl> <20150325090041.GY2379@kib.kiev.ua> <20150325223530.GA79065@stack.nl> <20150326214551.GG2379@kib.kiev.ua> <20150326225826.GA97319@stack.nl> <20150327132654.GJ2379@kib.kiev.ua> <20150329150316.GB95224@stack.nl> <20150329160338.GZ2379@kib.kiev.ua> <20150329172207.GC95224@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150329172207.GC95224@stack.nl> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-hackers@FreeBSD.org, Ivan Radovanovic X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 17:53:30 -0000 On Sun, Mar 29, 2015 at 07:22:07PM +0200, Jilles Tjoelker wrote: > Given that the special case for fcntl() is mentioned in > pthread_testcancel(3) (only a cancellation point if cmd is F_SETLKW), > the same should be done for kevent(). > > Looks good to me otherwise. Below is the update. I will commit after some tests finish. diff --git a/lib/libc/sys/kqueue.2 b/lib/libc/sys/kqueue.2 index 93223f1..c5e8caf 100644 --- a/lib/libc/sys/kqueue.2 +++ b/lib/libc/sys/kqueue.2 @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd July 18, 2014 +.Dd March 29, 2015 .Dt KQUEUE 2 .Os .Sh NAME @@ -41,7 +41,7 @@ .Fn kqueue "void" .Ft int .Fn kevent "int kq" "const struct kevent *changelist" "int nchanges" "struct kevent *eventlist" "int nevents" "const struct timespec *timeout" -.Fn EV_SET "&kev" ident filter flags fflags data udata +.Fn EV_SET "kev" ident filter flags fflags data udata .Sh DESCRIPTION The .Fn kqueue @@ -550,6 +550,16 @@ On return, .Va fflags contains the users defined flags in the lower 24 bits. .El +.Sh CANCELLATION BEHAVIOUR +If +.Fa nevents +is non-zero, i.e. the function is potentially blocking, the call +is a cancellation point. +Otherwise, i.e. if +.Fa nevents +is zero, the call is not cancellable. +Cancellation can only occur before any changes are made to the kqueue, +or when the call was blocked and no changes to the queue were requested. .Sh RETURN VALUES The .Fn kqueue @@ -620,6 +630,8 @@ The specified descriptor is invalid. .It Bq Er EINTR A signal was delivered before the timeout expired and before any events were placed on the kqueue for return. +.It Bq Er EINTR +A cancellation request was delivered to the thread, but not yet handled. .It Bq Er EINVAL The specified time limit or filter is invalid. .It Bq Er ENOENT @@ -634,6 +646,14 @@ sysctl. .It Bq Er ESRCH The specified process to attach to does not exist. .El +.Pp +When +.Fn kevent +call fails with +.Er EINTR +error, all changes in the +.Fa changelist +have been applied. .Sh SEE ALSO .Xr aio_error 2 , .Xr aio_read 2 , @@ -643,6 +663,7 @@ The specified process to attach to does not exist. .Xr select 2 , .Xr sigaction 2 , .Xr write 2 , +.Xr pthread_setcancelstate 3 , .Xr signal 3 .Sh HISTORY The diff --git a/share/man/man3/pthread_testcancel.3 b/share/man/man3/pthread_testcancel.3 index ef6a1c1..a6b27f6 100644 --- a/share/man/man3/pthread_testcancel.3 +++ b/share/man/man3/pthread_testcancel.3 @@ -1,5 +1,5 @@ .\" $FreeBSD$ -.Dd June 11, 2013 +.Dd March 29, 2015 .Dt PTHREAD_TESTCANCEL 3 .Os .Sh NAME @@ -107,6 +107,7 @@ functions: .Fn close , .Fn creat , .Fn fsync , +.Fn kevent , .Fn mq_receive , .Fn mq_send , .Fn mq_timedreceive , @@ -147,12 +148,20 @@ functions: .Fn waitpid , .Fn write , .Fn writev . +.Pp The .Fn fcntl function is a cancellation point if .Fa cmd is .Dv F_SETLKW . +.Pp +The +.Fn kevent +function is a cancellation point if it is potentially blocking, +i.e. when the +.Fa nevents +argument is non-zero. .Sh RETURN VALUES If successful, the .Fn pthread_setcancelstate From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 17:56:19 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4AC09F2E; Sun, 29 Mar 2015 17:56:19 +0000 (UTC) Received: from mail-wg0-x236.google.com (mail-wg0-x236.google.com [IPv6:2a00:1450:400c:c00::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CD20C10F; Sun, 29 Mar 2015 17:56:18 +0000 (UTC) Received: by wgra20 with SMTP id a20so147569058wgr.3; Sun, 29 Mar 2015 10:56:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=HOMhBqq746u+seZ4CauAkrPkuHenD+mPnGI6jhQhOUk=; b=W4QqRD3jAheb6W6Y+h1aeUuN8JPvxfVObzsb8MTGPOmtVrZwFOAn/dBvdgKLXIHFiI H8471m5B3n8mP6PRBL1UZNw4O7k/jkdZJQ7hAtfd2qAStX5VNCsM2EObswqtbXD1o7Qm 7XlCIYDAI6h6X/0Y1S0HfwSmOB/y0lPPDVWzRf2oUcfIl5tAk6Fmyzz18K2wfV0FqASY Y9CAE8944htAGLqCaW4tmfLwsVLYKc4W1w+rMUqNVqy+gXHaMCleKZ+0qY+ewCmmIZBE EFoezgqEVyNI2o+IEOq8Rm/FeRELEdenqLwhplpyIYQmQd9zBZ7f40FVUYcHcWS1NI+O VfHQ== MIME-Version: 1.0 X-Received: by 10.180.9.171 with SMTP id a11mr15369485wib.24.1427651777246; Sun, 29 Mar 2015 10:56:17 -0700 (PDT) Received: by 10.27.20.6 with HTTP; Sun, 29 Mar 2015 10:56:17 -0700 (PDT) In-Reply-To: <20150329155855.GO23643@zxy.spb.ru> References: <20150328224634.GH23643@zxy.spb.ru> <20150328230533.GI23643@zxy.spb.ru> <20150328234116.GJ23643@zxy.spb.ru> <20150329003354.GK23643@zxy.spb.ru> <20150329081902.GN23643@zxy.spb.ru> <20150329155855.GO23643@zxy.spb.ru> Date: Sun, 29 Mar 2015 20:56:17 +0300 Message-ID: Subject: Re: irq cpu binding From: Sergey Kandaurov To: Slawa Olhovchenkov Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" , Adrian Chadd X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 17:56:19 -0000 On 29 March 2015 at 18:58, Slawa Olhovchenkov wrote: > > nginx is multi-process application, not multi-thread, for example. > In modern versions, threads may be optionally used to offload the potentially blocking operations, without blocking a worker process. -- wbr, pluknet From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 29 18:03:39 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E071816C; Sun, 29 Mar 2015 18:03:39 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 91319209; Sun, 29 Mar 2015 18:03:39 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1YcHYf-000Q0b-H5; Sun, 29 Mar 2015 21:03:37 +0300 Date: Sun, 29 Mar 2015 21:03:37 +0300 From: Slawa Olhovchenkov To: Sergey Kandaurov Subject: Re: irq cpu binding Message-ID: <20150329180337.GQ23643@zxy.spb.ru> References: <20150328230533.GI23643@zxy.spb.ru> <20150328234116.GJ23643@zxy.spb.ru> <20150329003354.GK23643@zxy.spb.ru> <20150329081902.GN23643@zxy.spb.ru> <20150329155855.GO23643@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: "freebsd-hackers@freebsd.org" , Adrian Chadd X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2015 18:03:40 -0000 On Sun, Mar 29, 2015 at 08:56:17PM +0300, Sergey Kandaurov wrote: > On 29 March 2015 at 18:58, Slawa Olhovchenkov wrote: > > > > nginx is multi-process application, not multi-thread, for example. > > > > In modern versions, threads may be optionally used to offload the > potentially blocking operations, without blocking a worker process. Only for AIO and "Multi-threaded sending of files is only supported on Linux.". This is hack because Linux lack good aio support. From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 30 00:45:53 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5286352A for ; Mon, 30 Mar 2015 00:45:53 +0000 (UTC) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::103]) by mx1.freebsd.org (Postfix) with ESMTP id 2B6E7221 for ; Mon, 30 Mar 2015 00:45:53 +0000 (UTC) Received: from [172.16.1.67] (unknown [172.16.1.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id 594F212A6 for ; Mon, 30 Mar 2015 00:45:46 +0000 (UTC) Message-ID: <55189CBA.9040107@metricspace.net> Date: Sun, 29 Mar 2015 20:45:46 -0400 From: Eric McCorkle User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: ZFS support for EFI Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2015 00:45:53 -0000 Hi folks, I've been messing around off and on for a while with adding ZFS support to the EFI boot. It's been mostly exploratory and self-contained up to this point, but I've gotten to a point that warrants some discussion. First, I've converted boot1.c (the EFI boot block) to use an FS module framework. This facilitates the addition of ZFS, and should also come in handy if someone wants to add other functionality later (ie. crypto, netboot, etc.) More importantly, the EFI loader doesn't seem to make use of its command-line arguments at all. But a ZFS-enabled loader would really need the ability to take arguments from boot1 (or grub, or whatever else). On the boot1 side, with ZFS you need to load and parse /boot/loader.conf (which may cause you to switch pools), then hand off the information to loader. In the BIOS loader, that's done through a binary data object that gets passed in. Command-line strings seem like the most sensible way to do it with EFI. Would this be the right way to go, and if so, what ought these command-line strings look like? Thanks, Eric From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 30 04:28:50 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 08A9FB0A for ; Mon, 30 Mar 2015 04:28:50 +0000 (UTC) Received: from st11p02mm-asmtp002.mac.com (st11p02mm-asmtp002.mac.com [17.172.220.237]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D085FC3D for ; Mon, 30 Mar 2015 04:28:49 +0000 (UTC) Received: from fukuyama.hsd1.ca.comcast.net (c-73-162-13-215.hsd1.ca.comcast.net [73.162.13.215]) by st11p02mm-asmtp002.mac.com (Oracle Communications Messaging Server 7.0.5.35.0 64bit (built Dec 4 2014)) with ESMTPSA id <0NM000FD3CFXI720@st11p02mm-asmtp002.mac.com> for freebsd-hackers@freebsd.org; Mon, 30 Mar 2015 04:28:47 +0000 (GMT) Content-type: text/plain; charset=us-ascii MIME-version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: ZFS support for EFI From: Rui Paulo In-reply-to: <55189CBA.9040107@metricspace.net> Date: Sun, 29 Mar 2015 21:28:45 -0700 Content-transfer-encoding: quoted-printable Message-id: <543637C0-A4FF-4801-BE5C-859F2D968D48@me.com> References: <55189CBA.9040107@metricspace.net> To: Eric McCorkle X-Mailer: Apple Mail (2.2070.6) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.13.68,1.0.33,0.0.0000 definitions=2015-03-30_01:2015-03-28,2015-03-29,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1412110000 definitions=main-1503300046 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2015 04:28:50 -0000 Hi, > On Mar 29, 2015, at 17:45, Eric McCorkle wrote: >=20 > Hi folks, >=20 > I've been messing around off and on for a while with adding ZFS = support > to the EFI boot. It's been mostly exploratory and self-contained up = to > this point, but I've gotten to a point that warrants some discussion. >=20 >=20 > First, I've converted boot1.c (the EFI boot block) to use an FS module > framework. This facilitates the addition of ZFS, and should also come > in handy if someone wants to add other functionality later (ie. = crypto, > netboot, etc.) Good. :-) > More importantly, the EFI loader doesn't seem to make use of its > command-line arguments at all. But a ZFS-enabled loader would really > need the ability to take arguments from boot1 (or grub, or whatever > else). On the boot1 side, with ZFS you need to load and parse > /boot/loader.conf (which may cause you to switch pools), then hand off > the information to loader. In the BIOS loader, that's done through a > binary data object that gets passed in. Command-line strings seem = like > the most sensible way to do it with EFI. >=20 > Would this be the right way to go, and if so, what ought these > command-line strings look like? I have a crazy idea: why not use getopt() in loader.efi ? getopt() is = already part of libstand, so it should be easy to use it. Alternatively you can just use key value pairs. -- Rui Paulo From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 30 09:33:57 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3C3DBD9A for ; Mon, 30 Mar 2015 09:33:57 +0000 (UTC) Received: from mail-wg0-x236.google.com (mail-wg0-x236.google.com [IPv6:2a00:1450:400c:c00::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CAF0D136 for ; Mon, 30 Mar 2015 09:33:56 +0000 (UTC) Received: by wgbgs4 with SMTP id gs4so75721153wgb.0 for ; Mon, 30 Mar 2015 02:33:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:mime-version:content-type :content-disposition:content-transfer-encoding:user-agent; bh=Pbhg6FjR6ROZpUommR3W+Ru2YU5BbLoQUwvs5lc+8mU=; b=JGOEGRTvhEogImukD71d08th05EyJmWjSliKIN3kgrbcTtjUmpLnL2rDGRnZswYzM6 o3ssiZOGlty6Y45k89BJefYEKakT/8JdiRcAIDn1FJjcQkTy0eHYLTwk0uJsH3UkLUin JSNl3lBuLWFrdluPsTnHOJzIaFgNbJdlrakWd0Uu9WFyt9Li8bJK10j29yiW4ydXx7b8 ia4Onl2uSsjQ8StmoyO1a/NIfGh2yGavXyoFsd7gh4LVpkGPghiAbH0HoAVtJ502XkYj JVv3SHYc/vXXS4FpFVzUWMKWnIS7CZapNyjFFgEjY4Nl1bi7WmEhsmvYlZlmKpbi70I1 DVGA== X-Received: by 10.180.91.70 with SMTP id cc6mr20196447wib.78.1427708035248; Mon, 30 Mar 2015 02:33:55 -0700 (PDT) Received: from valhala.home ([78.193.58.122]) by mx.google.com with ESMTPSA id ew13sm14884884wid.1.2015.03.30.02.33.54 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 30 Mar 2015 02:33:54 -0700 (PDT) Date: Mon, 30 Mar 2015 11:33:53 +0200 From: Nicolas Martyanoff To: freebsd-hackers@freebsd.org Subject: Intel HD 5500 driver port Message-ID: <20150330093352.GA3937@valhala.home> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2015 09:33:57 -0000 Hi, I have trouble finding info about support for the Intel HD 5500 graphic chip (used on broadwell platforms such as new Thinkpads) on FreeBSD. On Linux, it seems to be supported by the Intel driver Since June 2014¹. Is there a port in progress, or some development being done ? Feel free to redirect me to another mailing list if this is not the correct one :) Thank you in advance. [1] https://01.org/linuxgraphics/downloads/2014/2014q2-intel-graphics-stack-release -- Nicolas Martyanoff http://wandrian.net khaelin@gmail.com From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 30 09:39:47 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 95A89AD for ; Mon, 30 Mar 2015 09:39:47 +0000 (UTC) Received: from mail-ob0-x22a.google.com (mail-ob0-x22a.google.com [IPv6:2607:f8b0:4003:c01::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 57A9D19F for ; Mon, 30 Mar 2015 09:39:47 +0000 (UTC) Received: by obcjt1 with SMTP id jt1so116163245obc.2 for ; Mon, 30 Mar 2015 02:39:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=x83q19a8o78K8P00rHVkl6QCO/kMswdEobFLVvzwT/E=; b=SC3zW46rBINZtbyL0VpaOBf0gsb+CetDeL7gGuZgaKKpL5XOWAxt7lnPmiFMlUzpae MOydnYXLd12R7WgNoB4aDLY/w89ZslvLLbhxLhzMxmRe/DvqmWsSUnbFMQBPcmD0cKHc GCFA1tE9G0XI08j4/IEuVULjalPTwlQVtNOjg146JMX7T9XFeoOkdosmiqZbYU9UBq4F IjtOq6wBRiY1Xhqp48h2FhL1ow2+uL7stNoZAECevQmMyqRgX7ifla2S4IJrO2CYFQDz ejeRdSq99t+m/RLxHZR5rrCg3WpxaEgdfrW1Ah9s643ZwnAxOX+jnEPRLCPTuav+CjSc Ke+w== MIME-Version: 1.0 X-Received: by 10.60.63.39 with SMTP id d7mr25839798oes.72.1427708386554; Mon, 30 Mar 2015 02:39:46 -0700 (PDT) Received: by 10.202.48.195 with HTTP; Mon, 30 Mar 2015 02:39:46 -0700 (PDT) In-Reply-To: <20150330093352.GA3937@valhala.home> References: <20150330093352.GA3937@valhala.home> Date: Mon, 30 Mar 2015 11:39:46 +0200 Message-ID: Subject: Re: Intel HD 5500 driver port From: Johannes Dieterich To: Nicolas Martyanoff Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2015 09:39:47 -0000 On Mon, Mar 30, 2015 at 11:33 AM, Nicolas Martyanoff wr= ote: > Hi, > > I have trouble finding info about support for the Intel HD 5500 graphic c= hip > (used on broadwell platforms such as new Thinkpads) on FreeBSD. On Linux,= it > seems to be supported by the Intel driver Since June 2014=C2=B9. > > Is there a port in progress, or some development being done ? > > Feel free to redirect me to another mailing list if this is not the corre= ct > one :) > > Thank you in advance. Dear Nicolas, neither Haswell nor Broadwell iGPUs are currently supported by any version of FreeBSD with the intel driver. However, work is underway by some developers (kib@, dumbbell@) to make this happen. Please see https://wiki.freebsd.org/Graphics for up-to-date status/information of all things graphics. Depending on your requirements, you can use the vesa driver with software rendering in the mean time. HTH Johannes > > [1] https://01.org/linuxgraphics/downloads/2014/2014q2-intel-graphics-sta= ck-release > > -- > Nicolas Martyanoff > http://wandrian.net > khaelin@gmail.com > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 30 09:46:14 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 750A4251 for ; Mon, 30 Mar 2015 09:46:14 +0000 (UTC) Received: from mail-wi0-x234.google.com (mail-wi0-x234.google.com [IPv6:2a00:1450:400c:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 051C2289 for ; Mon, 30 Mar 2015 09:46:14 +0000 (UTC) Received: by wiaa2 with SMTP id a2so120711043wia.0 for ; Mon, 30 Mar 2015 02:46:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=U5YpvxXh8wLfoSSHWRq8tA93TBxMOuFZ3zggPQaIALI=; b=isRuhsQJrinL5pgz2q5Xl2HIJ6XOsc3zCyy+/pEc/UxBBArGIH60azeOzHibTbJHDn 4ExV2nqJXr48HLfd4QhvMPnPwt3aPSuH7hGyYva65NXlovj3paMxsReUBs9LPmkxV+19 Mzn8l6XJW68AQg+ZKsQdshIwVLL2fXuWDBC3my3uElvMFS8jxzqy0Yj78y2lAPFv3sBo ss2uF8ZRWIpgmo+3y3xs+w1DP5mqsEAxrUTRlvoHR59bU8e6x8lHgni4W1YRT3p6uEQR Wx4p+f/VVowlW0zMQN60Th+DTgWXbVuPR+IvsY9eCeQ5oWI3aPorpceZKadIHSpx2ARV r2Eg== X-Received: by 10.194.81.1 with SMTP id v1mr59506410wjx.50.1427708772396; Mon, 30 Mar 2015 02:46:12 -0700 (PDT) Received: from valhala.home ([78.193.58.122]) by mx.google.com with ESMTPSA id cf12sm14950884wjb.10.2015.03.30.02.46.11 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 30 Mar 2015 02:46:11 -0700 (PDT) Date: Mon, 30 Mar 2015 11:46:10 +0200 From: Nicolas Martyanoff To: Johannes Dieterich Subject: Re: Intel HD 5500 driver port Message-ID: <20150330094610.GB3937@valhala.home> References: <20150330093352.GA3937@valhala.home> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2015 09:46:14 -0000 On 2015-03-30 11:39, Johannes Dieterich wrote: > On Mon, Mar 30, 2015 at 11:33 AM, Nicolas Martyanoff wrote: > > Hi, > > > > I have trouble finding info about support for the Intel HD 5500 graphic chip > > (used on broadwell platforms such as new Thinkpads) on FreeBSD. On Linux, it > > seems to be supported by the Intel driver Since June 2014¹. > > > > Is there a port in progress, or some development being done ? > neither Haswell nor Broadwell iGPUs are currently supported by any > version of FreeBSD with the intel driver. However, work is underway by > some developers (kib@, dumbbell@) to make this happen. Please see > https://wiki.freebsd.org/Graphics for up-to-date status/information of > all things graphics. > > Depending on your requirements, you can use the vesa driver with > software rendering in the mean time. Thank you for the information. I'll try software rendering when I get the laptop. -- Nicolas Martyanoff http://wandrian.net khaelin@gmail.com From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 30 13:11:23 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 80F8A788 for ; Mon, 30 Mar 2015 13:11:23 +0000 (UTC) Received: from mail.egr.msu.edu (hill.egr.msu.edu [35.9.37.162]) by mx1.freebsd.org (Postfix) with ESMTP id 55874E62 for ; Mon, 30 Mar 2015 13:11:22 +0000 (UTC) Received: from hill (localhost [127.0.0.1]) by mail.egr.msu.edu (Postfix) with ESMTP id C71C02E1D4 for ; Mon, 30 Mar 2015 09:03:10 -0400 (EDT) X-Virus-Scanned: amavisd-new at egr.msu.edu Received: from mail.egr.msu.edu ([127.0.0.1]) by hill (hill.egr.msu.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OvPQ3WWAAk-g for ; Mon, 30 Mar 2015 09:03:10 -0400 (EDT) Received: from EGR authenticated sender mcdouga9 Message-ID: <5519498E.1080302@egr.msu.edu> Date: Mon, 30 Mar 2015 09:03:10 -0400 From: Adam McDougall User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: Intel HD 5500 driver port References: <20150330093352.GA3937@valhala.home> <20150330094610.GB3937@valhala.home> In-Reply-To: <20150330094610.GB3937@valhala.home> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2015 13:11:23 -0000 On 03/30/2015 05:46, Nicolas Martyanoff wrote: > On 2015-03-30 11:39, Johannes Dieterich wrote: >> On Mon, Mar 30, 2015 at 11:33 AM, Nicolas Martyanoff wrote: >>> Hi, >>> >>> I have trouble finding info about support for the Intel HD 5500 graphic chip >>> (used on broadwell platforms such as new Thinkpads) on FreeBSD. On Linux, it >>> seems to be supported by the Intel driver Since June 2014¹. >>> >>> Is there a port in progress, or some development being done ? >> neither Haswell nor Broadwell iGPUs are currently supported by any >> version of FreeBSD with the intel driver. However, work is underway by >> some developers (kib@, dumbbell@) to make this happen. Please see >> https://wiki.freebsd.org/Graphics for up-to-date status/information of >> all things graphics. >> >> Depending on your requirements, you can use the vesa driver with >> software rendering in the mean time. > > Thank you for the information. I'll try software rendering when I get the > laptop. > This might help you too: https://forums.freebsd.org/threads/xorg-vesa-driver-massive-speedup-using-mtrr-write-combine.46723/ I don't have new enough hardware to try it out myself yet. From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 30 13:40:51 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4684F3E9 for ; Mon, 30 Mar 2015 13:40:51 +0000 (UTC) Received: from mail-wg0-x229.google.com (mail-wg0-x229.google.com [IPv6:2a00:1450:400c:c00::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C83E61DE for ; Mon, 30 Mar 2015 13:40:50 +0000 (UTC) Received: by wgbgs4 with SMTP id gs4so84195785wgb.0 for ; Mon, 30 Mar 2015 06:40:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=ZG2cFS4WpXUTuI1RQ/UNAGGIQxGFgNrlf+K3agm3ZT8=; b=FoCgI95o8/hQaLko5Smy6T0QMSA98bE6t2ZRYSyziiZsP6bXGxhQ7GcGOQI7zVv/58 cN4+k7dkFQdc195FuedztjV+r7Td5LkPuG76sJIg3bI6xw3tjPOdukgtNNgZ2Zp9Yyia AOkVAMrVjTglYvva4yIoRc6JdK0KDhJlJz9SuNSl/4nW0GKbqMvnJI3DD+ZyDWh5ul4J OA5MKczEVtblnJnKkRQnR/XPXr8vhTsyDLb6/WDxK78WJbxRFDLaB9xZ399D8BAOyekW NVYAZmO2MZsabXaGVswYtno7HYWaK38mLCteO0HZiVvYYThN/5Qfakl0rc2tKXj+3MBK 5WMA== X-Received: by 10.194.57.170 with SMTP id j10mr65345903wjq.102.1427722849022; Mon, 30 Mar 2015 06:40:49 -0700 (PDT) Received: from valhala.home ([78.193.58.122]) by mx.google.com with ESMTPSA id gd6sm16377793wib.17.2015.03.30.06.40.48 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 30 Mar 2015 06:40:48 -0700 (PDT) Date: Mon, 30 Mar 2015 15:40:46 +0200 From: Nicolas Martyanoff To: Adam McDougall Subject: Re: Intel HD 5500 driver port Message-ID: <20150330134046.GA937@valhala.home> References: <20150330093352.GA3937@valhala.home> <20150330094610.GB3937@valhala.home> <5519498E.1080302@egr.msu.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5519498E.1080302@egr.msu.edu> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2015 13:40:51 -0000 On 2015-03-30 09:03, Adam McDougall wrote: > This might help you too: > https://forums.freebsd.org/threads/xorg-vesa-driver-massive-speedup-using-mtrr-write-combine.46723/ > > I don't have new enough hardware to try it out myself yet. This is impressive indeed. I will make sure to test it once I get the hardware myself, and will report the results. I hope it is fast enough for a 1920x1080 panel. -- Nicolas Martyanoff http://wandrian.net khaelin@gmail.com From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 30 17:34:03 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 952A4319; Mon, 30 Mar 2015 17:34:03 +0000 (UTC) Received: from mail-wi0-x22f.google.com (mail-wi0-x22f.google.com [IPv6:2a00:1450:400c:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17999766; Mon, 30 Mar 2015 17:34:03 +0000 (UTC) Received: by wibgn9 with SMTP id gn9so140056940wib.1; Mon, 30 Mar 2015 10:34:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=6Ccns7CxnvzFZsbJBcxnHu1K/Qmyg+8RABcFYq+sm1g=; b=siKQTFbzmW3wwCWh2pSXDyRS+z7GTMNQ3i4dx5ojjEGgVyjuEEtUM+HfmT2iaAmJqX 4xzGAzYCRZLcPqJ2QNqbaZWDTB5d1gx3ilng1ZQJKU6fvoSvp4kTfz7Hr0f7D3K974sf I7saAq6ReYuvExZqBnFuO8ZqNT3RaMPEDgLwtXxXXRBKdYfENQJOUTsOCPUt3trVhI1S vKGKN9o3bEijZEsCE8DqmGjH0OLA6DF1sMfyNQyex3mJCxNPzQM2WIv0ZQ8NffYNno9H jiWp4V985NES6r6qY77pH3kSNVJGW3xXjm/jv2Ff9OF4getKSvA8Sv1hwi7FPyfRaM7+ P9ug== X-Received: by 10.194.185.68 with SMTP id fa4mr64303925wjc.111.1427736841623; Mon, 30 Mar 2015 10:34:01 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id uo6sm16650017wjc.49.2015.03.30.10.34.00 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Mon, 30 Mar 2015 10:34:00 -0700 (PDT) Date: Mon, 30 Mar 2015 19:33:58 +0200 From: Mateusz Guzik To: Oliver Pinter Subject: Re: How to traverse kernel threads? Message-ID: <20150330173358.GB9095@dft-labs.eu> Mail-Followup-To: Mateusz Guzik , Oliver Pinter , Yue Chen , Benjamin Kaduk , "freebsd-hackers@freebsd.org" , HardenedBSD Core , PaX Team References: <20150321220246.GE14650@dft-labs.eu> <20150321232622.GF14650@dft-labs.eu> <20150327194920.GB18158@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: PaX Team , Benjamin Kaduk , HardenedBSD Core , Yue Chen , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2015 17:34:03 -0000 On Fri, Mar 27, 2015 at 11:20:39PM +0100, Oliver Pinter wrote: > On Fri, Mar 27, 2015 at 9:55 PM, Yue Chen wrote: > >> Except it seems there routines are supposed to be only used when > >> execution is 'frozen' (e.g. when escaped to the debugger). > > > > It means probably we can run the code in ``smp_rendezvous()'' function, > > right? > > > >> Still nobody knows what you are trying to do. > > > > We are trying to enhance FreeBSD's security by randomizing kernel code basic > > blocks periodically at runtime, to mitigate the attacks like return-oriented > > programming (ROP). It is basically a stronger form of ASLR. > > After each randomization procedure, the function return addresses saved in > > the stack are the ``old'' addresses before randomization, so we need to > > update them to the new addresses. > > That's why we need to get all the stack ranges to find those addresses. > > > > Also, in kernel, we believe that not only the return addresses in stacks > > need to be updated, there may exist other ``old'' saved instruction (PC) > > addresses in memory. Like in exception handling (maybe, do not know), > > debugging-purpose code and restartable atomic sequences (RAS) > > implementation. That's why I asked how to traverse all the kernel pages and > > get their virtual addresses here: > > https://lists.freebsd.org/pipermail/freebsd-hackers/2015-March/047336.html > > > > Now we found that it seems needed to traverse the ``pv_entry'' structure for > > x86_64 MMU. > > > > Another problem is that we do not know if FreeBSD has any form of special > > encodings or offset form for 64-bit instruction addresses (e.g., saved %RIP) > > on X86_64, instead of hard-coded addresses. For example, using a 32-bit > > offset instead of the 64-bit full address; and doing what glibc does for the > > setjmp/longjmp jmp_buf (special encodings (PTR_MANGLE) for the saved > > register values). > > > > Any suggestion or correction are highly appreciated. > > > > Best, > > Yue > > (Added HardenedBSD core and PaXTeam to CC.) > > Until you can not fixed all of the infoleaks from kernel (try sysctl > -a | grep 0x or similar command) the KASLR and other kernel address > space randomization techniques are easily bypass-able... > I do not believe this can be implemented reliably without some serious tinkering around the kernel. Surely I'm not a live patching expert (not any other kind of expert), but hear my out. It seems proposed approach is to move the kernel around and then updated all relevant pointers in all pages. I do not believe this can be done reliably without corrupting data, unless a major surgery is performed on the kernel. Consider: struct meh { func_t *m_func; size_t m_len; data *m_buf; }; Here we have 'm_func' which needs to be updated, but we don't know if that's the only pointer so we have to scan the entire struct. But m_buf can have data which looks like kernel pointers (i.e. matches some kernel funcs) and how will you know not to modify it? What if this is some sort of a hack and in fact you *should* modify it? CTF or some other solutions like that don't help if you just traverse all allocated buffers since you have no idea what the object you found really is. So, assuming this has to be done in runtime, the only somewhat workable solution I see would require leaving function entry points at contant addresses and only insert jumps to new locations. This would be a performance hit and would leave a lot of data at constant addresses, defeating the point to some extent. Also runtime relocation of everything definitely has a lot of unknown unknowns, so all in all I would say this is a non-starter. One could consider a different approach where kernel data is randomly shuffled around with some granularity and relevant symbol relocated prior to booting. This should provide unique enough layout, which paired with big likelyhood of a kernel panic on first bad exploit attempt may be an acceptable solution. But then the kernel may still have enough info leaks for this to not matter, so I would not be so eager to implement it. That said, what prompted this entire effort? Is there an operating system which got this to work or what? > > > > > > > > On Fri, Mar 27, 2015 at 3:49 PM, Mateusz Guzik wrote: > >> > >> On Fri, Mar 27, 2015 at 02:35:55PM -0400, Yue Chen wrote: > >> > When using the following code on kernel module loading: > >> > > >> > ------------------------------------------------------------------------------------------ > >> > struct thread *td = kdb_thr_first(); > >> > td = kdb_thr_next(td); > >> > > >> > ------------------------------------------------------------------------------------------ > >> > The kernel panics. > >> > > >> > >> Panics how? > >> > >> Also you can easily see these functions don't lock anything, so it would > >> be assumed you took appropriate locks. > >> > >> Except it seems there routines are supposed to be only used when > >> execution is 'frozen' (e.g. when escaped to the debugger). > >> > >> > > >> > And when printing all threads in proc0 (all kernel threads?): > >> > > >> > ------------------------------------------------------------------------------------------ > >> > struct proc *p = pfind(0); > >> > FOREACH_THREAD_IN_PROC(p, td) { > >> > uprintf("td: %x\n", td); > >> > } > >> > > >> > >> proc0 is an exported symbol, no need to pfind. > >> > >> > td = curthread; > >> > uprintf("cur td: %x\n", td); > >> > > >> > ------------------------------------------------------------------------------------------ > >> > The ``curthread'' (from this kernel module running the above code) is > >> > not > >> > in the 0 process group. > >> > > >> > >> There is no 'curthread from kernel module'. > >> > >> My guess is you do this work from module initializator, and in that case > >> curthread is the thread which loads the module, and such a thread is > >> definitely not linked into proc0. > >> > >> Still nobody knows what you are trying to do. > >> > >> -- > >> Mateusz Guzik > > > > -- Mateusz Guzik From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 30 20:52:27 2015 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 00EB1413; Mon, 30 Mar 2015 20:52:26 +0000 (UTC) Received: from mail-wi0-x230.google.com (mail-wi0-x230.google.com [IPv6:2a00:1450:400c:c05::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 84CEF3C8; Mon, 30 Mar 2015 20:52:26 +0000 (UTC) Received: by wibg7 with SMTP id g7so790011wib.1; Mon, 30 Mar 2015 13:52:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=ngPMcVmOEAB/iUVaquIP2j2jMWUJqtLw5IhNsUQBgwM=; b=Hn+Y4mb2cifRA7Zi24jO8+P3LbMDKbsnG+AnLqBFBmN5l2MD4dc+MzT0tkZHBHbdcg o3Slwi93aCP/KO4Oi06bSOnqDuRkc7/IBXjDxn9o1oV47puZmnXWtLuPcX0zIb8unGJa 2Snh0Xc325kehjOJE+N8C0SZQ12ezQgQdULrFWzAHCv1kVPWmcDoHdDiexO/UlMT+eDo /eBXaNQj4xA9VbKMcj9Xq7JeuDgvbxkW9oxWi3v6wPFAW/0+3SeGXEv93SwkID3aAowk Ap5g3rQtB3pxweG8+UrTaXs2C6Ysz9/Lg41Y+MARYGPRwZebVmmYDQaTvL3GiEqX5G7a EzKA== X-Received: by 10.194.201.164 with SMTP id kb4mr66732244wjc.32.1427748745021; Mon, 30 Mar 2015 13:52:25 -0700 (PDT) Received: from brick.home (abvb139.neoplus.adsl.tpnet.pl. [83.8.199.139]) by mx.google.com with ESMTPSA id hq4sm12097396wib.0.2015.03.30.13.52.23 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 30 Mar 2015 13:52:24 -0700 (PDT) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Date: Mon, 30 Mar 2015 22:52:22 +0200 From: Edward Tomasz Napierala To: Chagin Dmitry Subject: Re: Linux core under FreeBSD. Message-ID: <20150330205222.GC12290@brick.home> References: <20150327124058.GA3991@brick.home> <20150327154348.GA90856@dchagin.static.corbina.net> <20150328084750.GA6548@brick.home> <20150328111000.GA97431@dchagin.static.corbina.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150328111000.GA97431@dchagin.static.corbina.net> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2015 20:52:27 -0000 On 0328T1410, Chagin Dmitry wrote: > On Sat, Mar 28, 2015 at 09:47:50AM +0100, Edward Tomasz Napierala wrote: > > On 0327T1843, Chagin Dmitry wrote: > > > On Fri, Mar 27, 2015 at 01:40:58PM +0100, Edward Tomasz Napierala wrote: > > > > Is there a way to debug a core generated by Linux process running > > > > under Linuxulator on 11-CURRENT? Thanks! > > > > > > > ptrace implemented only for i386, so ktrace/kdump and > > > machdep.uprintf_signal=1 > > > > Ok, but what about core files from Linux processes? Are they useless > > at this moment? If so, perhaps they should just be disabled? > I never debugged core from Linux procs. I can write sysctl to enable core > of Linux procs. Erm, the core _gets_ dumped. I'm just suggesting to disable dumping it, if there is no way to make use of it. From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 30 22:57:24 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A30EBC8B for ; Mon, 30 Mar 2015 22:57:24 +0000 (UTC) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::103]) by mx1.freebsd.org (Postfix) with ESMTP id 78EB9754 for ; Mon, 30 Mar 2015 22:57:24 +0000 (UTC) Received: from [172.16.1.67] (unknown [172.16.1.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id A63002E5E0 for ; Mon, 30 Mar 2015 22:57:23 +0000 (UTC) Message-ID: <5519D4D3.6080707@metricspace.net> Date: Mon, 30 Mar 2015 18:57:23 -0400 From: Eric McCorkle User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: ZFS support for EFI References: <55189CBA.9040107@metricspace.net> <543637C0-A4FF-4801-BE5C-859F2D968D48@me.com> In-Reply-To: <543637C0-A4FF-4801-BE5C-859F2D968D48@me.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2015 22:57:24 -0000 On 03/30/2015 12:28 AM, Rui Paulo wrote: >> First, I've converted boot1.c (the EFI boot block) to use an FS module >> framework. This facilitates the addition of ZFS, and should also come >> in handy if someone wants to add other functionality later (ie. crypto, >> netboot, etc.) > > Good. :-) Actually, would that be a good patch in its own right? I could certainly strip out the ZFS-related stuff for folks with UFS+EFI systems to test. (I only have ZFS drives myself) > I have a crazy idea: why not use getopt() in loader.efi ? getopt() is already part of libstand, so it should be easy to use it. > > Alternatively you can just use key value pairs. > I did a little more lookung. Turns out, I had it wrong. The old ZFS loader loads and parses /boot/config, not /boot/loader.conf. It appears that the existing EFI loader is ignoring /boot/config. Is that intentional, or just missing functionality? (It would probably make more sense to stash /boot/config or its analog on the ESP anyway.) From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 31 01:24:14 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8DE4EBD8 for ; Tue, 31 Mar 2015 01:24:14 +0000 (UTC) Received: from st11p02mm-asmtp001.mac.com (st11p02mm-asmtpout001.mac.com [17.172.220.236]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 617E58AE for ; Tue, 31 Mar 2015 01:24:13 +0000 (UTC) Received: from fukuyama.hsd1.ca.comcast.net (c-73-162-13-215.hsd1.ca.comcast.net [73.162.13.215]) by st11p02mm-asmtp001.mac.com (Oracle Communications Messaging Server 7.0.5.35.0 64bit (built Dec 4 2014)) with ESMTPSA id <0NM1001A0YJCH530@st11p02mm-asmtp001.mac.com> for freebsd-hackers@freebsd.org; Tue, 31 Mar 2015 01:23:38 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.13.68,1.0.33,0.0.0000 definitions=2015-03-30_04:2015-03-30,2015-03-30,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1412110000 definitions=main-1503310011 Content-type: text/plain; charset=us-ascii MIME-version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: ZFS support for EFI From: Rui Paulo In-reply-to: <5519D4D3.6080707@metricspace.net> Date: Mon, 30 Mar 2015 18:23:35 -0700 Content-transfer-encoding: quoted-printable Message-id: References: <55189CBA.9040107@metricspace.net> <543637C0-A4FF-4801-BE5C-859F2D968D48@me.com> <5519D4D3.6080707@metricspace.net> To: Eric McCorkle X-Mailer: Apple Mail (2.2070.6) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2015 01:24:14 -0000 On Mar 30, 2015, at 15:57, Eric McCorkle wrote: >=20 > On 03/30/2015 12:28 AM, Rui Paulo wrote: >=20 >>> First, I've converted boot1.c (the EFI boot block) to use an FS = module >>> framework. This facilitates the addition of ZFS, and should also = come >>> in handy if someone wants to add other functionality later (ie. = crypto, >>> netboot, etc.) >>=20 >> Good. :-) >=20 > Actually, would that be a good patch in its own right? I could > certainly strip out the ZFS-related stuff for folks with UFS+EFI = systems > to test. (I only have ZFS drives myself) This might actually be useful to split UFS1 and UFS2. Adding Warner. >> I have a crazy idea: why not use getopt() in loader.efi ? getopt() = is already part of libstand, so it should be easy to use it. >>=20 >> Alternatively you can just use key value pairs. >>=20 >=20 > I did a little more lookung. Turns out, I had it wrong. The old ZFS > loader loads and parses /boot/config, not /boot/loader.conf. >=20 > It appears that the existing EFI loader is ignoring /boot/config. Is > that intentional, or just missing functionality? (It would probably > make more sense to stash /boot/config or its analog on the ESP = anyway.) That's just a mistake in the EFI loader. /boot/config is the same thing = as /boot.config and both are missing from EFI. I don't quite understand = how this is related to the previous problem. -- Rui Paulo From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 31 02:59:47 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 84CBD25D; Tue, 31 Mar 2015 02:59:47 +0000 (UTC) Received: from mail-pa0-x233.google.com (mail-pa0-x233.google.com [IPv6:2607:f8b0:400e:c03::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ED0C2DE; Tue, 31 Mar 2015 02:59:47 +0000 (UTC) Received: by patj18 with SMTP id j18so5357591pat.2; Mon, 30 Mar 2015 19:59:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=4Rb0Uu/oxQ+BHIdmdgAvOSz3sxMNJ1Cw7Gb3DfgBtmA=; b=Rq9iElPrgFKQn/FhXj59i+UNE6BpXsEIQ4c9p0XgOls2bxTSAq9RWY3CsMhxKfRvAr rh46RmzeUk/TJtX8O3zQGQmognPan5jTYc6L7r/p1YRMz+rK9Wcx7ap229kmGw4zA5Yl 6BB699OaKYlxPvgYfnmLmEZd0Vpat3wqn8U3Eo+HUd4tIsxLyJzK0oZzVqnxKNVO5cTT X6Os67o4CrL/64I/4QFA/KZYIVjFgo6PboiH8bBunQrz/rG0n9HwHw21TlmN0uGbu4Qt dDg/crymG0Rma9JhJ9p8MWGn6I2sPKJ+8hwLEu5XjMUV3DukW63YtBj17nMMyg4YVj4I KpLg== X-Received: by 10.70.125.162 with SMTP id mr2mr63125099pdb.21.1427770786662; Mon, 30 Mar 2015 19:59:46 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.250.131 with HTTP; Mon, 30 Mar 2015 19:59:16 -0700 (PDT) In-Reply-To: <20150330173358.GB9095@dft-labs.eu> References: <20150321220246.GE14650@dft-labs.eu> <20150321232622.GF14650@dft-labs.eu> <20150327194920.GB18158@dft-labs.eu> <20150330173358.GB9095@dft-labs.eu> From: Yue Chen Date: Mon, 30 Mar 2015 22:59:16 -0400 Message-ID: Subject: Re: How to traverse kernel threads? To: Mateusz Guzik , Oliver Pinter , Yue Chen , Benjamin Kaduk , "freebsd-hackers@freebsd.org" , HardenedBSD Core , PaX Team Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2015 02:59:47 -0000 > This would be a performance hit. The cost is to traverse the .text area and some other data in memory. It's linear and only happens after a given time interval, like several minutes, and does not put overhead on the normal execution. So the performance overhead won't be high. > The only somewhat workable solution I see would require leaving function entry points at constant addresses and only insert jumps to new locations. Yes, we do this. For the PIC direct jump/call/"instructions including %RIP", we just need to update them. > Also runtime relocation of everything definitely has a lot of unknown unknowns, so all in all I would say this is a non-starter. FreeBSD kernel itself only has 1500+ indirect jumps. There are two types: 1. jump table. Like switch-case. 2. Tail call. We have not found other types like manual assembly code, or context restoration like that in longjmp in libc. Maybe there are some other but we have not found. > Leave a lot of data at constant addresses, defeating the point to some extent. > One could consider a different approach where kernel data is randomly shuffled around with some granularity and > relevant symbol relocated prior to booting. Why do you think the data address exposure could be a serious problem? It is non-executable and the code is randomized. Although for data itself, it is really hard to protect from tampering and may cause other problems. Why the randomized data leads to kernel panic? I think if the data goes somewhere else, the instruction addressing would follow, or page fault may happen. If randomizing code, the exploit attempt may hit an illegal instruction and the kernel panics. > Is there an operating system which got this to work or what? Not yet. On Mon, Mar 30, 2015 at 1:33 PM, Mateusz Guzik wrote: > On Fri, Mar 27, 2015 at 11:20:39PM +0100, Oliver Pinter wrote: > > On Fri, Mar 27, 2015 at 9:55 PM, Yue Chen wrote: > > >> Except it seems there routines are supposed to be only used when > > >> execution is 'frozen' (e.g. when escaped to the debugger). > > > > > > It means probably we can run the code in ``smp_rendezvous()'' function, > > > right? > > > > > >> Still nobody knows what you are trying to do. > > > > > > We are trying to enhance FreeBSD's security by randomizing kernel code > basic > > > blocks periodically at runtime, to mitigate the attacks like > return-oriented > > > programming (ROP). It is basically a stronger form of ASLR. > > > After each randomization procedure, the function return addresses > saved in > > > the stack are the ``old'' addresses before randomization, so we need to > > > update them to the new addresses. > > > That's why we need to get all the stack ranges to find those addresses. > > > > > > Also, in kernel, we believe that not only the return addresses in > stacks > > > need to be updated, there may exist other ``old'' saved instruction > (PC) > > > addresses in memory. Like in exception handling (maybe, do not know), > > > debugging-purpose code and restartable atomic sequences (RAS) > > > implementation. That's why I asked how to traverse all the kernel > pages and > > > get their virtual addresses here: > > > > https://lists.freebsd.org/pipermail/freebsd-hackers/2015-March/047336.html > > > > > > Now we found that it seems needed to traverse the ``pv_entry'' > structure for > > > x86_64 MMU. > > > > > > Another problem is that we do not know if FreeBSD has any form of > special > > > encodings or offset form for 64-bit instruction addresses (e.g., saved > %RIP) > > > on X86_64, instead of hard-coded addresses. For example, using a 32-bit > > > offset instead of the 64-bit full address; and doing what glibc does > for the > > > setjmp/longjmp jmp_buf (special encodings (PTR_MANGLE) for the saved > > > register values). > > > > > > Any suggestion or correction are highly appreciated. > > > > > > Best, > > > Yue > > > > (Added HardenedBSD core and PaXTeam to CC.) > > > > Until you can not fixed all of the infoleaks from kernel (try sysctl > > -a | grep 0x or similar command) the KASLR and other kernel address > > space randomization techniques are easily bypass-able... > > > > I do not believe this can be implemented reliably without some serious > tinkering around the kernel. Surely I'm not a live patching expert (not > any other kind of expert), but hear my out. > > It seems proposed approach is to move the kernel around and then updated > all relevant pointers in all pages. > > I do not believe this can be done reliably without corrupting data, > unless a major surgery is performed on the kernel. > > Consider: > struct meh { > func_t *m_func; > size_t m_len; > data *m_buf; > }; > > Here we have 'm_func' which needs to be updated, but we don't know if > that's the only pointer so we have to scan the entire struct. But m_buf > can have data which looks like kernel pointers (i.e. matches some kernel > funcs) and how will you know not to modify it? What if this is some sort > of a hack and in fact you *should* modify it? > > CTF or some other solutions like that don't help if you just traverse > all allocated buffers since you have no idea what the object you found > really is. > > So, assuming this has to be done in runtime, the only somewhat workable > solution I see would require leaving function entry points at contant > addresses and only insert jumps to new locations. > > This would be a performance hit and would leave a lot of data at > constant addresses, defeating the point to some extent. > > Also runtime relocation of everything definitely has a lot of unknown > unknowns, so all in all I would say this is a non-starter. > > One could consider a different approach where kernel data is randomly > shuffled around with some granularity and relevant symbol relocated > prior to booting. > > This should provide unique enough layout, which paired with big > likelyhood of a kernel panic on first bad exploit attempt may be an > acceptable solution. > > But then the kernel may still have enough info leaks for this to not > matter, so I would not be so eager to implement it. > > That said, what prompted this entire effort? Is there an operating > system which got this to work or what? > > > > > > > > > > > > > On Fri, Mar 27, 2015 at 3:49 PM, Mateusz Guzik > wrote: > > >> > > >> On Fri, Mar 27, 2015 at 02:35:55PM -0400, Yue Chen wrote: > > >> > When using the following code on kernel module loading: > > >> > > > >> > > ------------------------------------------------------------------------------------------ > > >> > struct thread *td = kdb_thr_first(); > > >> > td = kdb_thr_next(td); > > >> > > > >> > > ------------------------------------------------------------------------------------------ > > >> > The kernel panics. > > >> > > > >> > > >> Panics how? > > >> > > >> Also you can easily see these functions don't lock anything, so it > would > > >> be assumed you took appropriate locks. > > >> > > >> Except it seems there routines are supposed to be only used when > > >> execution is 'frozen' (e.g. when escaped to the debugger). > > >> > > >> > > > >> > And when printing all threads in proc0 (all kernel threads?): > > >> > > > >> > > ------------------------------------------------------------------------------------------ > > >> > struct proc *p = pfind(0); > > >> > FOREACH_THREAD_IN_PROC(p, td) { > > >> > uprintf("td: %x\n", td); > > >> > } > > >> > > > >> > > >> proc0 is an exported symbol, no need to pfind. > > >> > > >> > td = curthread; > > >> > uprintf("cur td: %x\n", td); > > >> > > > >> > > ------------------------------------------------------------------------------------------ > > >> > The ``curthread'' (from this kernel module running the above code) > is > > >> > not > > >> > in the 0 process group. > > >> > > > >> > > >> There is no 'curthread from kernel module'. > > >> > > >> My guess is you do this work from module initializator, and in that > case > > >> curthread is the thread which loads the module, and such a thread is > > >> definitely not linked into proc0. > > >> > > >> Still nobody knows what you are trying to do. > > >> > > >> -- > > >> Mateusz Guzik > > > > > > > > -- > Mateusz Guzik > From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 31 22:59:51 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 61BE554A; Tue, 31 Mar 2015 22:59:51 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 14F5C6C6; Tue, 31 Mar 2015 22:59:51 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1Yd58N-000HbM-M4; Wed, 01 Apr 2015 01:59:47 +0300 Date: Wed, 1 Apr 2015 01:59:47 +0300 From: Slawa Olhovchenkov To: Adrian Chadd Subject: Re: irq cpu binding Message-ID: <20150331225947.GC23643@zxy.spb.ru> References: <20150328112035.GZ23643@zxy.spb.ru> <20150328154031.GA23643@zxy.spb.ru> <20150328181026.GB23643@zxy.spb.ru> <20150328183147.GC23643@zxy.spb.ru> <20150328192505.GD23643@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2015 22:59:51 -0000 On Sat, Mar 28, 2015 at 12:33:52PM -0700, Adrian Chadd wrote: > That's done deferred by the bus interrupt wiring. That's something > John's been looking into as part of the general NUMA work (and I'm > trying to debug right now, on dual-socket boxes with ixgbe. :-) > > Look at bus_bind_intr() and the twisty path to intr_event_bind(), then > x86/x86/intr_machdep.c:intr_assign_cpu(), then intr_shuffle_cpus() at > boot, versus what happens via calls to pic_assign_cpu to setup the > wiring. I am do simple, ugle hack ixgbe driver for let start cpu binding. I am still see ixgbe in pmc output. What may be wrong? What may be miss? ===== static int ixgbe_start_cpu = 0; TUNABLE_INT("hw.ix.start_cpu", &ixgbe_start_cpu); SYSCTL_INT(_hw_ix, OID_AUTO, start_cpu, CTLFLAG_RDTUN, &ixgbe_start_cpu, 0, "Start CPU for next IRQ binding"); [...] if (adapter->num_queues > 1) bus_bind_intr(dev, que->res, i+ixgbe_start_cpu); #ifndef IXGBE_LEGACY_TX TASK_INIT(&txr->txq_task, 0, ixgbe_deferred_mq_start, txr); #endif TASK_INIT(&que->que_task, 0, ixgbe_handle_que, que); que->tq = taskqueue_create_fast("ixgbe_que", M_NOWAIT, taskqueue_thread_enqueue, &que->tq); taskqueue_start_threads(&que->tq, 1, PI_NET, "%s que", device_get_nameunit(adapter->dev)); } ixgbe_start_cpu += adapter->num_queues; From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 31 23:14:06 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF844A5A for ; Tue, 31 Mar 2015 23:14:05 +0000 (UTC) Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE64C8D8 for ; Tue, 31 Mar 2015 23:14:05 +0000 (UTC) Received: by iedm5 with SMTP id m5so29631043ied.3 for ; Tue, 31 Mar 2015 16:14:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=hc1tcvzMxkrs/3IywrCkP0FmiJ4QKui14w+46T0W3uM=; b=PZoCiDUNfJAUInqa+eQWRwKs9osihtWYzTLguUVvqcQJ1TK30TsBhNzoAsd6mUCIPi 2blqHoOcIs3ytQG/zVlFzDhbenxwUN4Tf9sleZwUGuPBk/hMY1hFriY7Ln7BNuhtkdZT 8CDWbPAfR5y24eppP5cQkwJpdx3HUo4htoTm0DGSWW1gnIXxHin/Bi1acToNG5zjBA3V AVwqFi6CNFIEJdu1+2yd9mgqymV8GQTVYNUh4wo9PxB3aQ/lv0h3yQAjdeIrh+tuKERA qa6PC6lRzJcRTvIVfYqzqt5bnmyFN1GzgoMtKf0rzM+hEj0G7piGHP1PdjbkMFoYN9Rb AlGA== MIME-Version: 1.0 X-Received: by 10.107.39.72 with SMTP id n69mr44465639ion.8.1427843644983; Tue, 31 Mar 2015 16:14:04 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Tue, 31 Mar 2015 16:14:04 -0700 (PDT) In-Reply-To: <20150331225947.GC23643@zxy.spb.ru> References: <20150328112035.GZ23643@zxy.spb.ru> <20150328154031.GA23643@zxy.spb.ru> <20150328181026.GB23643@zxy.spb.ru> <20150328183147.GC23643@zxy.spb.ru> <20150328192505.GD23643@zxy.spb.ru> <20150331225947.GC23643@zxy.spb.ru> Date: Tue, 31 Mar 2015 16:14:04 -0700 X-Google-Sender-Auth: CVgL8MyGBQP3YyD7U6qQL0X8ktQ Message-ID: Subject: Re: irq cpu binding From: Adrian Chadd To: Slawa Olhovchenkov Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2015 23:14:06 -0000 You also have to do the taskqueue threads. (I mean, pmc could also be broken...) -a On 31 March 2015 at 15:59, Slawa Olhovchenkov wrote: > On Sat, Mar 28, 2015 at 12:33:52PM -0700, Adrian Chadd wrote: > >> That's done deferred by the bus interrupt wiring. That's something >> John's been looking into as part of the general NUMA work (and I'm >> trying to debug right now, on dual-socket boxes with ixgbe. :-) >> >> Look at bus_bind_intr() and the twisty path to intr_event_bind(), then >> x86/x86/intr_machdep.c:intr_assign_cpu(), then intr_shuffle_cpus() at >> boot, versus what happens via calls to pic_assign_cpu to setup the >> wiring. > > I am do simple, ugle hack ixgbe driver for let start cpu binding. > I am still see ixgbe in pmc output. > > What may be wrong? > What may be miss? > > ===== > static int ixgbe_start_cpu = 0; > TUNABLE_INT("hw.ix.start_cpu", &ixgbe_start_cpu); > SYSCTL_INT(_hw_ix, OID_AUTO, start_cpu, CTLFLAG_RDTUN, &ixgbe_start_cpu, 0, > "Start CPU for next IRQ binding"); > > [...] > if (adapter->num_queues > 1) > bus_bind_intr(dev, que->res, i+ixgbe_start_cpu); > > #ifndef IXGBE_LEGACY_TX > TASK_INIT(&txr->txq_task, 0, ixgbe_deferred_mq_start, txr); > #endif > TASK_INIT(&que->que_task, 0, ixgbe_handle_que, que); > que->tq = taskqueue_create_fast("ixgbe_que", M_NOWAIT, > taskqueue_thread_enqueue, &que->tq); > taskqueue_start_threads(&que->tq, 1, PI_NET, "%s que", > device_get_nameunit(adapter->dev)); > } > ixgbe_start_cpu += adapter->num_queues; > From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 01:15:10 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3466E1F2; Wed, 1 Apr 2015 01:15:10 +0000 (UTC) Received: from mx142.netapp.com (mx142.netapp.com [216.240.21.19]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mx142.netapp.com", Issuer "VeriSign Class 3 International Server CA - G3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 01FC57B6; Wed, 1 Apr 2015 01:15:09 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.11,503,1422950400"; d="scan'208";a="32239945" Received: from hioexcmbx04-prd.hq.netapp.com ([10.122.105.37]) by mx142-out.netapp.com with ESMTP; 31 Mar 2015 17:54:19 -0700 Received: from HIOEXCMBX03-PRD.hq.netapp.com (10.122.105.36) by hioexcmbx04-prd.hq.netapp.com (10.122.105.37) with Microsoft SMTP Server (TLS) id 15.0.995.29; Tue, 31 Mar 2015 17:54:19 -0700 Received: from HIOEXCMBX03-PRD.hq.netapp.com ([::1]) by hioexcmbx03-prd.hq.netapp.com ([fe80::4c9:7f9e:4b9f:2c9c%21]) with mapi id 15.00.0995.031; Tue, 31 Mar 2015 17:54:19 -0700 From: "Gumpula, Suresh" To: "freebsd-hackers@freebsd.org" Subject: Re: BSD 8.1 and 9.1 memory increase Thread-Topic: BSD 8.1 and 9.1 memory increase Thread-Index: AQHQYN1kO9bXTk+QcE+8E2hd8pdV+Z0hK5sAgBZxt4A= Date: Wed, 1 Apr 2015 00:54:18 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.7.141117 x-originating-ip: [10.122.56.79] Content-Type: text/plain; charset="Windows-1252" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "alc@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 01:15:10 -0000 Still trying to find out the reason for more memory foot print on 9.1 compared to 8.1 . Does some thing like clustering changes in page fault handling cause memory foot print ? https://svnweb.freebsd.org/base?view=3Drevision&revision=3D235876 Copying Alan Cox , if could throw some inputs on this. Thank you Suresh On 3/17/15, 2:09 PM, "Gumpula, Suresh" wrote: >On similar notes , when I looked at procstat -v , it does not show us >the shared libraries resident usage . >I have added a column to display the shared resident count. > >kern_proc.c: > if (obj->shadow_count > 1) > kve->kve_shared_resident =3D > obj->resident_page_count; > >procstat_vm.c : > > if (kve->kve_private_resident || kve->kve_shared_resident) > entry_total =3D(kve->kve_private_resident + >kve->kve_shared_resident); > else > entry_total =3D kve->kve_resident; > > proc_total +=3D entry_total; > shared_total +=3D kve->kve_shared_resident; > private_total +=3D kve->kve_private_resident; > mapped_total +=3D kve->kve_resident; > > printf("%8d ", kve->kve_shared_resident); > printf("%8d ", entry_total); > > > > > >% sudo procstat -v `pgrep bash` > PID START END PRT RES PRES >SHRD TOTAL REF SHD FL TP PATH > 3485 0x400000 0x4b0000 r-x 128 151 >0 151 2 1 CN-- vn /usr/bin/bash > 3485 0x6b0000 0x6b9000 rw- 9 0 >0 9 1 0 C--- vn /usr/bin/bash > 3485 0x6b9000 0x800000 rw- 5 0 >0 5 1 0 C--- df > 3485 0x8006b0000 0x8006c9000 r-x 25 0 >0 25 38 0 CN-- vn /libexec/ld-elf.so.1 > 3485 0x8006c9000 0x8006ef000 rw- 26 0 >0 26 1 0 CN-- df > 3485 0x8008c8000 0x8008ca000 rw- 2 0 >0 2 1 0 CN-- df > 3485 0x8008ca000 0x800914000 r-x 40 0 >79 79 6 3 CN-- vn /lib/libncurses.so.8 > 3485 0x800914000 0x800b14000 --- 0 0 >0 0 1 0 CN-- df > 3485 0x800b14000 0x800b19000 rw- 5 0 >0 5 1 0 CN-- vn /lib/libncurses.so.8 > 3485 0x800b19000 0x800c71000 r-x 318 0 >331 331 75 37 CN-- vn /lib/libc.so.7 > 3485 0x800c71000 0x800e71000 --- 0 0 >0 0 1 0 CN-- df > 3485 0x800e71000 0x800e7c000 rw- 11 0 >0 11 1 0 CN-- vn /lib/libc.so.7 > 3485 0x800e7c000 0x800ee2000 rw- 9 0 >0 9 1 0 CN-- df > 3485 0x801000000 0x801200000 rw- 35 0 >0 35 1 0 C--- df > 3485 0x7ffffffdf000 0x7ffffffff000 rw- 5 0 >0 5 1 0 C--D df > 3485 0x7ffffffff000 0x800000000000 r-x 1 0 >0 1 40 0 CN-- ph > Totals(pages) 619 151 >410 694 > > >Can somebody pick up this change if it looks good ? > >Thanks >Suresh > > >From: , suresh gumpula >> >Date: Tuesday, March 17, 2015 at 2:08 PM >To: "freebsd-hackers@freebsd.org" >> >Subject: BSD 8.1 and 9.1 memory increase > >Hello VM experts, > I am trying to figure out what has been changed from 8.1 to 9.1 which >results more memory footprint on all processes. looking at one of the big >processes we have on a idle machine, >its about 35M resident size increase. Looking at map entries by procstat >-v shows me that two libraries , one is our internal library(libmgwd) and >other one is boost consume more now. >There are no changes made with respect to process, just comparing after >the upgrade to 9.1. > >Are here any knows things with respect VM have been changed and could >result in more resident memory ? Can somebody please help on this to know >what exactly causing this behaviour ? >Details below. > >Thank you! > > >8.1 >=8B >% sudo procstat -v `pgrep mgwd` > PID START END PRT RES PRES REF SHD FL TP >PATH > > 2213 0x800a46000 0x807b41000 r-x 18209 22827 2 1 CN vn >/usr/lib/libmgwd.so.1 > 2213 0x807d40000 0x808a90000 rw- 3019 0 1 0 C- vn >/usr/lib/libmgwd.so.1 > > > 2213 0x833406000 0x833407000 rw- 1 0 1 0 CN vn >/usr/lib/libboost_atomic.so.1.56.0 > 2213 0x833600000 0x835600000 rw- 8134 0 1 0 C- df > 2213 0x835600000 0x835800000 rw- 20 0 1 0 -- df > 2213 0x7ffff6f99000 0x7ffff6fb9000 rw- 1 0 1 0 -- df > 2213 0x7ffff719a000 0x7ffff71ba000 rw- 1 0 1 0 -- df > 2213 0x7ffff739b000 0x7ffff73bb000 rw- 12 0 1 0 -- df > > >9.1 >=8B=8B >% sudo procstat -v `pgrep mgwd` > PID START END PRT RES PRES REF SHD FL TP >PATH > > 2158 0x800a1c000 0x807c87000 r-x 23328 26760 2 1 CN-- >vn /usr/lib/libmgwd.so.1 > 2158 0x807e87000 0x808bd4000 rw- 3283 0 1 0 C--- >vn /usr/lib/libmgwd.so.1 > > 2158 0x8336d7000 0x8336d8000 rw- 1 0 1 0 CN-- vn >/usr/lib/libboost_atomic.so.1.56.0 > 2158 0x8336d8000 0x8342cf000 rw- 2105 0 1 0 C-S- df > 2158 0x8342cf000 0x8342ea000 rw- 27 0 1 0 C--- df > 2158 0x8342ea000 0x8342f3000 rw- 7 0 1 0 ---- df > 2158 0x834400000 0x834600000 rw- 511 0 1 0 C--- df > 2158 0x834600000 0x836400000 rw- 7375 0 1 0 C--- df > 2158 0x836400000 0x836600000 rw- 228 0 1 0 ---- df > >BSD 8.1 >=3D=3D=3D=3D=3D=3D >last pid: 5116; load averages: 5.22, 4.29, 2.34 up 0+00:10:15 >17:34:00 >352 processes: 1 running, 350 sleeping, 1 zombie >CPU: 0.2% user, 0.0% nice, 4.5% system, 1.7% interrupt, 93.5% idle >Mem: 297M Active, 648M Inact, 139M Wired, 6948K Cache, 7520K Buf, 1862M >Free >Swap: 1536M Total, 1536M Free > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU >COMMAND > 2219 root 68 96 0 859M 226M ucond 1 0:24 0.00% mgwd > >BSD 9.1 >=8B=8B=8B >last pid: 5344; load averages: 5.17, 4.47, 2.79 up 0+00:26:12 >17:22:57 >39 processes: 1 running, 37 sleeping, 1 zombie >CPU: 0.2% user, 0.0% nice, 2.2% system, 0.6% interrupt, 97.0% idle >Mem: 338M Active, 669M Inact, 147M Wired, 392K Cache, 7488K Buf, 1799M >Free >Swap: 1536M Total, 1536M Fre > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU >COMMAND > 2158 root 68 40 0 874M 262M uwait 1 0:23 0.00% mgwd > >=3D=3D=3D=3D=3D=3D=3D=3D > >% ldd /usr/lib/libmgwd.so.1 >/usr/lib/libmgwd.so.1: >% ldd /usr/lib/libboost_atomic.so.1.56.0 >/usr/lib/libboost_atomic.so.1.56.0: > librt.so.1 =3D> /usr/lib/librt.so.1 (0x801002000) > libthr.so.3 =3D> /lib/libthr.so.3 (0x801207000) > > >;;image size comparison on different builds shows TEXT size of a libray >went up by ~1.5M > > text data bss dec hex filename >118484184 139568805724944 13816600883c3ef8 >devN_150110_0500/mgmtgateway/bedrock/internal/x86_64/ulibso/libmgwd.so.1 >119973051 139460085724944 139644003852cc63 >devN_150110_1035/mgmtgateway/bedrock/internal/x86_64/ulibso/libmgwd.so.1 > >BSD 9.1 >=8B=8B >% size /sbin/mgwd > text data bss dec hex filename >2106422 42388 371584 2520394 26754a /sbin/mgwd >% size /usr/lib/libboost_atomic.so.1.56.0 > text data bss dec hex filename > 1309 264 2624 4197 1065 /usr/lib/libboost_atomic.so.1.56.0 > >BSd 8.1 >=8B=8B=8B >% size /sbin/mgwd > text data bss dec hex filename >2091457 42364 371520 2505341 263a7d /sbin/mgwd >% size /usr/lib/libboost_atomic.so.1.56.0 > text data bss dec hex filename > 1309 264 2624 4197 1065 /usr/lib/libboost_atomic.so.1.56.0 > >Thanks >Suresh > >_______________________________________________ >freebsd-hackers@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 03:11:39 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 484BC3AE; Wed, 1 Apr 2015 03:11:39 +0000 (UTC) Received: from mail-pd0-x22e.google.com (mail-pd0-x22e.google.com [IPv6:2607:f8b0:400e:c02::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 13900873; Wed, 1 Apr 2015 03:11:39 +0000 (UTC) Received: by pdrw1 with SMTP id w1so32275032pdr.0; Tue, 31 Mar 2015 20:11:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=7yqS1FynNbbNcK/lcGUM1QOXioGvDaleo2Gc1achTqA=; b=VmY6zq6UuU6Aff06LcRiA8BnG5Ic0Jb0KbZ3CNlEbbRNBl/1KS3CrVDVJzWe6+isln K2XUj66aU8j9kEQ0xj8mxrInykMJmZR6x5BAOnrvZVDWoWQ32C3RKEBxnnaH2ZjanRaq Lxel6dMF4BcqMwWNftzDMAdVhi7IPML3YzA3bR78b6mF3ymbiBPDqeOWnKVul4wupppB GW0fvpB1D6IXhutijPGn32z3vYAhctBxZRu3FoAOTXI0LwqPp9xq6NK3GWPPg0kI0f16 HVsnhA4uJrLEr5fCOixyYoqqgENbqWHTw/x65EOAlvsrwLDJVK0HAtL6FTsPVJbFl0G4 hkqw== X-Received: by 10.66.218.129 with SMTP id pg1mr73606182pac.65.1427857898598; Tue, 31 Mar 2015 20:11:38 -0700 (PDT) Received: from [10.142.111.76] ([166.170.40.159]) by mx.google.com with ESMTPSA id dl2sm357974pbc.28.2015.03.31.20.11.37 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 31 Mar 2015 20:11:37 -0700 (PDT) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: X-Mailer: iPhone Mail (12D508) From: Garrett Cooper Subject: Re: BSD 8.1 and 9.1 memory increase Date: Tue, 31 Mar 2015 20:11:32 -0700 To: "Gumpula, Suresh" Cc: "alc@freebsd.org" , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 03:11:39 -0000 > On Mar 31, 2015, at 17:54, Gumpula, Suresh wro= te: >=20 > Still trying to find out the reason for more memory foot print on 9.1 > compared to 8.1 . > Does some thing like clustering changes in page fault handling cause > memory foot print ? > https://svnweb.freebsd.org/base?view=3Drevision&revision=3D235876 >=20 > Copying Alan Cox , if could throw some inputs on this. Superpages and how FreeBSD does its best to put runtime libraries in superpa= ge-able comes to mind.. The VMEM for libraries is what caught us off guard last year when dealing wi= th applications -- more libraries =3D=3D greater footprint past either 8.0 o= r 9.0 because of changes to VM/rtld. Conrad Meyer had a change out to reduce the footprint for libraries, but it w= as racy/incomplete unfortunately :/.. Hope that maybe helps...= From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 06:23:09 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1A80ED0C; Wed, 1 Apr 2015 06:23:09 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C013B7D3; Wed, 1 Apr 2015 06:23:08 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1YdC3I-000PIx-4Q; Wed, 01 Apr 2015 09:23:00 +0300 Date: Wed, 1 Apr 2015 09:23:00 +0300 From: Slawa Olhovchenkov To: Adrian Chadd Subject: Re: irq cpu binding Message-ID: <20150401062259.GD23643@zxy.spb.ru> References: <20150328154031.GA23643@zxy.spb.ru> <20150328181026.GB23643@zxy.spb.ru> <20150328183147.GC23643@zxy.spb.ru> <20150328192505.GD23643@zxy.spb.ru> <20150331225947.GC23643@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 06:23:09 -0000 On Tue, Mar 31, 2015 at 04:14:04PM -0700, Adrian Chadd wrote: > You also have to do the taskqueue threads. # procstat -ta | grep ix 0 101305 kernel ix0 que 6 8 sleep - 0 101306 kernel ix0 que 7 8 sleep - 0 101307 kernel ix0 que 8 8 sleep - 0 101308 kernel ix0 linkq 7 8 sleep - 0 101309 kernel ix1 que 9 8 sleep - 0 101310 kernel ix1 que 10 8 sleep - 0 101311 kernel ix1 que 11 8 sleep - 0 101312 kernel ix1 linkq 1 8 sleep - 12 100064 intr irq270: ix0:que 6 8 run - 12 100066 intr irq271: ix0:que 7 8 run - 12 100068 intr irq272: ix0:que 8 8 run - 12 100070 intr irq273: ix0:link 5 8 wait - 12 100072 intr irq274: ix1:que 9 8 wait - 12 100074 intr irq275: ix1:que 10 8 wait - 12 100076 intr irq276: ix1:que 11 8 run - 12 100078 intr irq277: ix1:link 9 8 wait - # cpuset -g -t 101305 tid 101305 mask: 6 # cpuset -g -t 101306 tid 101306 mask: 7 # cpuset -g -t 101307 tid 101307 mask: 8 # cpuset -g -t 101309 tid 101309 mask: 9 # cpuset -g -t 101310 tid 101310 mask: 10 # cpuset -g -t 101311 tid 101311 mask: 11 # cpuset -g -t 100064 tid 100064 mask: 6 # cpuset -g -t 100066 tid 100066 mask: 7 # cpuset -g -t 100068 tid 100068 mask: 8 # cpuset -g -t 100072 tid 100072 mask: 9 # cpuset -g -t 100074 tid 100074 mask: 10 # cpuset -g -t 100076 tid 100076 mask: 11 Or you talk abot somewere different? > (I mean, pmc could also be broken...) Why? > -a > > > On 31 March 2015 at 15:59, Slawa Olhovchenkov wrote: > > On Sat, Mar 28, 2015 at 12:33:52PM -0700, Adrian Chadd wrote: > > > >> That's done deferred by the bus interrupt wiring. That's something > >> John's been looking into as part of the general NUMA work (and I'm > >> trying to debug right now, on dual-socket boxes with ixgbe. :-) > >> > >> Look at bus_bind_intr() and the twisty path to intr_event_bind(), then > >> x86/x86/intr_machdep.c:intr_assign_cpu(), then intr_shuffle_cpus() at > >> boot, versus what happens via calls to pic_assign_cpu to setup the > >> wiring. > > > > I am do simple, ugle hack ixgbe driver for let start cpu binding. > > I am still see ixgbe in pmc output. > > > > What may be wrong? > > What may be miss? > > > > ===== > > static int ixgbe_start_cpu = 0; > > TUNABLE_INT("hw.ix.start_cpu", &ixgbe_start_cpu); > > SYSCTL_INT(_hw_ix, OID_AUTO, start_cpu, CTLFLAG_RDTUN, &ixgbe_start_cpu, 0, > > "Start CPU for next IRQ binding"); > > > > [...] > > if (adapter->num_queues > 1) > > bus_bind_intr(dev, que->res, i+ixgbe_start_cpu); > > > > #ifndef IXGBE_LEGACY_TX > > TASK_INIT(&txr->txq_task, 0, ixgbe_deferred_mq_start, txr); > > #endif > > TASK_INIT(&que->que_task, 0, ixgbe_handle_que, que); > > que->tq = taskqueue_create_fast("ixgbe_que", M_NOWAIT, > > taskqueue_thread_enqueue, &que->tq); > > taskqueue_start_threads(&que->tq, 1, PI_NET, "%s que", > > device_get_nameunit(adapter->dev)); > > } > > ixgbe_start_cpu += adapter->num_queues; > > From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 06:57:28 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B24E4C64 for ; Wed, 1 Apr 2015 06:57:28 +0000 (UTC) Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6FA74AED for ; Wed, 1 Apr 2015 06:57:28 +0000 (UTC) Received: by iedm5 with SMTP id m5so35735815ied.3 for ; Tue, 31 Mar 2015 23:57:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ehJCC6XcMSYfwI/OowsFT8ZbYKPSp2ibl82eW979Y44=; b=RwlfBvUhVDo6n3i7SUHhEkjbsw7vBINhIjgEg3S8qUIUVPOAhGrhoMzkMkBEn29J2C G0t5ife6Svq6sqtBXfLIw1cb59o9F1m3cZQeuRNow4iQeTNCg8xYyhFXhTqgMr1IlEEf dnDuOuC2vdLKzLh/WyxyBsdDAubbxbEGRAWBPTVddya2wtGGArKssrqjZSQlytjmE4kh 8AtGcSeSe4rL+1OOqRmgCB5TRr6KRGwOsqZsGgcQDAkK5R3vhZO4ITC36WfwqB6vZhDS SITl5QeUydbSDZpe0hTchO4kqVMToWfjda7pwgicArWjNiGz0KDVuAtzoKgUzqVlk5DR KYGQ== MIME-Version: 1.0 X-Received: by 10.107.155.13 with SMTP id d13mr61064556ioe.29.1427871447834; Tue, 31 Mar 2015 23:57:27 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Tue, 31 Mar 2015 23:57:27 -0700 (PDT) In-Reply-To: <20150401062259.GD23643@zxy.spb.ru> References: <20150328154031.GA23643@zxy.spb.ru> <20150328181026.GB23643@zxy.spb.ru> <20150328183147.GC23643@zxy.spb.ru> <20150328192505.GD23643@zxy.spb.ru> <20150331225947.GC23643@zxy.spb.ru> <20150401062259.GD23643@zxy.spb.ru> Date: Tue, 31 Mar 2015 23:57:27 -0700 X-Google-Sender-Auth: rrej2uMZ78AJwlJqb9nj5qMibGQ Message-ID: Subject: Re: irq cpu binding From: Adrian Chadd To: Slawa Olhovchenkov Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 06:57:28 -0000 I'll go take a look over the weekend at the non-RSS case. It shouldn't be scheduling on the wrong CPU(s) if you've pinned things. -a From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 07:00:13 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 11A3CDF5 for ; Wed, 1 Apr 2015 07:00:13 +0000 (UTC) Received: from mail.embedded-brains.de (host-82-135-62-35.customer.m-online.net [82.135.62.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AEF5AB56 for ; Wed, 1 Apr 2015 07:00:12 +0000 (UTC) Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id 77B762A194E for ; Wed, 1 Apr 2015 08:51:01 +0200 (CEST) Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id H3qJW2FAdZ9P for ; Wed, 1 Apr 2015 08:50:59 +0200 (CEST) Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id 40FA82A1938 for ; Wed, 1 Apr 2015 08:50:59 +0200 (CEST) X-Virus-Scanned: amavisd-new at zimbra.eb.localhost Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id g7ESbMtt4F50 for ; Wed, 1 Apr 2015 08:50:59 +0200 (CEST) Received: from [192.168.96.129] (unknown [192.168.96.129]) by mail.embedded-brains.de (Postfix) with ESMTPSA id 21D452A192E for ; Wed, 1 Apr 2015 08:50:59 +0200 (CEST) Message-ID: <551B9531.4040902@embedded-brains.de> Date: Wed, 01 Apr 2015 08:50:25 +0200 From: Sebastian Huber User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Uptime starts with one second Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 07:00:13 -0000 Hello, we port currently the FreeBSD timecounters to the RTEMS real-time=20 operating system. https://devel.rtems.org/ticket/2271 On FreeBSD the uptime starts with one second. On RTEMS the uptime starts=20 with zero seconds. I would like to preserve the existing behaviour and=20 it would be easy to adjust kern_tc.c accordingly. The problem is that we=20 also use the FreeBSD network stack and it uses for example=20 (microuptime()). Has a uptime value of zero a special meaning inside the=20 kernel so that we must not start at zero? --=20 Sebastian Huber, embedded brains GmbH Address : Dornierstr. 4, D-82178 Puchheim, Germany Phone : +49 89 189 47 41-16 Fax : +49 89 189 47 41-09 E-Mail : sebastian.huber@embedded-brains.de PGP : Public key available on request. Diese Nachricht ist keine gesch=C3=A4ftliche Mitteilung im Sinne des EHUG= . From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 07:32:33 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3BB9F52B for ; Wed, 1 Apr 2015 07:32:33 +0000 (UTC) Received: from mail-ie0-x22f.google.com (mail-ie0-x22f.google.com [IPv6:2607:f8b0:4001:c03::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EFE9FED4 for ; Wed, 1 Apr 2015 07:32:32 +0000 (UTC) Received: by iedfl3 with SMTP id fl3so40638974ied.1 for ; Wed, 01 Apr 2015 00:32:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dataix.net; s=rsa; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=LDv0R8OGXErn6vQj70Qr64SeeRUdO6Ca6c5v38yJvFc=; b=X+nlBQ/OoamdkstLyb5B5PTuRn4YOd7RIdrssTKPQG6fSUZneWBvJRqd5oq5h4Q+Hi L/3YoVClmtX08bA+oQG2huxkGvPm2nmiqybHbrH339XCqFklA9sKBzNcp3qbrD+0wiI2 F53wM+1+X/8G4owz5Xnlz2dSfwkkbQHnaahOY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=LDv0R8OGXErn6vQj70Qr64SeeRUdO6Ca6c5v38yJvFc=; b=NsD3wS+psO5oDxRTBK+99ZpbE/DOh3ler+7V2xHPOY0zw2owiMIgaams3uGEUEwj3Y cN7hjf8QFabUvR+29dOuXyruEiO9I9rGqcOqjoX+2nGDTlO6gO3CAgMpYngCLhGOQiI1 iSI6e/l/NFB9ZGSZ5xfQ/u3Dh/mGrk7ggXhciz+Sw4d1v5vUZA3i/EuOFgbzCYQKbA2J efXk1TRnqJ/heJw3ps1ih9nX0W099XlslSc+rLOpofk7FofkjqIbFjhikZtNKxwsLUkv 07cZMxeeooY8kffvsOFOrvFKd3I6oysM+FPAT1ygQ7A52XN/LxDLrVYYe4yiMcne6pID gsMA== X-Gm-Message-State: ALoCoQlcU0y5c5xdMZmiH0PuY4w29TktOv1OUYYmDsNRh05ZIpCWg6AbMfCzOLoSNV8Y5kiFmJkg X-Received: by 10.42.88.206 with SMTP id d14mr72371590icm.40.1427873552048; Wed, 01 Apr 2015 00:32:32 -0700 (PDT) Received: from sentient.dataix.local (107-133-113-194.lightspeed.milwwi.sbcglobal.net. [107.133.113.194]) by mx.google.com with ESMTPSA id m191sm761325ioe.23.2015.04.01.00.32.31 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 01 Apr 2015 00:32:31 -0700 (PDT) Subject: Re: Uptime starts with one second Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Content-Type: text/plain; charset="utf8"; X-Pgp-Agent: GPGMail 2.5b6 From: Jason Hellenthal In-Reply-To: <551B9531.4040902@embedded-brains.de> Date: Wed, 1 Apr 2015 02:32:30 -0500 Content-Transfer-Encoding: 8bit Message-Id: <516B61EB-173D-44D8-B199-44E915738EF3@dataix.net> References: <551B9531.4040902@embedded-brains.de> To: Sebastian Huber X-Mailer: Apple Mail (2.2070.6) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 07:32:33 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Just a brief exploration of the source and a quick test, I yield no seen negative results… as of yet but changes are very minimal to a current updated 10-STABLE and have not tracked further if more may be needed. Index: sys/kern/kern_tc.c =================================================================== - --- sys/kern/kern_tc.c (revision 280928) +++ sys/kern/kern_tc.c (working copy) @@ -105,7 +105,7 @@ int tc_min_ticktock_freq = 1; volatile time_t time_second = 1; - -volatile time_t time_uptime = 1; +volatile time_t time_uptime = 0; struct bintime boottimebin; struct timeval boottime; On Apr 1, 2015, at 01:50, Sebastian Huber wrote: Hello, we port currently the FreeBSD timecounters to the RTEMS real-time operating system. https://devel.rtems.org/ticket/2271 On FreeBSD the uptime starts with one second. On RTEMS the uptime starts with zero seconds. I would like to preserve the existing behaviour and it would be easy to adjust kern_tc.c accordingly. The problem is that we also use the FreeBSD network stack and it uses for example (microuptime()). Has a uptime value of zero a special meaning inside the kernel so that we must not start at zero? - -- Sebastian Huber, embedded brains GmbH Address : Dornierstr. 4, D-82178 Puchheim, Germany Phone : +49 89 189 47 41-16 Fax : +49 89 189 47 41-09 E-Mail : sebastian.huber@embedded-brains.de PGP : Public key available on request. Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG. _______________________________________________ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" - -- Jason Hellenthal Mobile: +1 (616) 953-0176 jhellenthal@DataIX.net JJH48-ARIN -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJVG58OAAoJEDLu+wRc4KcI+CUIAMZnulwHBIg497IVyI+wK47Y Hz3eaebuI0IxVadfFcRZMXgGNN30bSVkxN9BVQzcJ9FOOAYb1O3AyJNYWLJw1dA6 ilyF6jCV+uOtuXfBdxch/3ZzSHP4wtMO1JzL2YgG0KzQW71DsXnFlXE3mNveFQx8 QNHpsHNOMPdC4pyq/Ew6ltiBELzIa7LJqcPdnjh4cSzxVnVlihjta41FhbDh4+bO 7m087eXH8AuRUdK5sGOBRB26hgOB1iJhis2YwFucdlAytUfE26eFDJtbS8NOMSOo fiViLMcE1nsEXIdorg7o5vRIUzakFGfcqJlXxdmAGUGZ52B6E2jaDTN/0wnGVh8= =xYyK -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 09:36:38 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14BCDB1B for ; Wed, 1 Apr 2015 09:36:38 +0000 (UTC) Received: from puchar.net (puchar.net [188.252.31.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8EFAAEA7 for ; Wed, 1 Apr 2015 09:36:37 +0000 (UTC) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.14.9/8.14.9) with ESMTP id t319aQsa040292 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Wed, 1 Apr 2015 11:36:27 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from laptop.wojtek.intra (localhost [127.0.0.1]) by laptop.wojtek.intra (8.14.9/8.14.9) with ESMTP id t319aPqG001492 for ; Wed, 1 Apr 2015 11:36:25 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from localhost (wojtek@localhost) by laptop.wojtek.intra (8.14.9/8.14.9/Submit) with ESMTP id t319aKWO001489 for ; Wed, 1 Apr 2015 11:36:20 +0200 (CEST) (envelope-from wojtek@puchar.net) X-Authentication-Warning: laptop.wojtek.intra: wojtek owned process doing -bs Date: Wed, 1 Apr 2015 11:36:19 +0200 (CEST) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: freebsd-hackers@freebsd.org Subject: mess with syslogd Message-ID: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Wed, 01 Apr 2015 11:36:27 +0200 (CEST) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 09:36:38 -0000 no idea how to debug a problem with syslogd. please help i use syslogd to log messages from multiple other unix machines, now i wanted to add logging from windows server (with evtsys program). if i run syslogd with syslogd_enable="YES" # Run syslog daemon (or NO). syslogd_flags="-v -4 -8 -b 10.100.100.1" it logs messages fine from windows server as well as others. if i run it as syslogd_flags="-v -4 -8 -b 10.100.100.1 -a 10.100.0.0/16" it logs messages fine from everything except windows servers, WHICH ARE IN 10.100.0.0/16 network. Now i just use firewall rules to block logging from unwanted places, but no idea why just using -a blocks logs from windows/evtsys any idea? From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 10:05:11 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8EF472B9 for ; Wed, 1 Apr 2015 10:05:11 +0000 (UTC) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "ca.infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 370F7236 for ; Wed, 1 Apr 2015 10:05:11 +0000 (UTC) Received: from ox-dell39.ox.adestra.com (no-reverse-dns.metronet-uk.com [85.199.232.226] (may be forged)) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.15.1/8.15.1) with ESMTPSA id t31A4uVW001656 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Wed, 1 Apr 2015 11:04:57 +0100 (BST) (envelope-from matthew@freebsd.org) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none header.from=freebsd.org DKIM-Filter: OpenDKIM Filter v2.9.2 smtp.infracaninophile.co.uk t31A4uVW001656 Authentication-Results: smtp.infracaninophile.co.uk/t31A4uVW001656; dkim=none reason="no signature"; dkim-adsp=none; dkim-atps=neutral X-Authentication-Warning: lucid-nonsense.infracaninophile.co.uk: Host no-reverse-dns.metronet-uk.com [85.199.232.226] (may be forged) claimed to be ox-dell39.ox.adestra.com Message-ID: <551BC2C8.8020806@freebsd.org> Date: Wed, 01 Apr 2015 11:04:56 +0100 From: Matthew Seaman User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: mess with syslogd References: In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="mfxsutj3NW9S6UgEfpwi352lTGuATPCmm" X-Virus-Scanned: clamav-milter 0.98.6 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.7 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lucid-nonsense.infracaninophile.co.uk X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 10:05:11 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --mfxsutj3NW9S6UgEfpwi352lTGuATPCmm Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 04/01/15 10:36, Wojciech Puchar wrote: > no idea how to debug a problem with syslogd. please help >=20 > i use syslogd to log messages from multiple other unix machines, now i > wanted to add logging from windows server (with evtsys program). >=20 > if i run syslogd with >=20 > syslogd_enable=3D"YES" # Run syslog daemon (or NO). > syslogd_flags=3D"-v -4 -8 -b 10.100.100.1" >=20 >=20 > it logs messages fine from windows server as well as others. >=20 >=20 > if i run it as >=20 > syslogd_flags=3D"-v -4 -8 -b 10.100.100.1 -a 10.100.0.0/16" >=20 > it logs messages fine from everything except windows servers, WHICH ARE= > IN 10.100.0.0/16 network. >=20 > Now i just use firewall rules to block logging from unwanted places, bu= t > no idea why just using -a blocks logs from windows/evtsys >=20 > any idea? >=20 You're implicitly telling syslogd what port numbers to accept on the sending side. The default is only to allow sending from port 514. Instead, try: syslogd_flags=3D"-v -4 -8 -b 10.100.100.1 -a 10.100.0.0/16:*" In theory you should be able to limit to only accepting packets sent from port 514 but I've found various different devices may use different ports. Looking at: # tcpdump -i em0 -A host 10.100.100.1 and port 514 should show what your systems are actually using. Cheers, Matthew --mfxsutj3NW9S6UgEfpwi352lTGuATPCmm Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJVG8LIXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQxOUYxNTRFQ0JGMTEyRTUwNTQ0RTNGMzAw MDUxM0YxMEUwQTlFNEU3AAoJEABRPxDgqeTnXnwQAJhdeT6GAJWssi4OZxsLLWCd blZHEUcOW0rnYFq1cle5LpDWlyUOZ2t/USRhk+G2O4gk1ceXjywmByCGpykxx9d9 W+zaYyZwqHD4m22hveYH/uRV/IT6qMEYjI8cOeHBA5zoF8wpmi4tKWDP7RdJ0efQ tki8J07xWXvKSCWi4H4xvOvvTL2ZUwXec+DVJEuXqJTK3Hrf+vaJdfgR/pfsESIR RSgD6HFsRvhKjJX7YYZBMN/PjlGVLjMWDN7YlelflcDhuNxcQmmw9zzFzzPpVl19 fL+h86cCfKH2lgmQsCI7bFVKCWTT/kFaIv/9zQMfM119Yuga1g7BdRC8y5N7vXTI NosZNgL54Nv1rOnm8G/+vtW16PmyntN9hta9rQQLChGFHJed5XlngVA4Li1dxDNB 66lu6uxQe5J66+Thi7QkCdjtkx9PnJTZxUtwRyYJT7/keAt6frUBDQhJwB7B9hwM OXt7kVCKT51GtvzJCvHGX/YYr2jbMk/JmVQAANUy3t50FFDMFzPRvaMmfRFEEZJi aA8123udoVULtFaq2I7LiBQwMpaDw0cjbNO0T1333XG9veEtNeUPg85dGwcFEc2v wv+E5BERRsCqzhNwQguQeIRfwReSOP3CtqKXrbGoNJfCrzWXj553JTHl39cRZv5J vgvVnMFzbWoOk1Ak450s =fEuG -----END PGP SIGNATURE----- --mfxsutj3NW9S6UgEfpwi352lTGuATPCmm-- From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 10:16:34 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D8F2B907; Wed, 1 Apr 2015 10:16:34 +0000 (UTC) Received: from mail-wg0-x233.google.com (mail-wg0-x233.google.com [IPv6:2a00:1450:400c:c00::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7EE783DF; Wed, 1 Apr 2015 10:16:34 +0000 (UTC) Received: by wgdm6 with SMTP id m6so47897868wgd.2; Wed, 01 Apr 2015 03:16:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :content-type:content-transfer-encoding; bh=ed5Uz9Ct7ankujhrn3o/I9MLizxCg+7SzlI9XCE8/EE=; b=eBOPpvphcM7gQ/KKcJ6l0lYDBuF8Sa7fvbQF5wIuyPHjVZhRDvlhygb0b51p0UmHjy 4t9kQ4hf175usJMOA/fw+ZqAB7Bdk4ZoVQpx0EZobNIPHxbkM3w3rNTOlG1nX0hilepW /tbX9P8+cxY3OZrw94mbFaTfMLfVZGN/15FmYTMNFMZFyMrENGIt5zzywWQXF8x4z4Uc +P0ok2Y9fBVWLe0BkQrX/ljOwTVoI6NTecnX14EvjmPYNL+okctJEZyh6LRadTXiDS7q sFM4F3WedlGZu3CUZvRyYs/MmoPelNBRXc1nt8xVwn3STZPM40Uj/PZFakAGGbjvtkl6 CycA== X-Received: by 10.180.87.66 with SMTP id v2mr13088738wiz.51.1427883392751; Wed, 01 Apr 2015 03:16:32 -0700 (PDT) Received: from [192.168.43.73] ([89.204.135.212]) by mx.google.com with ESMTPSA id r3sm1998719wjw.7.2015.04.01.03.16.31 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Apr 2015 03:16:31 -0700 (PDT) Message-ID: <551BC57D.5070101@gmail.com> Date: Wed, 01 Apr 2015 12:16:29 +0200 From: Tobias Oberstein User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: NVMe performance 4x slower than expected Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: jimharris@freebsd.org, Michael Fuckner , kib@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 10:16:34 -0000 Hi, I am testing performance of a NVMe device (Intel P3700) using FIO at the block device level and get 4x slower performance than expected: 4kB Random Read Intel Datasheet FIO Measurement Match P3700 450,000 107,092 24% DC S3700 75,000 67,186 90% The 2nd line are results for an Intel DC S3700 for comparison (with this device, I do see the performance expected, but not for the P3700). Hardware: - 4 sockets, 48 core x86-64, 3TB RAM - 8 x Intel P3700 2TB - 12 x Intel DC S3700 800GB (via LSI HBAs) Software: FreeBSD 11 Current with patches (DMAR and ZFS patches, otherwise the box doesn't boot at all .. because of 3TB RAM and the amount of periphery). Complete info and test logs are here: https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/perftests.md Right now I am running Linux on the box (openSUSE 13.2). Using the exact same FIO control file, the values for the DC S3700 are very close to FreeBSD, but the values for the P3700 are much higher: https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/perftests.md#more-numbers-linux I am looking for tuning hints or general advice for FreeBSD and NVMe. I would like to go with FreeBSD (a major aspect is ZFS), but the performance issues with NVMe might be a deal breaker. Cheers, /Tobias From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 11:26:49 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2F181D98 for ; Wed, 1 Apr 2015 11:26:49 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 01A45D3D for ; Wed, 1 Apr 2015 11:26:48 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-255-201.lns20.per4.internode.on.net [121.45.255.201]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id t31BQYIR024417 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 1 Apr 2015 04:26:40 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <551BD5E5.6050404@freebsd.org> Date: Wed, 01 Apr 2015 19:26:29 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Mateusz Guzik , Oliver Pinter , Yue Chen , Benjamin Kaduk , "freebsd-hackers@freebsd.org" , HardenedBSD Core , PaX Team Subject: Re: How to traverse kernel threads? References: <20150321220246.GE14650@dft-labs.eu> <20150321232622.GF14650@dft-labs.eu> <20150327194920.GB18158@dft-labs.eu> <20150330173358.GB9095@dft-labs.eu> In-Reply-To: <20150330173358.GB9095@dft-labs.eu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 11:26:49 -0000 On 3/31/15 1:33 AM, Mateusz Guzik wrote: > > I do not believe this can be implemented reliably without some serious > tinkering around the kernel. Surely I'm not a live patching expert (not > any other kind of expert), but hear my out. > > It seems proposed approach is to move the kernel around and then updated > all relevant pointers in all pages. > > I do not believe this can be done reliably without corrupting data, > unless a major surgery is performed on the kernel. > > Consider: > struct meh { > func_t *m_func; > size_t m_len; > data *m_buf; > }; > > Here we have 'm_func' which needs to be updated, but we don't know if > that's the only pointer so we have to scan the entire struct. But m_buf > can have data which looks like kernel pointers (i.e. matches some kernel > funcs) and how will you know not to modify it? What if this is some sort > of a hack and in fact you *should* modify it? two examples.. in the VFS code it is not unknown for a local stack variable to be initialised to a method pointer and used separately from the structure.. methodp = struct foo->method; { some code } result = methodp(args); also things like network mbufs can (AND DO!) have functions pointers in them. They can contain pointers to their own 'free' routines. so every packet in transit needs to be scanned as well. one could say that all this difficulty will make your success even more amazing :-) > > CTF or some other solutions like that don't help if you just traverse > all allocated buffers since you have no idea what the object you found > really is. > > So, assuming this has to be done in runtime, the only somewhat workable > solution I see would require leaving function entry points at contant > addresses and only insert jumps to new locations. > > This would be a performance hit and would leave a lot of data at > constant addresses, defeating the point to some extent. > > Also runtime relocation of everything definitely has a lot of unknown > unknowns, so all in all I would say this is a non-starter. > > One could consider a different approach where kernel data is randomly > shuffled around with some granularity and relevant symbol relocated > prior to booting. > > This should provide unique enough layout, which paired with big > likelyhood of a kernel panic on first bad exploit attempt may be an > acceptable solution. > > But then the kernel may still have enough info leaks for this to not > matter, so I would not be so eager to implement it. > > That said, what prompted this entire effort? Is there an operating > system which got this to work or what? > > From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 07:40:34 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7595F77A for ; Wed, 1 Apr 2015 07:40:34 +0000 (UTC) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 361A3F29 for ; Wed, 1 Apr 2015 07:40:33 +0000 (UTC) Received: from critter.freebsd.dk (unknown [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id D1BE53B8A2; Wed, 1 Apr 2015 07:40:26 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.9/8.14.9) with ESMTP id t317eOqA042744; Wed, 1 Apr 2015 07:40:25 GMT (envelope-from phk@phk.freebsd.dk) To: Jason Hellenthal Subject: Re: Uptime starts with one second In-reply-to: <516B61EB-173D-44D8-B199-44E915738EF3@dataix.net> From: "Poul-Henning Kamp" References: <551B9531.4040902@embedded-brains.de> <516B61EB-173D-44D8-B199-44E915738EF3@dataix.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <42742.1427874024.1@critter.freebsd.dk> Date: Wed, 01 Apr 2015 07:40:24 +0000 Message-ID: <42743.1427874024@critter.freebsd.dk> X-Mailman-Approved-At: Wed, 01 Apr 2015 11:27:53 +0000 Cc: Sebastian Huber , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 07:40:34 -0000 -------- As I recall (author of timecounters here...) the ARP code was the reason we had to start uptime at one. Can't remember what it did if uptime was zero, but probably some variant of "not working". -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 12:34:36 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 95A845DC for ; Wed, 1 Apr 2015 12:34:36 +0000 (UTC) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailuogwprd51.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 434BD781 for ; Wed, 1 Apr 2015 12:34:35 +0000 (UTC) Received: from maildlpprd56.lss.emc.com (maildlpprd56.lss.emc.com [10.106.48.160]) by mailuogwprd53.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id t31CYRuw028381 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 1 Apr 2015 08:34:32 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd53.lss.emc.com t31CYRuw028381 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1427891674; bh=6cYeZs2bleX1ouk+4UZHbPW+p0U=; h=From:To:CC:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:Content-Transfer-Encoding:MIME-Version; b=m7n2NDBAkQ5bTxkp+v4ypLWUnFGCaxyjVyOrXVx6pwUTmK5bU3/d4kNfxXiRntvSp F9IbPGIuL6MjYIhMudyAaBznbvuAd5tIfxv3bpQnZ7PYKDWJ/jas3M9vBMhb7Al6aO iVPgR6w3ggFNoxKKcc7ZyovYRIZbQD85OUEbR1tg= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd53.lss.emc.com t31CYRuw028381 Received: from mailusrhubprd51.lss.emc.com (mailusrhubprd51.lss.emc.com [10.106.48.24]) by maildlpprd56.lss.emc.com (RSA Interceptor); Wed, 1 Apr 2015 08:33:50 -0400 Received: from mxhub09.corp.emc.com (mxhub09.corp.emc.com [10.254.92.104]) by mailusrhubprd51.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id t31CYARp027043 (version=TLSv1 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 1 Apr 2015 08:34:11 -0400 Received: from MXHUB101.corp.emc.com (10.253.50.15) by mxhub09.corp.emc.com (10.254.92.104) with Microsoft SMTP Server (TLS) id 8.3.327.1; Wed, 1 Apr 2015 08:32:49 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.202]) by MXHUB101.corp.emc.com ([::1]) with mapi id 14.03.0224.002; Wed, 1 Apr 2015 08:34:10 -0400 From: "Meyer, Conrad" To: Garrett Cooper , "Gumpula, Suresh" Subject: RE: BSD 8.1 and 9.1 memory increase Thread-Topic: BSD 8.1 and 9.1 memory increase Thread-Index: AQHQYN1kO9bXTk+QcE+8E2hd8pdV+Z0hK5sAgBZxt4CAADccAIAAWDtE Date: Wed, 1 Apr 2015 12:34:09 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.13.36.127] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd51.lss.emc.com X-RSA-Classifications: public Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 12:34:36 -0000 > On Mar 31, 2015, at 17:54, Gumpula, Suresh wr= ote:=0A= >=0A= > Still trying to find out the reason for more memory foot print on 9.1=0A= > compared to 8.1 .=0A= > Does some thing like clustering changes in page fault handling cause=0A= > memory foot print ?=0A= > https://svnweb.freebsd.org/base?view=3Drevision&revision=3D235876=0A= >=0A= > Copying Alan Cox , if could throw some inputs on this.=0A= =0A= Superpages and how FreeBSD does its best to put runtime libraries in superp= age-able comes to mind..=0A= =0A= The VMEM for libraries is what caught us off guard last year when dealing w= ith applications -- more libraries =3D=3D greater footprint past either 8.0= or 9.0 because of changes to VM/rtld.=0A= =0A= Conrad Meyer had a change out to reduce the footprint for libraries, but it= was racy/incomplete unfortunately :/..=0A= =0A= Hope that maybe helps...=0A= =0A= -----------------------------------=0A= =0A= Right. So the linker and RTLD map each binary segment with 2MB virtual page= s, because that way you only need one mapping / TLB entry per segment (or a= t least, up to 2MB... most libraries are much smaller than this). This is a= performance optimization. The discussion around unmapping unused portions = of the 2MB range can be found here: https://reviews.freebsd.org/D1263 .=0A= =0A= To summarize: larger-than-necessary superpage mappings affect only vmem acc= ounting; actually use less resources (PTE's and any additional per-PTE vm a= ccounting) than 4k pages; and use fewer TLB entries. Unmapping the unused p= ortions is useless even if you get it right.=0A= =0A= Are you actually seeing greater memory footprint, or just greater vmem foot= print? I don't actually use FreeBSD8 or 9.=0A= =0A= Cheers,=0A= Conrad= From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 14:43:05 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 347932D9; Wed, 1 Apr 2015 14:43:05 +0000 (UTC) Received: from mail-ob0-x22e.google.com (mail-ob0-x22e.google.com [IPv6:2607:f8b0:4003:c01::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E3E05859; Wed, 1 Apr 2015 14:43:04 +0000 (UTC) Received: by obvd1 with SMTP id d1so82780002obv.0; Wed, 01 Apr 2015 07:43:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=TisC5IgROSeSNWtZIjwsQ83UedHNwjPEoekuaU+qQ1k=; b=r0Igkcta5XEa3CwJcBhb5W6g7REWoZV5d72vDKnv3gBJj6A40EjtHnkEqXDQJvHHHZ C0tRRxMIp9f7VYx8DU31cl2GLvpjcJyhDrZaxrHoie+zFmFQuS6fZqke0fCHibeGM9eF N7WPGARbOy3oG/HLQsKXKKhCmnPQvSPKQNJlmwnbXr5w/3UE6f8QwMtp2y8SbHm8Y/Mt Hyy6ZZmYJaRw3IEz49FXn2De6aRJ1/0vBzdsa+WWEwQ5TxZ9S+/pAXYKhN36mWVwGg6d uV6Tj+RKiHc6sTZIVNZB8Tt4Vbw0NW3eM4D4St1tcqWXdCaOyy+rgL0WX3iar7hKlqJw Mupg== MIME-Version: 1.0 X-Received: by 10.182.241.197 with SMTP id wk5mr11706681obc.0.1427899371005; Wed, 01 Apr 2015 07:42:51 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.202.215.7 with HTTP; Wed, 1 Apr 2015 07:42:50 -0700 (PDT) In-Reply-To: <551BC57D.5070101@gmail.com> References: <551BC57D.5070101@gmail.com> Date: Wed, 1 Apr 2015 08:42:50 -0600 X-Google-Sender-Auth: X7Cma5wLHbeOkUkQxetYaXHX_1I Message-ID: Subject: Re: NVMe performance 4x slower than expected From: Alan Somers To: Tobias Oberstein Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" , Michael Fuckner , kib@freebsd.org, jimharris@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 14:43:05 -0000 On Wed, Apr 1, 2015 at 4:16 AM, Tobias Oberstein wrote: > Hi, > > I am testing performance of a NVMe device (Intel P3700) using FIO at the > block device level and get 4x slower performance than expected: > > 4kB Random Read > > Intel Datasheet FIO Measurement Match > P3700 450,000 107,092 24% > DC S3700 75,000 67,186 90% > > The 2nd line are results for an Intel DC S3700 for comparison (with this > device, I do see the performance expected, but not for the P3700). > > Hardware: > > - 4 sockets, 48 core x86-64, 3TB RAM > - 8 x Intel P3700 2TB > - 12 x Intel DC S3700 800GB (via LSI HBAs) > > Software: > > FreeBSD 11 Current with patches (DMAR and ZFS patches, otherwise the box > doesn't boot at all .. because of 3TB RAM and the amount of periphery). Do you still have WITNESS and INVARIANTS turned on in your kernel config? They're turned on by default for Current, but they do have some performance impact. To turn them off, just build a GENERIC-NODEBUG kernel . > > Complete info and test logs are here: > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/perftests.md > > Right now I am running Linux on the box (openSUSE 13.2). Using the exact > same FIO control file, the values for the DC S3700 are very close to > FreeBSD, but the values for the P3700 are much higher: > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/perftests.md#more-numbers-linux > > I am looking for tuning hints or general advice for FreeBSD and NVMe. > > I would like to go with FreeBSD (a major aspect is ZFS), but the performance > issues with NVMe might be a deal breaker. > > Cheers, > /Tobias > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 16:06:08 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4CE3675; Wed, 1 Apr 2015 16:06:08 +0000 (UTC) Received: from mx141.netapp.com (mx141.netapp.com [216.240.21.12]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mx141.netapp.com", Issuer "VeriSign Class 3 International Server CA - G3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8CA071F3; Wed, 1 Apr 2015 16:06:08 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.11,504,1422950400"; d="scan'208";a="34000524" Received: from hioexcmbx08-prd.hq.netapp.com ([10.122.105.41]) by mx141-out.netapp.com with ESMTP; 01 Apr 2015 09:04:06 -0700 Received: from HIOEXCMBX03-PRD.hq.netapp.com (10.122.105.36) by hioexcmbx08-prd.hq.netapp.com (10.122.105.41) with Microsoft SMTP Server (TLS) id 15.0.995.29; Wed, 1 Apr 2015 09:04:06 -0700 Received: from HIOEXCMBX03-PRD.hq.netapp.com ([::1]) by hioexcmbx03-prd.hq.netapp.com ([fe80::4c9:7f9e:4b9f:2c9c%21]) with mapi id 15.00.0995.031; Wed, 1 Apr 2015 09:04:05 -0700 From: "Gumpula, Suresh" To: "Meyer, Conrad" , Garrett Cooper , "Gumpula, Suresh" Subject: Re: BSD 8.1 and 9.1 memory increase Thread-Topic: BSD 8.1 and 9.1 memory increase Thread-Index: AQHQYN1kO9bXTk+QcE+8E2hd8pdV+Z0hK5sAgBZxt4CAAGlnAIAAnTGA///3mQA= Date: Wed, 1 Apr 2015 16:04:05 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.7.141117 x-originating-ip: [10.122.56.79] Content-Type: text/plain; charset="Windows-1252" Content-ID: <2662754A4E985D47A2B819C8E63381B4@hq.netapp.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "alc@freebsd.org" , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 16:06:08 -0000 Thanks a lot for Conrad/Cooper. I am seeing more physical memory footprint. Looking at the top command stats on an idle machine, there is an increase in active/inactive pages. Then I looked at one of the huge applications we run(mgwd) , there is an increase in ~15M vmem and ~35M of RSS. Closely looking at its mapping entries With procstat -v , I see that both resident(18209 to 23328) and private resident(22827 to 26760) pages gone up for one of the libraries. And similarly on boost atomic library too there is an increase in resident pages. This is observed for most of applications. BSD 8.1 : last pid: 5116; load averages: 5.22, 4.29, 2.34 up 0+00:10:15 17:34:00 352 processes: 1 running, 350 sleeping, 1 zombie CPU: 0.2% user, 0.0% nice, 4.5% system, 1.7% interrupt, 93.5% idle Mem: 297M Active, 648M Inact, 139M Wired, 6948K Cache, 7520K Buf, 1862M Free Swap: 1536M Total, 1536M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 5112 diag 1 96 0 19012K 5668K CPU0 0 0:00 2.00% top 2219 root 68 96 0 859M 226M ucond 1 0:24 0.00% mgwd % sudo procstat -v `pgrep mgwd` PID START END PRT RES PRES REF SHD FL TP PATH 2213 0x800a46000 0x807b41000 r-x 18209 22827 2 1 CN vn /usr/lib/libmgwd.so.1 BSD 9.1 : last pid: 5344; load averages: 5.17, 4.47, 2.79 up 0+00:26:12 17:22:57 39 processes: 1 running, 37 sleeping, 1 zombie CPU: 0.2% user, 0.0% nice, 2.2% system, 0.6% interrupt, 97.0% idle Mem: 338M Active, 669M Inact, 147M Wired, 392K Cache, 7488K Buf, 1799M Free Swap: 1536M Total, 1536M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 5328 diag 1 40 0 17028K 5204K CPU1 0 0:00 0.40% top 2158 root 68 40 0 874M 262M uwait 1 0:23 0.00% mgwd % sudo procstat -v `pgrep mgwd` PID START END PRT RES PRES REF SHD FL TP PATH 2158 0x800a1c000 0x807c87000 r-x 23328 26760 2 1 CN=8B vn /usr/lib/libmgwd.so.1 Thanks Suresh On 4/1/15, 8:34 AM, "Meyer, Conrad" wrote: >> On Mar 31, 2015, at 17:54, Gumpula, Suresh >>wrote: >> >> Still trying to find out the reason for more memory foot print on 9.1 >> compared to 8.1 . >> Does some thing like clustering changes in page fault handling cause >> memory foot print ? >> https://svnweb.freebsd.org/base?view=3Drevision&revision=3D235876 >> >> Copying Alan Cox , if could throw some inputs on this. > >Superpages and how FreeBSD does its best to put runtime libraries in >superpage-able comes to mind.. > >The VMEM for libraries is what caught us off guard last year when dealing >with applications -- more libraries =3D=3D greater footprint past either 8= .0 >or 9.0 because of changes to VM/rtld. > >Conrad Meyer had a change out to reduce the footprint for libraries, but >it was racy/incomplete unfortunately :/.. > >Hope that maybe helps... > >----------------------------------- > >Right. So the linker and RTLD map each binary segment with 2MB virtual >pages, because that way you only need one mapping / TLB entry per segment >(or at least, up to 2MB... most libraries are much smaller than this). >This is a performance optimization. The discussion around unmapping >unused portions of the 2MB range can be found here: >https://reviews.freebsd.org/D1263 . > >To summarize: larger-than-necessary superpage mappings affect only vmem >accounting; actually use less resources (PTE's and any additional per-PTE >vm accounting) than 4k pages; and use fewer TLB entries. Unmapping the >unused portions is useless even if you get it right. > >Are you actually seeing greater memory footprint, or just greater vmem >footprint? I don't actually use FreeBSD8 or 9. > >Cheers, >Conrad From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 16:55:45 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 13E93B09 for ; Wed, 1 Apr 2015 16:55:45 +0000 (UTC) Received: from mail-wg0-x22b.google.com (mail-wg0-x22b.google.com [IPv6:2a00:1450:400c:c00::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8FF29CA for ; Wed, 1 Apr 2015 16:55:44 +0000 (UTC) Received: by wgoe14 with SMTP id e14so59846284wgo.0 for ; Wed, 01 Apr 2015 09:55:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:from:date:message-id:subject:to:content-type; bh=jeqlEaXgQK/+CDz9vTihV3g2UoyERvuwN1OM6HnIa4Y=; b=TkaBcxteT+CbaYBZReqBzqYp1uxtkn2URMFN2cu4DmCrjwHFvus6fUr3nBeGQtbn4N 7NaBJInB8GPDacvs7w+oSzDZ05IlYyi6evpSa5sTq3mKm/zNIF8pDHKu8F82PI/bNCPc 0MyhCTn+YmWQ5+j/JPEsVDppnONS9Wr1is9Mo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to :content-type; bh=jeqlEaXgQK/+CDz9vTihV3g2UoyERvuwN1OM6HnIa4Y=; b=bHIyn3rYKU70lupH+RFNf8vPX2pV8X9BPg249m8ORBV1vTuFvcJJeUge5Sb5/ElGBa gJcIzTbOI30JlJ9PNl20OZ6uGhROqTXWUhgsPycKvGj6T8TN9MBYsoZf7TJjTwwnsdJ9 XwFefLX64hCFN7/ZKBft8vUEJ9Yg40g813bT2r9bH7fMFRf4nlcW8z7lRUpBArh7bnl8 PHwt23xETmss9rlHDImCW67mO+alp9xcuspt7N/MHSTlvzjWIPJ62fM7BTw2RBQIRXa9 QKPDC+our+DMczApgU+g94Ttt6T8ox4tiK3eh0mzdcp7P52ehjkPnsF6RC6PS6lziOr2 7HSg== X-Gm-Message-State: ALoCoQmx919KCnXfuSxRIuCYGvj8C37Y4hzlBtuQx0nEi1M6N3Aup+JE/VlkbGHNmXpUwR9L0KEs X-Received: by 10.180.77.40 with SMTP id p8mr16616050wiw.1.1427907343117; Wed, 01 Apr 2015 09:55:43 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.211.135 with HTTP; Wed, 1 Apr 2015 09:55:11 -0700 (PDT) From: Eitan Adler Date: Wed, 1 Apr 2015 09:55:11 -0700 Message-ID: Subject: Bazaaring the cathedral (Lowering the Barrier to Entry) To: FreeBSD Hackers , freebsd-current Current Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 16:55:45 -0000 Hi all, FreeBSD today has a significant problem of attracting new developers. The lack of new developers manifests itself in a dearth of driver support, inadequate documentation, and trailing third party packages. One of the key reasons for the lack of people is the high barrier of entry to joining the FreeBSD project. While every modern project uses git (usually hosted on github), FreeBSD uses self-hosted subversion. The use of git goes beyond just the choice of version control. It allows for workflows that FreeBSD can't even dream of. The linux kernel has no concept of a committer. Instead anyone can clone the git tree, build a kernel, and call themselves a Linux distribution. Our wiki and code review tools are in an even worse position than our repository. In order to contribute you need to send an email to clusteradm@ and hope they deem you worthy enough of access. The bug tracking system is in a better shape but isn't perfect: while any user can create an account, their privileges are limited: they can't help close out bad PRs, reclassify good ones, or do other generally useful work. To solve this problem I propose a simple solution: self-serve commit access. We create a web service on accounts.freebsd.org via which users can create themselves a freefall account. In addition to a freefall account, the identical username would be created for the wiki and phabricator, bugzilla, and any other service we might provide. This will allow us to quickly gain large number of contributors, who will be able to make better progress on our missing features, and rectify some of the drawbacks listed above. Today I hope we turn ourselves from the cathedral into a truly open and democratic project! -- Eitan Adler From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 17:29:07 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CFB09A0A; Wed, 1 Apr 2015 17:29:07 +0000 (UTC) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D802DAF; Wed, 1 Apr 2015 17:29:07 +0000 (UTC) Received: by lahf3 with SMTP id f3so41586381lah.2; Wed, 01 Apr 2015 10:29:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=4Rl+8yuxKvNtdD6G45a/kp5iYeKktLh6tKua40pyXfY=; b=rDDQ+KZ5mvaFSJQ8AAlP6UHoRFXPZFwrmcq6Jvvt/5rUcd1E/tgh1P06dYitOSWflf JKYfwViKjIsGfri6LD8yhPisVbEQkEnRyJeOartzd1B5Av8Ek1Rh+ByG6LIpCB+WUpc+ kZA5fF77kG97GOYfmXjRF1DW41o7dWKJXjsjkQHNTSXS8vLS2Ntx6cOQCtMsfSJ9BgAw LyUVv6yypVCpI6c/0M5lvTMDpMPQrK5vOtOtQK7Uw9njwyqZKjgjcaMJW/vmmlFKvD1S jbgJ2lOlc0mskYsunCZOhWpcmgzY7zbKTsqsAjgdgFzMCrOGLVBakixMPDwln6+LnTm9 ZxBQ== MIME-Version: 1.0 X-Received: by 10.112.8.76 with SMTP id p12mr33189488lba.29.1427909345423; Wed, 01 Apr 2015 10:29:05 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.108.168 with HTTP; Wed, 1 Apr 2015 10:29:05 -0700 (PDT) In-Reply-To: References: Date: Wed, 1 Apr 2015 10:29:05 -0700 X-Google-Sender-Auth: E3k1euLDb8bhsd4DB6j-pDs5vII Message-ID: Subject: Re: Bazaaring the cathedral (Lowering the Barrier to Entry) From: Craig Rodrigues To: Eitan Adler Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: FreeBSD Hackers , freebsd-current Current X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 17:29:07 -0000 On Wed, Apr 1, 2015 at 9:55 AM, Eitan Adler wrote: > > To solve this problem I propose a simple solution: self-serve commit > access. We create a web service on accounts.freebsd.org via which > users can create themselves a freefall account. In addition to a > freefall account, the identical username would be created for the wiki > and phabricator, bugzilla, and any other service we might provide. > I support the creation of accounts.freebsd.org. I suggest that we use PWM ( http://code.google.com/p/pwm/ ) as a web-based front-end to a back-end LDAP database. The FreeBSD cluster already has LDAP ( https://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/article.html#kerberos-ldap ) The FreeBSD cluster LDAP + Kerberos back-end infrastructure is something developed by clusteradm (most likely Peter Wemm). It works quite well. I succeeded in using the FreeBSD cluster LDAP system for Jenkins authentication, and it just worked like a champ. The FreeBSD cluster LDAP system just needs a better front-end that is more user friendly, and easier to manage. If you log into hub.freebsd.org and look at /etc/aliases, you will see that there are 12 people who receive clusteradm e-mails. My experience working with Jenkins is that only about 2-3 people actively do clusteradm work, and they are *way* overloaded. If we could have accounts.freebsd.org which streamlines a lot of the account creation and management, and works across many modern web applications, that would be super cool, and would hopefully reduce the load on clusteradm! -- Craig From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 18:47:36 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D3B93E11; Wed, 1 Apr 2015 18:47:36 +0000 (UTC) Received: from mx142.netapp.com (mx142.netapp.com [216.240.21.19]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mx142.netapp.com", Issuer "VeriSign Class 3 International Server CA - G3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D49D8F7; Wed, 1 Apr 2015 18:47:36 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.11,505,1422950400"; d="scan'208";a="32400205" Received: from hioexcmbx02-prd.hq.netapp.com ([10.122.105.35]) by mx142-out.netapp.com with ESMTP; 01 Apr 2015 11:42:35 -0700 Received: from HIOEXCMBX03-PRD.hq.netapp.com (10.122.105.36) by hioexcmbx02-prd.hq.netapp.com (10.122.105.35) with Microsoft SMTP Server (TLS) id 15.0.995.29; Wed, 1 Apr 2015 11:42:34 -0700 Received: from HIOEXCMBX03-PRD.hq.netapp.com ([::1]) by hioexcmbx03-prd.hq.netapp.com ([fe80::4c9:7f9e:4b9f:2c9c%21]) with mapi id 15.00.0995.031; Wed, 1 Apr 2015 11:42:34 -0700 From: "Gumpula, Suresh" To: "Gumpula, Suresh" , "Meyer, Conrad" , Garrett Cooper Subject: Re: BSD 8.1 and 9.1 memory increase Thread-Topic: BSD 8.1 and 9.1 memory increase Thread-Index: AQHQYN1kO9bXTk+QcE+8E2hd8pdV+Z0hK5sAgBZxt4CAAGlnAIAAnTGA///3mQCAACxGAA== Date: Wed, 1 Apr 2015 18:42:33 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.7.141117 x-originating-ip: [10.122.56.79] Content-Type: text/plain; charset="utf-8" Content-ID: <0BC4FDD847745C42ABCA4B1EC252D237@hq.netapp.com> Content-Transfer-Encoding: base64 MIME-Version: 1.0 Cc: "alc@freebsd.org" , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 18:47:37 -0000 SSBqdXN0IHRyaWVkIHRoZSBjaGFuZ2UgeW91IG1lbnRpb25lZCBvbiB1bm1hcHBpbmcgdW51c2Vk IGVudHJpZXMgLCBhbmQNCmFwcGVhcnMgdGhpcyBzYXZlcyBnb29kIGFtb3VudCBvZiByZWFsIG1l bW9yeS4gIE9uIG15IGlkbGUgbWFjaGluZSBpdA0Kc2hvd3MgfjIwTSArICByZWFsIG1lbW9yeSBz YXZpbmdzLiBTcGVjaWZpY2FsbHkgdGhlIGluYWN0aXZlL3dpcmVkIHBhZ2VzDQpkcm9wcGVkIGRv d24uDQpodHRwczovL3Jldmlld3MuZnJlZWJzZC5vcmcvRDEyNjMNCg0KQW5kIGFsc28gZHJhc3Rp YyBWU1Mgc2l6ZSBkb3duIGZvciBtb3N0IG9mIHRoZSBwcm9jZXNzZXMuICAgIElzIHRoaXMNCmlu Y29tcGxldGUgb3IgQ2FuIEkgdGFrZSB0aGlzIGNoYW5nZSAgPw0KDQpUaGFua3MNClN1cmVzaA0K DQoNCk9uIDQvMS8xNSwgMTI6MDQgUE0sICJHdW1wdWxhLCBTdXJlc2giIDxTdXJlc2guR3VtcHVs YUBuZXRhcHAuY29tPiB3cm90ZToNCg0KPlRoYW5rcyBhIGxvdCBmb3IgQ29ucmFkL0Nvb3Blci4N Cj5JIGFtIHNlZWluZyBtb3JlIHBoeXNpY2FsIG1lbW9yeSBmb290cHJpbnQuICAgTG9va2luZyBh dCB0aGUgdG9wIGNvbW1hbmQNCj5zdGF0cyBvbiBhbiBpZGxlIG1hY2hpbmUsIHRoZXJlIGlzIGFu IGluY3JlYXNlIGluIGFjdGl2ZS9pbmFjdGl2ZSBwYWdlcy4NCj5UaGVuIEkgbG9va2VkIGF0IG9u ZSBvZiB0aGUgaHVnZSBhcHBsaWNhdGlvbnMgd2UgcnVuKG1nd2QpICwgdGhlcmUgaXMgYW4NCj5p bmNyZWFzZSBpbiB+MTVNIHZtZW0gYW5kIH4zNU0gb2YgUlNTLiAgIENsb3NlbHkgbG9va2luZyBh dCBpdHMgbWFwcGluZw0KPmVudHJpZXMNCj5XaXRoIHByb2NzdGF0IC12ICwgSSBzZWUgdGhhdCAg Ym90aCByZXNpZGVudCgxODIwOSB0byAyMzMyOCkgYW5kIHByaXZhdGUNCj5yZXNpZGVudCgyMjgy NyB0byAyNjc2MCkgcGFnZXMgZ29uZSB1cCBmb3Igb25lIG9mIHRoZSBsaWJyYXJpZXMuDQo+QW5k IHNpbWlsYXJseSBvbiBib29zdCBhdG9taWMgbGlicmFyeSB0b28gdGhlcmUgaXMgYW4gaW5jcmVh c2UgaW4gcmVzaWRlbnQNCj5wYWdlcy4gIFRoaXMgaXMgb2JzZXJ2ZWQgZm9yIG1vc3Qgb2YgYXBw bGljYXRpb25zLg0KPg0KPkJTRCA4LjEgOg0KPg0KPmxhc3QgcGlkOiAgNTExNjsgIGxvYWQgYXZl cmFnZXM6ICA1LjIyLCAgNC4yOSwgIDIuMzQgICAgdXAgMCswMDoxMDoxNQ0KPjE3OjM0OjAwDQo+ MzUyIHByb2Nlc3NlczogMSBydW5uaW5nLCAzNTAgc2xlZXBpbmcsIDEgem9tYmllDQo+Q1BVOiAg MC4yJSB1c2VyLCAgMC4wJSBuaWNlLCAgNC41JSBzeXN0ZW0sICAxLjclIGludGVycnVwdCwgOTMu NSUgaWRsZQ0KPk1lbTogMjk3TSBBY3RpdmUsIDY0OE0gSW5hY3QsIDEzOU0gV2lyZWQsIDY5NDhL IENhY2hlLCA3NTIwSyBCdWYsIDE4NjJNDQo+RnJlZQ0KPlN3YXA6IDE1MzZNIFRvdGFsLCAxNTM2 TSBGcmVlDQo+DQo+DQo+UElEIFVTRVJOQU1FIFRIUiBQUkkgTklDRSBTSVpFIFJFUyBTVEFURSBD IFRJTUUgV0NQVSBDT01NQU5EDQo+NTExMiBkaWFnIDEgOTYgMCAxOTAxMksgNTY2OEsgQ1BVMCAw IDA6MDAgMi4wMCUgdG9wDQo+MjIxOSByb290IDY4IDk2IDAgODU5TSAyMjZNIHVjb25kIDEgMDoy NCAwLjAwJSBtZ3dkDQo+DQo+JSBzdWRvIHByb2NzdGF0IC12IGBwZ3JlcCBtZ3dkYA0KPlBJRCBT VEFSVCBFTkQgUFJUIFJFUyBQUkVTIFJFRiBTSEQgRkwgVFAgUEFUSA0KPjIyMTMgMHg4MDBhNDYw MDAgMHg4MDdiNDEwMDAgci14IDE4MjA5IDIyODI3IDIgMSBDTiB2bg0KPi91c3IvbGliL2xpYm1n d2Quc28uMQ0KPg0KPg0KPg0KPg0KPkJTRCA5LjEgOg0KPg0KPmxhc3QgcGlkOiAgNTM0NDsgIGxv YWQgYXZlcmFnZXM6ICA1LjE3LCAgNC40NywgIDIuNzkgICAgdXAgMCswMDoyNjoxMg0KPjE3OjIy OjU3DQo+MzkgcHJvY2Vzc2VzOiAgMSBydW5uaW5nLCAzNyBzbGVlcGluZywgMSB6b21iaWUNCj5D UFU6ICAwLjIlIHVzZXIsICAwLjAlIG5pY2UsICAyLjIlIHN5c3RlbSwgIDAuNiUgaW50ZXJydXB0 LCA5Ny4wJSBpZGxlDQo+TWVtOiAzMzhNIEFjdGl2ZSwgNjY5TSBJbmFjdCwgMTQ3TSBXaXJlZCwg MzkySyBDYWNoZSwgNzQ4OEsgQnVmLCAxNzk5TQ0KPkZyZWUNCj5Td2FwOiAxNTM2TSBUb3RhbCwg MTUzNk0gRnJlZQ0KPg0KPg0KPlBJRCBVU0VSTkFNRSBUSFIgUFJJIE5JQ0UgU0laRSBSRVMgU1RB VEUgQyBUSU1FIFdDUFUgQ09NTUFORA0KPjUzMjggZGlhZyAxIDQwIDAgMTcwMjhLIDUyMDRLIENQ VTEgMCAwOjAwIDAuNDAlIHRvcA0KPjIxNTggcm9vdCA2OCA0MCAwIDg3NE0gMjYyTSB1d2FpdCAx IDA6MjMgMC4wMCUgbWd3ZA0KPg0KPiUgc3VkbyBwcm9jc3RhdCAtdiBgcGdyZXAgbWd3ZGANCj5Q SUQgU1RBUlQgRU5EIFBSVCBSRVMgUFJFUyBSRUYgU0hEIEZMIFRQIFBBVEgNCj4yMTU4IDB4ODAw YTFjMDAwIDB4ODA3Yzg3MDAwIHIteCAyMzMyOCAyNjc2MCAyIDEgQ07igLkgIHZuDQo+L3Vzci9s aWIvbGlibWd3ZC5zby4xDQo+DQo+DQo+DQo+DQo+DQo+VGhhbmtzDQo+U3VyZXNoDQo+DQo+DQo+ DQo+DQo+T24gNC8xLzE1LCA4OjM0IEFNLCAiTWV5ZXIsIENvbnJhZCIgPGNvbnJhZC5tZXllckBp c2lsb24uY29tPiB3cm90ZToNCj4NCj4+PiBPbiBNYXIgMzEsIDIwMTUsIGF0IDE3OjU0LCBHdW1w dWxhLCBTdXJlc2ggPFN1cmVzaC5HdW1wdWxhQG5ldGFwcC5jb20+DQo+Pj53cm90ZToNCj4+Pg0K Pj4+IFN0aWxsIHRyeWluZyB0byBmaW5kIG91dCB0aGUgcmVhc29uIGZvciBtb3JlIG1lbW9yeSBm b290IHByaW50IG9uIDkuMQ0KPj4+IGNvbXBhcmVkIHRvIDguMSAuDQo+Pj4gRG9lcyBzb21lIHRo aW5nIGxpa2UgY2x1c3RlcmluZyBjaGFuZ2VzIGluIHBhZ2UgZmF1bHQgaGFuZGxpbmcgY2F1c2UN Cj4+PiBtZW1vcnkgZm9vdCBwcmludCA/DQo+Pj4gaHR0cHM6Ly9zdm53ZWIuZnJlZWJzZC5vcmcv YmFzZT92aWV3PXJldmlzaW9uJnJldmlzaW9uPTIzNTg3Ng0KPj4+DQo+Pj4gQ29weWluZyBBbGFu IENveCAsIGlmIGNvdWxkIHRocm93IHNvbWUgaW5wdXRzIG9uIHRoaXMuDQo+Pg0KPj5TdXBlcnBh Z2VzIGFuZCBob3cgRnJlZUJTRCBkb2VzIGl0cyBiZXN0IHRvIHB1dCBydW50aW1lIGxpYnJhcmll cyBpbg0KPj5zdXBlcnBhZ2UtYWJsZSBjb21lcyB0byBtaW5kLi4NCj4+DQo+PlRoZSBWTUVNIGZv ciBsaWJyYXJpZXMgaXMgd2hhdCBjYXVnaHQgdXMgb2ZmIGd1YXJkIGxhc3QgeWVhciB3aGVuIGRl YWxpbmcNCj4+d2l0aCBhcHBsaWNhdGlvbnMgLS0gbW9yZSBsaWJyYXJpZXMgPT0gZ3JlYXRlciBm b290cHJpbnQgcGFzdCBlaXRoZXIgOC4wDQo+Pm9yIDkuMCBiZWNhdXNlIG9mIGNoYW5nZXMgdG8g Vk0vcnRsZC4NCj4+DQo+PkNvbnJhZCBNZXllciBoYWQgYSBjaGFuZ2Ugb3V0IHRvIHJlZHVjZSB0 aGUgZm9vdHByaW50IGZvciBsaWJyYXJpZXMsIGJ1dA0KPj5pdCB3YXMgcmFjeS9pbmNvbXBsZXRl IHVuZm9ydHVuYXRlbHkgOi8uLg0KPj4NCj4+SG9wZSB0aGF0IG1heWJlIGhlbHBzLi4uDQo+Pg0K Pj4tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KPj4NCj4+UmlnaHQuIFNvIHRo ZSBsaW5rZXIgYW5kIFJUTEQgbWFwIGVhY2ggYmluYXJ5IHNlZ21lbnQgd2l0aCAyTUIgdmlydHVh bA0KPj5wYWdlcywgYmVjYXVzZSB0aGF0IHdheSB5b3Ugb25seSBuZWVkIG9uZSBtYXBwaW5nIC8g VExCIGVudHJ5IHBlciBzZWdtZW50DQo+PihvciBhdCBsZWFzdCwgdXAgdG8gMk1CLi4uIG1vc3Qg bGlicmFyaWVzIGFyZSBtdWNoIHNtYWxsZXIgdGhhbiB0aGlzKS4NCj4+VGhpcyBpcyBhIHBlcmZv cm1hbmNlIG9wdGltaXphdGlvbi4gVGhlIGRpc2N1c3Npb24gYXJvdW5kIHVubWFwcGluZw0KPj51 bnVzZWQgcG9ydGlvbnMgb2YgdGhlIDJNQiByYW5nZSBjYW4gYmUgZm91bmQgaGVyZToNCj4+aHR0 cHM6Ly9yZXZpZXdzLmZyZWVic2Qub3JnL0QxMjYzIC4NCj4+DQo+PlRvIHN1bW1hcml6ZTogbGFy Z2VyLXRoYW4tbmVjZXNzYXJ5IHN1cGVycGFnZSBtYXBwaW5ncyBhZmZlY3Qgb25seSB2bWVtDQo+ PmFjY291bnRpbmc7IGFjdHVhbGx5IHVzZSBsZXNzIHJlc291cmNlcyAoUFRFJ3MgYW5kIGFueSBh ZGRpdGlvbmFsIHBlci1QVEUNCj4+dm0gYWNjb3VudGluZykgdGhhbiA0ayBwYWdlczsgYW5kIHVz ZSBmZXdlciBUTEIgZW50cmllcy4gVW5tYXBwaW5nIHRoZQ0KPj51bnVzZWQgcG9ydGlvbnMgaXMg dXNlbGVzcyBldmVuIGlmIHlvdSBnZXQgaXQgcmlnaHQuDQo+Pg0KPj5BcmUgeW91IGFjdHVhbGx5 IHNlZWluZyBncmVhdGVyIG1lbW9yeSBmb290cHJpbnQsIG9yIGp1c3QgZ3JlYXRlciB2bWVtDQo+ PmZvb3RwcmludD8gSSBkb24ndCBhY3R1YWxseSB1c2UgRnJlZUJTRDggb3IgOS4NCj4+DQo+PkNo ZWVycywNCj4+Q29ucmFkDQo+DQoNCg== From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 18:52:57 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A9097137 for ; Wed, 1 Apr 2015 18:52:57 +0000 (UTC) Received: from mailuogwdur.emc.com (mailuogwdur.emc.com [128.221.224.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailuogwprd51.lss.emc.com", Issuer "RSA Corporate Server CA v2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 54E739B9 for ; Wed, 1 Apr 2015 18:52:56 +0000 (UTC) Received: from maildlpprd56.lss.emc.com (maildlpprd56.lss.emc.com [10.106.48.160]) by mailuogwprd52.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id t31Iqov2030318 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 1 Apr 2015 14:52:54 -0400 X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd52.lss.emc.com t31Iqov2030318 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=isilon.com; s=jan2013; t=1427914375; bh=PV7giu9X0lGsgdvwaubtW0x5mZk=; h=From:To:CC:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:Content-Transfer-Encoding:MIME-Version; b=n/cIdKL4C1R3ItaMcWjC4XNGMmRbbsD54Vm4PX5j9ytVHssUkmRjtWxp25OZ5/jJJ 2etsDc0Lp41R3IU9Ymz1+Nehp4LjeQjZkIRRenYLxMwnB1mMkYchUfaAtPFXwDoRiA TSyJsrF/NSBSCwHwlFpdlaT0KeiwCAQXzWwk9wJA= X-DKIM: OpenDKIM Filter v2.4.3 mailuogwprd52.lss.emc.com t31Iqov2030318 Received: from mailusrhubprd02.lss.emc.com (mailusrhubprd02.lss.emc.com [10.253.24.20]) by maildlpprd56.lss.emc.com (RSA Interceptor); Wed, 1 Apr 2015 14:52:12 -0400 Received: from mxhub27.corp.emc.com (mxhub27.corp.emc.com [10.254.110.183]) by mailusrhubprd02.lss.emc.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.0) with ESMTP id t31IqXnn030872 (version=TLSv1 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 1 Apr 2015 14:52:34 -0400 Received: from MXHUB103.corp.emc.com (10.253.50.16) by mxhub27.corp.emc.com (10.254.110.183) with Microsoft SMTP Server (TLS) id 8.3.327.1; Wed, 1 Apr 2015 14:52:33 -0400 Received: from MX103CL02.corp.emc.com ([169.254.6.202]) by MXHUB103.corp.emc.com ([::1]) with mapi id 14.03.0224.002; Wed, 1 Apr 2015 14:52:33 -0400 From: "Meyer, Conrad" To: "Gumpula, Suresh" Subject: RE: BSD 8.1 and 9.1 memory increase Thread-Topic: BSD 8.1 and 9.1 memory increase Thread-Index: AQHQYN1kO9bXTk+QcE+8E2hd8pdV+Z0hK5sAgBZxt4CAADccAIAAWDtEgAB/noCAACxHgP//vyvE Date: Wed, 1 Apr 2015 18:52:32 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.13.36.127] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Sentrion-Hostname: mailusrhubprd02.lss.emc.com X-RSA-Classifications: public Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 18:52:57 -0000 From: Gumpula, Suresh [Suresh.Gumpula@netapp.com]=0A= Sent: Wednesday, April 01, 2015 11:42 AM=0A= To: Gumpula, Suresh; Meyer, Conrad; Garrett Cooper=0A= Cc: freebsd-hackers@freebsd.org; alc@freebsd.org=0A= Subject: Re: BSD 8.1 and 9.1 memory increase=0A= =0A= I just tried the change you mentioned on unmapping unused entries , and=0A= appears this saves good amount of real memory. On my idle machine it=0A= shows ~20M + real memory savings. Specifically the inactive/wired pages=0A= dropped down.=0A= https://reviews.freebsd.org/D1263=0A= =0A= And also drastic VSS size down for most of the processes. Is this=0A= incomplete or Can I take this change ?=0A= =0A= Thanks=0A= Suresh=0A= =0A= ________________________________________=0A= =0A= Hi,=0A= =0A= The criticisms in the comments on that review are still perfectly valid. I = would not use the patch as-is.=0A= =0A= It is surprising that you see rsz reductions with it. I think it's probably= an apples-to-oranges comparison and the application(s) have not been warme= d up yet.=0A= =0A= Best,=0A= Conrad= From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 19:09:15 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3382C68C; Wed, 1 Apr 2015 19:09:15 +0000 (UTC) Received: from mail-ig0-x231.google.com (mail-ig0-x231.google.com [IPv6:2607:f8b0:4001:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E9B10B72; Wed, 1 Apr 2015 19:09:14 +0000 (UTC) Received: by igbud6 with SMTP id ud6so57297017igb.1; Wed, 01 Apr 2015 12:09:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=QFB8CU9/vT7S6LG9GcJZPnJyiYLb7qZm/KY2EBE3bvQ=; b=og0vZpteMugp5LFsfWuTVma23iv9ie8iWInF+AVf1zO4QxzEDjQzAaFwc6v2lKUqDE xntj3v8YFzqkYrJsflfIsRKEd6RtRcoFKKjRjx5cYCnRxO7C3CLZMN42PXLVybmbwxX3 eaq+fd14bxAJifslijlwxnX5/ZxQ73WwY3GAjzZfQCpxIyPYL9BeAUNgufqFdxy3BSfH 50CsNJgzhsZVrIz5WjWGtM/8rtzNfxMZujSVeAluUsUls113FS8n1cHytdb0kRA0D6va IEtA7hTbeczelWA4XmarfMTR8ltMiBR9xHiP3rw65viXkzaHwsrzCLzR61hCA45u6qLT 9Bbw== MIME-Version: 1.0 X-Received: by 10.50.67.100 with SMTP id m4mr14345667igt.32.1427915354423; Wed, 01 Apr 2015 12:09:14 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Wed, 1 Apr 2015 12:09:14 -0700 (PDT) In-Reply-To: References: Date: Wed, 1 Apr 2015 12:09:14 -0700 X-Google-Sender-Auth: mYRIk_gGE835OskVq20rZWHcB5s Message-ID: Subject: Re: Bazaaring the cathedral (Lowering the Barrier to Entry) From: Adrian Chadd To: Craig Rodrigues Content-Type: text/plain; charset=UTF-8 Cc: Eitan Adler , freebsd-current Current , FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 19:09:15 -0000 You mean, like sourceforge offered in 2000? :) -a From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 19:12:17 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8DF08A5A for ; Wed, 1 Apr 2015 19:12:17 +0000 (UTC) Received: from mx142.netapp.com (mx142.netapp.com [216.240.21.19]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mx142.netapp.com", Issuer "VeriSign Class 3 International Server CA - G3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 42D61C41 for ; Wed, 1 Apr 2015 19:12:16 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.11,505,1422950400"; d="scan'208";a="32403693" Received: from hioexcmbx04-prd.hq.netapp.com ([10.122.105.37]) by mx142-out.netapp.com with ESMTP; 01 Apr 2015 12:07:16 -0700 Received: from HIOEXCMBX03-PRD.hq.netapp.com (10.122.105.36) by hioexcmbx04-prd.hq.netapp.com (10.122.105.37) with Microsoft SMTP Server (TLS) id 15.0.995.29; Wed, 1 Apr 2015 12:07:15 -0700 Received: from HIOEXCMBX03-PRD.hq.netapp.com ([::1]) by hioexcmbx03-prd.hq.netapp.com ([fe80::4c9:7f9e:4b9f:2c9c%21]) with mapi id 15.00.0995.031; Wed, 1 Apr 2015 12:07:15 -0700 From: "Gumpula, Suresh" To: "Meyer, Conrad" , "Gumpula, Suresh" Subject: Re: BSD 8.1 and 9.1 memory increase Thread-Topic: BSD 8.1 and 9.1 memory increase Thread-Index: AQHQYN1kO9bXTk+QcE+8E2hd8pdV+Z0hK5sAgBZxt4CAAGlnAIAAnTGA///3mQCAACxGAIAARdkA///BDIA= Date: Wed, 1 Apr 2015 19:07:14 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.7.141117 x-originating-ip: [10.122.56.79] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 19:12:17 -0000 Ok thanks!. I just booted up a machine with and without the change. And after an hour or so, looked at top output. Thanks Suresh On 4/1/15, 2:52 PM, "Meyer, Conrad" wrote: >From: Gumpula, Suresh [Suresh.Gumpula@netapp.com] >Sent: Wednesday, April 01, 2015 11:42 AM >To: Gumpula, Suresh; Meyer, Conrad; Garrett Cooper >Cc: freebsd-hackers@freebsd.org; alc@freebsd.org >Subject: Re: BSD 8.1 and 9.1 memory increase > >I just tried the change you mentioned on unmapping unused entries , and >appears this saves good amount of real memory. On my idle machine it >shows ~20M + real memory savings. Specifically the inactive/wired pages >dropped down. >https://reviews.freebsd.org/D1263 > >And also drastic VSS size down for most of the processes. Is this >incomplete or Can I take this change ? > >Thanks >Suresh > >________________________________________ > >Hi, > >The criticisms in the comments on that review are still perfectly valid. >I would not use the patch as-is. > >It is surprising that you see rsz reductions with it. I think it's >probably an apples-to-oranges comparison and the application(s) have not >been warmed up yet. > >Best, >Conrad From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 20:05:03 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90B04F9A; Wed, 1 Apr 2015 20:05:03 +0000 (UTC) Received: from mail-qg0-x235.google.com (mail-qg0-x235.google.com [IPv6:2607:f8b0:400d:c04::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44BFB1D8; Wed, 1 Apr 2015 20:05:03 +0000 (UTC) Received: by qgh3 with SMTP id 3so52765716qgh.2; Wed, 01 Apr 2015 13:05:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=CYHxygrnAyQNNTLjSMXb5oNsh6QgtB4DGpbHP7GLodA=; b=BAFraz9p9dHt7lKihWMgan1cJhfQbq3y7AcLrmIvjzYA89ho0LD05hyJRd5o9hFgd5 zvpsbTj6TeTXTvWHUW03NtLBddRclcOwDEmuLLzGQ3D8V+1yMjzxtMcJShzx5oiXB4i7 lZ/smqkndtC5ihEaWx4OZEJFQrQ+A7FapVJe11pLN0We+Ji6VlBjdjiVophoKhJJC1n9 k7NH6YpzfVS5/mzoSK+NQoR2+fDefPDourufa7NKTqCM9U255oA2LSUIyVhPqTUeJW0Y +0Yt5QqpGt2Y00n/IhIucQ04WcrRrCGcG6Xw7cgUuz8kNyOixTad8niAgKkV1X6+PF5a +ryg== MIME-Version: 1.0 X-Received: by 10.140.83.19 with SMTP id i19mr32958061qgd.97.1427918702349; Wed, 01 Apr 2015 13:05:02 -0700 (PDT) Received: by 10.140.38.73 with HTTP; Wed, 1 Apr 2015 13:05:02 -0700 (PDT) In-Reply-To: References: <551BC57D.5070101@gmail.com> Date: Wed, 1 Apr 2015 13:05:02 -0700 Message-ID: Subject: Re: NVMe performance 4x slower than expected From: Jim Harris To: Alan Somers Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-hackers@freebsd.org" , Tobias Oberstein , Michael Fuckner , Konstantin Belousov X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 20:05:03 -0000 On Wed, Apr 1, 2015 at 7:42 AM, Alan Somers wrote: > On Wed, Apr 1, 2015 at 4:16 AM, Tobias Oberstein > wrote: > > Hi, > > > > I am testing performance of a NVMe device (Intel P3700) using FIO at the > > block device level and get 4x slower performance than expected: > > > > 4kB Random Read > > > > Intel Datasheet FIO Measurement Match > > P3700 450,000 107,092 24% > > DC S3700 75,000 67,186 90% > > > > The 2nd line are results for an Intel DC S3700 for comparison (with this > > device, I do see the performance expected, but not for the P3700). > > > > Hardware: > > > > - 4 sockets, 48 core x86-64, 3TB RAM > > - 8 x Intel P3700 2TB > > - 12 x Intel DC S3700 800GB (via LSI HBAs) > > > > Software: > > > > FreeBSD 11 Current with patches (DMAR and ZFS patches, otherwise the box > > doesn't boot at all .. because of 3TB RAM and the amount of periphery). > > > Do you still have WITNESS and INVARIANTS turned on in your kernel > config? They're turned on by default for Current, but they do have > some performance impact. To turn them off, just build a > GENERIC-NODEBUG kernel . > > Could you also post full dmesg output as well as vmstat -i? > > > > > Complete info and test logs are here: > > > > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/perftests.md > > > > Right now I am running Linux on the box (openSUSE 13.2). Using the exact > > same FIO control file, the values for the DC S3700 are very close to > > FreeBSD, but the values for the P3700 are much higher: > > > > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/perftests.md#more-numbers-linux > > > > I am looking for tuning hints or general advice for FreeBSD and NVMe. > > > > I would like to go with FreeBSD (a major aspect is ZFS), but the > performance > > issues with NVMe might be a deal breaker. > > > > Cheers, > > /Tobias > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 20:52:21 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 78FEEF67; Wed, 1 Apr 2015 20:52:21 +0000 (UTC) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 069A7924; Wed, 1 Apr 2015 20:52:21 +0000 (UTC) Received: by widdi4 with SMTP id di4so58280309wid.0; Wed, 01 Apr 2015 13:52:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=ae8+fGkJBec3jxpbWZRCYEVqh3a+tJMkWwBzsCbTiSo=; b=uFb7PLQTG1hTCWs0ANYKF15ZjBDGEdevhCxHKXkcv1aweO5n00VcgdstJSaFPIt5nz XuTlfhL+NNejSu+8uzcnNttrDbBOAoaig9c+EwABlZUssS3yB0g+wFMC4D9pBdm4Epz1 kz01GIMTDRdf8WdORALAZK1PfY3yK0i/dg/YgTL99FWPg3lGuMx+UizS1P1A1TRi46ZE j2SC1NX8ekhLu4yMRhcy7ZyfezRllIqK0v+rp5OuZTlo8zHGv1F3w4kOzX0nBsxJiE// nZLOG316/f/wDvydQT8MAp7pMaqCVIsA+Xinr38iCa1wVAwZ2uLVWRBvJbP82klRW5zl mk6w== X-Received: by 10.194.77.230 with SMTP id v6mr89147447wjw.25.1427921539485; Wed, 01 Apr 2015 13:52:19 -0700 (PDT) Received: from [172.22.3.210] (p578b7aa0.dip0.t-ipconnect.de. [87.139.122.160]) by mx.google.com with ESMTPSA id dx11sm4267099wjb.23.2015.04.01.13.52.18 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Apr 2015 13:52:18 -0700 (PDT) Message-ID: <551C5A82.2090306@gmail.com> Date: Wed, 01 Apr 2015 22:52:18 +0200 From: Tobias Oberstein User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Jim Harris , Alan Somers Subject: Re: NVMe performance 4x slower than expected References: <551BC57D.5070101@gmail.com> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , Michael Fuckner , Konstantin Belousov X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 20:52:21 -0000 > > FreeBSD 11 Current with patches (DMAR and ZFS patches, otherwise the box > > doesn't boot at all .. because of 3TB RAM and the amount of periphery). > > Do you still have WITNESS and INVARIANTS turned on in your kernel > config? They're turned on by default for Current, but they do have > some performance impact. To turn them off, just build a > GENERIC-NODEBUG kernel . WITNESS is off, INVARIANTS is still on. Here is complete config: https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_kernel_conf.md This is the aggregated patch (work was done by Konstantin - thanks again btw!) https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_patch.md > Could you also post full dmesg output as well as vmstat -i? dmesg: https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_dmesg.md vmstat: https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_vmstat.md === Here are results from FIO under FreeBSD: https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd.md Here are results using _same_ FIO control file under Linux: https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/linux.md === The firmware for the P3700 cards was updated to the very latest as of today (using isdct under Linux). /Tobias From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 21:23:14 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EDF6BEF5; Wed, 1 Apr 2015 21:23:14 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8F30FC85; Wed, 1 Apr 2015 21:23:14 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t31LN4DP075967 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 2 Apr 2015 00:23:04 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t31LN4DP075967 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t31LN3r3075966; Thu, 2 Apr 2015 00:23:03 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 2 Apr 2015 00:23:03 +0300 From: Konstantin Belousov To: Tobias Oberstein Subject: Re: NVMe performance 4x slower than expected Message-ID: <20150401212303.GB2379@kib.kiev.ua> References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <551C5A82.2090306@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: "freebsd-hackers@freebsd.org" , Michael Fuckner , Jim Harris , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 21:23:15 -0000 On Wed, Apr 01, 2015 at 10:52:18PM +0200, Tobias Oberstein wrote: > > > FreeBSD 11 Current with patches (DMAR and ZFS patches, otherwise the box > > > doesn't boot at all .. because of 3TB RAM and the amount of periphery). > > > > Do you still have WITNESS and INVARIANTS turned on in your kernel > > config? They're turned on by default for Current, but they do have > > some performance impact. To turn them off, just build a > > GENERIC-NODEBUG kernel . > > WITNESS is off, INVARIANTS is still on. INVARIANTS are costly. > > Here is complete config: > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_kernel_conf.md > > This is the aggregated patch (work was done by Konstantin - thanks again > btw!) > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_patch.md > > > Could you also post full dmesg output as well as vmstat -i? > > dmesg: > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_dmesg.md > > vmstat: > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_vmstat.md > > === > > Here are results from FIO under FreeBSD: > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd.md > > Here are results using _same_ FIO control file under Linux: > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/linux.md Is this vmstat after the test ? Somewhat funny is that nvme does not use MSI(X). I have the following patch for a long time, it allowed to increase pps in iperf and similar tests when DMAR is enabled. In your case it could reduce the rate of the DMAR interrupts. diff --git a/sys/x86/iommu/intel_ctx.c b/sys/x86/iommu/intel_ctx.c index a18adcf..b23a4c1 100644 --- a/sys/x86/iommu/intel_ctx.c +++ b/sys/x86/iommu/intel_ctx.c @@ -586,6 +586,18 @@ dmar_ctx_unload_entry(struct dmar_map_entry *entry, bool free) } } +static struct dmar_qi_genseq * +dmar_ctx_unload_gseq(struct dmar_ctx *ctx, struct dmar_map_entry *entry, + struct dmar_qi_genseq *gseq) +{ + + if (TAILQ_NEXT(entry, dmamap_link) != NULL) + return (NULL); + if (ctx->batch_no++ % dmar_batch_coalesce != 0) + return (NULL); + return (gseq); +} + void dmar_ctx_unload(struct dmar_ctx *ctx, struct dmar_map_entries_tailq *entries, bool cansleep) @@ -619,8 +631,7 @@ dmar_ctx_unload(struct dmar_ctx *ctx, struct dmar_map_entries_tailq *entries, entry->gseq.gen = 0; entry->gseq.seq = 0; dmar_qi_invalidate_locked(ctx, entry->start, entry->end - - entry->start, TAILQ_NEXT(entry, dmamap_link) == NULL ? - &gseq : NULL); + entry->start, dmar_ctx_unload_gseq(ctx, entry, &gseq)); } TAILQ_FOREACH_SAFE(entry, entries, dmamap_link, entry1) { entry->gseq = gseq; diff --git a/sys/x86/iommu/intel_dmar.h b/sys/x86/iommu/intel_dmar.h index 2865ab5..6e0ab7f 100644 --- a/sys/x86/iommu/intel_dmar.h +++ b/sys/x86/iommu/intel_dmar.h @@ -93,6 +93,7 @@ struct dmar_ctx { u_int entries_cnt; u_long loads; u_long unloads; + u_int batch_no; struct dmar_gas_entries_tree rb_root; struct dmar_map_entries_tailq unload_entries; /* Entries to unload */ struct dmar_map_entry *first_place, *last_place; @@ -339,6 +340,7 @@ extern dmar_haddr_t dmar_high; extern int haw; extern int dmar_tbl_pagecnt; extern int dmar_match_verbose; +extern int dmar_batch_coalesce; extern int dmar_check_free; static inline uint32_t diff --git a/sys/x86/iommu/intel_drv.c b/sys/x86/iommu/intel_drv.c index c239579..e7dc3f9 100644 --- a/sys/x86/iommu/intel_drv.c +++ b/sys/x86/iommu/intel_drv.c @@ -153,7 +153,7 @@ dmar_count_iter(ACPI_DMAR_HEADER *dmarh, void *arg) return (1); } -static int dmar_enable = 0; +static int dmar_enable = 1; static void dmar_identify(driver_t *driver, device_t parent) { diff --git a/sys/x86/iommu/intel_utils.c b/sys/x86/iommu/intel_utils.c index f696f9d..d3c3267 100644 --- a/sys/x86/iommu/intel_utils.c +++ b/sys/x86/iommu/intel_utils.c @@ -624,6 +624,7 @@ dmar_barrier_exit(struct dmar_unit *dmar, u_int barrier_id) } int dmar_match_verbose; +int dmar_batch_coalesce = 100; static SYSCTL_NODE(_hw, OID_AUTO, dmar, CTLFLAG_RD, NULL, ""); SYSCTL_INT(_hw_dmar, OID_AUTO, tbl_pagecnt, CTLFLAG_RD, @@ -632,6 +633,9 @@ SYSCTL_INT(_hw_dmar, OID_AUTO, tbl_pagecnt, CTLFLAG_RD, SYSCTL_INT(_hw_dmar, OID_AUTO, match_verbose, CTLFLAG_RWTUN, &dmar_match_verbose, 0, "Verbose matching of the PCI devices to DMAR paths"); +SYSCTL_INT(_hw_dmar, OID_AUTO, batch_coalesce, CTLFLAG_RW | CTLFLAG_TUN, + &dmar_batch_coalesce, 0, + "Number of qi batches between interrupt"); #ifdef INVARIANTS int dmar_check_free; SYSCTL_INT(_hw_dmar, OID_AUTO, check_free, CTLFLAG_RWTUN, From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 21:34:58 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 634AB2FE; Wed, 1 Apr 2015 21:34:58 +0000 (UTC) Received: from mail-qg0-x22d.google.com (mail-qg0-x22d.google.com [IPv6:2607:f8b0:400d:c04::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 14D5AD8C; Wed, 1 Apr 2015 21:34:58 +0000 (UTC) Received: by qgep97 with SMTP id p97so54879301qge.1; Wed, 01 Apr 2015 14:34:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=5NMhpa9CLwBZZX4OcNsNQPC2YV7fEC50ndQ3Nbl0m8A=; b=EJ6GISIgPq5sMvL3w1CT3FTq+sN0tCfg8jh6RV7sxFuXv2DuLNprYh33OLd4MxAFb2 PDZxaJsEM7wLdZ+CcG2D1FStUZPkOUdk6WrovE4wk7n7fr5/R6lgWnEAHOIGShWtLJF7 NEsKYCNTvBtznHJfQX/MsyPZuMCHZ9/guCFnLVZbRlEokTHFHomAgp0tHbrx9hoRxDlh rjgy7TpmvZrnt7dW6FdWCcJVhDvVCVzOUR+LoCvE9SvQNEcHZT4BoQuBstoBwX5UR+q7 VaTYZgmhGf9Mg7LrNYCAZBiFEURSx+bqUSqsuKEn4snyR4qSPT1M+OJigZGTHtC28W/R WeyQ== MIME-Version: 1.0 X-Received: by 10.55.53.137 with SMTP id c131mr29896704qka.102.1427924097244; Wed, 01 Apr 2015 14:34:57 -0700 (PDT) Received: by 10.140.38.73 with HTTP; Wed, 1 Apr 2015 14:34:57 -0700 (PDT) In-Reply-To: <20150401212303.GB2379@kib.kiev.ua> References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> Date: Wed, 1 Apr 2015 14:34:57 -0700 Message-ID: Subject: Re: NVMe performance 4x slower than expected From: Jim Harris To: Konstantin Belousov Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-hackers@freebsd.org" , Tobias Oberstein , Michael Fuckner , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 21:34:58 -0000 On Wed, Apr 1, 2015 at 2:23 PM, Konstantin Belousov wrote: > On Wed, Apr 01, 2015 at 10:52:18PM +0200, Tobias Oberstein wrote: > > > > FreeBSD 11 Current with patches (DMAR and ZFS patches, otherwise > the box > > > > doesn't boot at all .. because of 3TB RAM and the amount of > periphery). > > > > > > Do you still have WITNESS and INVARIANTS turned on in your kernel > > > config? They're turned on by default for Current, but they do have > > > some performance impact. To turn them off, just build a > > > GENERIC-NODEBUG kernel . > > > > WITNESS is off, INVARIANTS is still on. > INVARIANTS are costly. > > > > > Here is complete config: > > > > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_kernel_conf.md > > > > This is the aggregated patch (work was done by Konstantin - thanks again > > btw!) > > > > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_patch.md > > > > > Could you also post full dmesg output as well as vmstat -i? > > > > dmesg: > > > > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_dmesg.md > > > > vmstat: > > > > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_vmstat.md > > > > === > > > > Here are results from FIO under FreeBSD: > > > > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd.md > > > > Here are results using _same_ FIO control file under Linux: > > > > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/linux.md > > Is this vmstat after the test ? > Somewhat funny is that nvme does not use MSI(X). > Yes - this is exactly the problem. nvme does use MSI-X if it can allocate the vectors (one per core). With 48 cores, I suspect we are quickly running out of vectors, so NVMe is reverting to INTx. Could you actually send vmstat -ia (I left off the 'a' previously) - just so we can see all allocated interrupt vectors. As an experiment, can you try disabling hyperthreading - this will reduce the number of cores and should let you get MSI-X vectors allocated for at least the first couple of NVMe controllers. Then please re-run your performance test on one of those controllers. sys/x86/x86/local_apic.c defines APIC_NUM_IOINTS to 191 - it looks like this is the actual limit for MSI-X vectors, even though NUM_MSI_INTS is set to 512. > > I have the following patch for a long time, it allowed to increase pps > in iperf and similar tests when DMAR is enabled. In your case it could > reduce the rate of the DMAR interrupts. > > diff --git a/sys/x86/iommu/intel_ctx.c b/sys/x86/iommu/intel_ctx.c > index a18adcf..b23a4c1 100644 > --- a/sys/x86/iommu/intel_ctx.c > +++ b/sys/x86/iommu/intel_ctx.c > @@ -586,6 +586,18 @@ dmar_ctx_unload_entry(struct dmar_map_entry *entry, > bool free) > } > } > > +static struct dmar_qi_genseq * > +dmar_ctx_unload_gseq(struct dmar_ctx *ctx, struct dmar_map_entry *entry, > + struct dmar_qi_genseq *gseq) > +{ > + > + if (TAILQ_NEXT(entry, dmamap_link) != NULL) > + return (NULL); > + if (ctx->batch_no++ % dmar_batch_coalesce != 0) > + return (NULL); > + return (gseq); > +} > + > void > dmar_ctx_unload(struct dmar_ctx *ctx, struct dmar_map_entries_tailq > *entries, > bool cansleep) > @@ -619,8 +631,7 @@ dmar_ctx_unload(struct dmar_ctx *ctx, struct > dmar_map_entries_tailq *entries, > entry->gseq.gen = 0; > entry->gseq.seq = 0; > dmar_qi_invalidate_locked(ctx, entry->start, entry->end - > - entry->start, TAILQ_NEXT(entry, dmamap_link) == NULL ? > - &gseq : NULL); > + entry->start, dmar_ctx_unload_gseq(ctx, entry, &gseq)); > } > TAILQ_FOREACH_SAFE(entry, entries, dmamap_link, entry1) { > entry->gseq = gseq; > diff --git a/sys/x86/iommu/intel_dmar.h b/sys/x86/iommu/intel_dmar.h > index 2865ab5..6e0ab7f 100644 > --- a/sys/x86/iommu/intel_dmar.h > +++ b/sys/x86/iommu/intel_dmar.h > @@ -93,6 +93,7 @@ struct dmar_ctx { > u_int entries_cnt; > u_long loads; > u_long unloads; > + u_int batch_no; > struct dmar_gas_entries_tree rb_root; > struct dmar_map_entries_tailq unload_entries; /* Entries to unload > */ > struct dmar_map_entry *first_place, *last_place; > @@ -339,6 +340,7 @@ extern dmar_haddr_t dmar_high; > extern int haw; > extern int dmar_tbl_pagecnt; > extern int dmar_match_verbose; > +extern int dmar_batch_coalesce; > extern int dmar_check_free; > > static inline uint32_t > diff --git a/sys/x86/iommu/intel_drv.c b/sys/x86/iommu/intel_drv.c > index c239579..e7dc3f9 100644 > --- a/sys/x86/iommu/intel_drv.c > +++ b/sys/x86/iommu/intel_drv.c > @@ -153,7 +153,7 @@ dmar_count_iter(ACPI_DMAR_HEADER *dmarh, void *arg) > return (1); > } > > -static int dmar_enable = 0; > +static int dmar_enable = 1; > static void > dmar_identify(driver_t *driver, device_t parent) > { > diff --git a/sys/x86/iommu/intel_utils.c b/sys/x86/iommu/intel_utils.c > index f696f9d..d3c3267 100644 > --- a/sys/x86/iommu/intel_utils.c > +++ b/sys/x86/iommu/intel_utils.c > @@ -624,6 +624,7 @@ dmar_barrier_exit(struct dmar_unit *dmar, u_int > barrier_id) > } > > int dmar_match_verbose; > +int dmar_batch_coalesce = 100; > > static SYSCTL_NODE(_hw, OID_AUTO, dmar, CTLFLAG_RD, NULL, ""); > SYSCTL_INT(_hw_dmar, OID_AUTO, tbl_pagecnt, CTLFLAG_RD, > @@ -632,6 +633,9 @@ SYSCTL_INT(_hw_dmar, OID_AUTO, tbl_pagecnt, CTLFLAG_RD, > SYSCTL_INT(_hw_dmar, OID_AUTO, match_verbose, CTLFLAG_RWTUN, > &dmar_match_verbose, 0, > "Verbose matching of the PCI devices to DMAR paths"); > +SYSCTL_INT(_hw_dmar, OID_AUTO, batch_coalesce, CTLFLAG_RW | CTLFLAG_TUN, > + &dmar_batch_coalesce, 0, > + "Number of qi batches between interrupt"); > #ifdef INVARIANTS > int dmar_check_free; > SYSCTL_INT(_hw_dmar, OID_AUTO, check_free, CTLFLAG_RWTUN, > From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 22:04:21 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3E000E66; Wed, 1 Apr 2015 22:04:21 +0000 (UTC) Received: from mail-wi0-x231.google.com (mail-wi0-x231.google.com [IPv6:2a00:1450:400c:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BE424104; Wed, 1 Apr 2015 22:04:20 +0000 (UTC) Received: by wibgn9 with SMTP id gn9so83385704wib.1; Wed, 01 Apr 2015 15:04:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=e6+8Cx4RDKgAtPb6N7f5AX5PNR7KrvoP5TaAhEhni18=; b=UKLHoHcVsI2HlnTwzjivr7Te+nf2j9UMye38i7zRznOpVgFSp5+bfeQHVBiYSztUmb slvBzlKF4O+JnaKTQXvCadLbRiCw2ji41O2L9xl7WcXj4DtbG//Hc3dZuJFXjeb1BHNy SUTsKxfm9K43XOEpl9vz7k9yh+Yz8pLqpA+IFR/lzZ/4ZvGYBpSlRkBhPzV1G9MoiZBq wKxWQougJ8f5pTmNH3s/AIM/Hst5nyz35o743/2o4JcGdHGGEM69uqkx9aY2TTU6gYpQ KQodYj3qm+a0QSR8oebOs12v3SJBo0D49ViIhcHxf+tJiwAMoOh42ovUEgY+ofgJdQ3r wj6w== X-Received: by 10.180.208.46 with SMTP id mb14mr18914615wic.31.1427925859360; Wed, 01 Apr 2015 15:04:19 -0700 (PDT) Received: from [172.22.3.210] (p578b7aa0.dip0.t-ipconnect.de. [87.139.122.160]) by mx.google.com with ESMTPSA id r14sm4957351wiv.13.2015.04.01.15.04.18 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Apr 2015 15:04:18 -0700 (PDT) Message-ID: <551C6B62.7080205@gmail.com> Date: Thu, 02 Apr 2015 00:04:18 +0200 From: Tobias Oberstein User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Jim Harris , Konstantin Belousov Subject: Re: NVMe performance 4x slower than expected References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , Michael Fuckner , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 22:04:21 -0000 > Is this vmstat after the test ? No, it wasn't (I ran vmstat hours after the test). Here is right after test (shortened test duration, otherwise exactly the same FIO config): https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_vmstat.md#nvd7 > Somewhat funny is that nvme does not use MSI(X). > > > Yes - this is exactly the problem. > > nvme does use MSI-X if it can allocate the vectors (one per core). With > 48 cores, > I suspect we are quickly running out of vectors, so NVMe is reverting to > INTx. > > Could you actually send vmstat -ia (I left off the 'a' previously) - > just so we can > see all allocated interrupt vectors. > > As an experiment, can you try disabling hyperthreading - this will > reduce the The CPUs in this box root@s4l-zfs:~/src/sys/amd64/conf # sysctl hw.model hw.model: Intel(R) Xeon(R) CPU E7-8857 v2 @ 3.00GHz don't have hyperthreading (we deliberately selected CPU model for max. clock rather than HT) http://ark.intel.com/products/75254/Intel-Xeon-Processor-E7-8857-v2-30M-Cache-3_00-GHz > number of cores and should let you get MSI-X vectors allocated for at least > the first couple of NVMe controllers. Then please re-run your performance > test on one of those controllers. > You mean I should run against nvdN where N is a controller that still got MSI-X while other controllers did not? How would I find out which controller N? I don't know which nvdN is mounted in a PCIe slot directly assigned to which CPU socket, and I don't know which one's still got MSI-X and which not. I could arrange for disabling all but 1 CPU and retest. Would that help? === Right after running against nvd7 irq56: nvme0 6440 0 ... irq106: nvme7 145056 3 Then, immediately thereafter, running against nvd0 https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_vmstat.md#nvd0 irq56: nvme0 9233 0 ... irq106: nvme7 145056 3 === Earlier this day, I ran multiple longer tests .. all against nvd7. So if these are cumulative numbers since last boot, that would make sense. From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 22:24:50 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 918C7908; Wed, 1 Apr 2015 22:24:50 +0000 (UTC) Received: from mail-wg0-x236.google.com (mail-wg0-x236.google.com [IPv6:2a00:1450:400c:c00::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1E32A36C; Wed, 1 Apr 2015 22:24:50 +0000 (UTC) Received: by wgoe14 with SMTP id e14so67750731wgo.0; Wed, 01 Apr 2015 15:24:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=sbwvBEDLPZc4xuoFlDvRpXBuO2lCPfN09/YZMISFyso=; b=CvkbQ3pOrYo93mSaVb1dJS1I8jMExWq+6/U3+rblJeunowsWVIAcBje8UoI4/DRXkb FxBvAtQWFrW4XeYaCapPolXVjtkH9MZKf84idL5CbZpGzayah3KFFOSoF8HxdpNUJn0A b9BOdFMiJvUfNNsEiovMI0+sIj8NWpqN9OfGcDahE14OvmDHY3MABUqwwE0VpIOBFGhm oqrqc8rRED9OTAD60UiRhO0LhaZGuRsN8KK3eXYbci7WWGq4p2WYjy5D0b8op8zd71Oe G5Rk1xHGu8Th5SKONegGzvFLeBG1JzlTmeH0uZy08yr69cdSxMY6yt5/DaDR2cXNsAe0 ZrJA== X-Received: by 10.194.177.132 with SMTP id cq4mr84237742wjc.99.1427927088497; Wed, 01 Apr 2015 15:24:48 -0700 (PDT) Received: from [172.22.3.210] (p578b7aa0.dip0.t-ipconnect.de. [87.139.122.160]) by mx.google.com with ESMTPSA id 17sm4521152wjt.45.2015.04.01.15.24.47 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Apr 2015 15:24:47 -0700 (PDT) Message-ID: <551C702D.2070009@gmail.com> Date: Thu, 02 Apr 2015 00:24:45 +0200 From: Tobias Oberstein User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: NVMe performance 4x slower than expected References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> In-Reply-To: <20150401212303.GB2379@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , Michael Fuckner , Jim Harris , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 22:24:50 -0000 Am 01.04.2015 um 23:23 schrieb Konstantin Belousov: > On Wed, Apr 01, 2015 at 10:52:18PM +0200, Tobias Oberstein wrote: >>> > FreeBSD 11 Current with patches (DMAR and ZFS patches, otherwise the box >>> > doesn't boot at all .. because of 3TB RAM and the amount of periphery). >>> >>> Do you still have WITNESS and INVARIANTS turned on in your kernel >>> config? They're turned on by default for Current, but they do have >>> some performance impact. To turn them off, just build a >>> GENERIC-NODEBUG kernel . >> >> WITNESS is off, INVARIANTS is still on. > INVARIANTS are costly. ah, ok. will rebuild without this option. > I have the following patch for a long time, it allowed to increase pps > in iperf and similar tests when DMAR is enabled. In your case it could > reduce the rate of the DMAR interrupts. You mean these lines from vmstat? irq257: dmar0:qi 22312 0 irq259: dmar1:qi 22652 0 irq261: dmar2:qi 261874194 6911 irq263: dmar3:qi 124939 3 So these dmar2 interrupts come from DMAR region 2 which is used by nvd7? From dmesg: dmar0: iomem 0xc7ffc000-0xc7ffcfff on acpi0 dmar1: iomem 0xe3ffc000-0xe3ffcfff on acpi0 dmar2: iomem 0xfbffc000-0xfbffcfff on acpi0 dmar3: iomem 0xabffc000-0xabffcfff on acpi0 mpr0: dmar3 pci0:4:0:0 rid 400 domain 4 mgaw 48 agaw 48 re-mapped mpr1: dmar2 pci0:195:0:0 rid c300 domain 2 mgaw 48 agaw 48 re-mapped nvme0: dmar0 pci0:65:0:0 rid 4100 domain 0 mgaw 48 agaw 48 re-mapped nvme1: dmar0 pci0:67:0:0 rid 4300 domain 1 mgaw 48 agaw 48 re-mapped nvme2: dmar0 pci0:69:0:0 rid 4500 domain 2 mgaw 48 agaw 48 re-mapped nvme3: dmar1 pci0:129:0:0 rid 8100 domain 0 mgaw 48 agaw 48 re-mapped nvme4: dmar1 pci0:131:0:0 rid 8300 domain 1 mgaw 48 agaw 48 re-mapped nvme5: dmar1 pci0:132:0:0 rid 8400 domain 2 mgaw 48 agaw 48 re-mapped nvme6: dmar2 pci0:193:0:0 rid c100 domain 0 mgaw 48 agaw 48 re-mapped nvme7: dmar2 pci0:194:0:0 rid c200 domain 1 mgaw 48 agaw 48 re-mapped unknown: dmar3 pci0:0:29:0 rid e8 domain 0 mgaw 48 agaw 48 re-mapped unknown: dmar3 pci0:0:26:0 rid d0 domain 1 mgaw 48 agaw 48 re-mapped ix0: dmar3 pci0:1:0:0 rid 100 domain 2 mgaw 48 agaw 48 re-mapped ix1: dmar3 pci0:1:0:1 rid 101 domain 3 mgaw 48 agaw 48 re-mapped ix0: Using MSIX interrupts with 49 vectors ix1: Using MSIX interrupts with 49 vectors -- So the LSI HBAs, Intel NICs and NVMe are all using DMAR, but only the NICs use MSI-X? But 2 * 49 = 98, and that is smaller than the 191 which Jim mentions. And what are those "unknown" devices on dmar3? Actually, I don't really know what I am talking about here .. just puzzled. /Tobias From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 22:55:13 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B0ED8264; Wed, 1 Apr 2015 22:55:13 +0000 (UTC) Received: from mail-ie0-x231.google.com (mail-ie0-x231.google.com [IPv6:2607:f8b0:4001:c03::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 72EB68E7; Wed, 1 Apr 2015 22:55:13 +0000 (UTC) Received: by iedm5 with SMTP id m5so56117436ied.3; Wed, 01 Apr 2015 15:55:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=VbzJLTYIR5L4vfq9uXXQo5nBhLOWj7oI4yuSPT2zC4Q=; b=l9JwfEJ1QPJQbCGGumVidGfKaBHIcQEx6Duj0EsnmmzXipDzZxnpZmAHfMubeiargJ U34utaPI+jUMcCja4m0IBnO+KQaa5KNOpyfHHmYfA3crtYslB4b9405W5CTOds++hDqx NvZ4GpBdxNSavQRrC95mYoGk5xFfObcV5pVbCOKyBHm/b5BUlS/eyzOg64lHRabpJ7ky 90PKszQGp4xk82lXnDjRmiugOc90cWABkKrWGouNroUUizmt+LEn8aJkN8VL23JDWwFc aHvOipuC6Cg8+o3fQ3enRMde09NgYXkSXWZy8hnoxjslpeJVISZwY0Dx1IXe9yM6up8M mUxA== MIME-Version: 1.0 X-Received: by 10.43.82.137 with SMTP id ac9mr20510509icc.37.1427928912875; Wed, 01 Apr 2015 15:55:12 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Wed, 1 Apr 2015 15:55:12 -0700 (PDT) In-Reply-To: <551C702D.2070009@gmail.com> References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <551C702D.2070009@gmail.com> Date: Wed, 1 Apr 2015 15:55:12 -0700 X-Google-Sender-Auth: hIVAUzvCpL9G-REGkq_zJZvpHxY Message-ID: Subject: Re: NVMe performance 4x slower than expected From: Adrian Chadd To: Tobias Oberstein Content-Type: text/plain; charset=UTF-8 Cc: Konstantin Belousov , "freebsd-hackers@freebsd.org" , Michael Fuckner , Jim Harris , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 22:55:13 -0000 Out of curiousity - why not cap how many MSI's are created? Why's it need one per CPU? -a From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 23:09:55 2015 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E1FFA6E7; Wed, 1 Apr 2015 23:09:55 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id F0A55A39; Wed, 1 Apr 2015 23:09:54 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id CAA01636; Thu, 02 Apr 2015 02:07:05 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1YdRgS-0005DF-5B; Thu, 02 Apr 2015 02:04:28 +0300 Message-ID: <551C7840.4020409@FreeBSD.org> Date: Thu, 02 Apr 2015 01:59:12 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Eitan Adler , FreeBSD Hackers , freebsd-current Current Subject: Re: Bazaaring the cathedral (Lowering the Barrier to Entry) References: In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 23:09:56 -0000 On 01/04/2015 19:55, Eitan Adler wrote: > To solve this problem I propose a simple solution: self-serve commit > access. We create a web service on accounts.freebsd.org via which > users can create themselves a freefall account. In addition to a > freefall account, the identical username would be created for the wiki > and phabricator, bugzilla, and any other service we might provide. Your proposal is incomplete because itd oesn't say anything about booking a mentor. Even if it's a bazaar a neophyte still has to get themselves a mentor, JIMHO. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 1 23:24:53 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D9184D80; Wed, 1 Apr 2015 23:24:53 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8B91BC4D; Wed, 1 Apr 2015 23:24:53 +0000 (UTC) Received: by qgeb100 with SMTP id b100so16089231qge.3; Wed, 01 Apr 2015 16:24:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=YYXc5iRGwUJr54pWm5TDGBdrsHOKWsbK8L06JXYkxwc=; b=XetcNOV5Bm8NmMHH4lk3lPcimN7JphLJjPpOlWacuhppWGmj9X3b+yr6nHF4JAiaXK nx2iACADfr7p5vAHJEnGVuQnKZGlfAkvJVP5kxVN/MrF3w6vEAJROZqIqZESF7KqC10T 9fSSqWQp0KSt6ykhz2ywpOK6tn1C340qQbs8Ou+gpekqCLznMRw/wxu84sCUSzIL9N3R Xk3ItPjCCQBNHNEWmhTBgLLJEDBMDDx8LTscTSTharMfTrPzx/S2ahGv7q+/0jaGEuvM BvPXlfKuyjDuGJP+BlCP7qhqDg57Rb4irvNCwAMLG40ftt+f2YRHtFfWbNAoM1BnBJak NAWQ== MIME-Version: 1.0 X-Received: by 10.141.18.131 with SMTP id u125mr41690420qhd.78.1427930692729; Wed, 01 Apr 2015 16:24:52 -0700 (PDT) Received: by 10.140.38.73 with HTTP; Wed, 1 Apr 2015 16:24:52 -0700 (PDT) In-Reply-To: <551C6B62.7080205@gmail.com> References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <551C6B62.7080205@gmail.com> Date: Wed, 1 Apr 2015 16:24:52 -0700 Message-ID: Subject: Re: NVMe performance 4x slower than expected From: Jim Harris To: Tobias Oberstein Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Konstantin Belousov , "freebsd-hackers@freebsd.org" , Michael Fuckner , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 23:24:53 -0000 On Wed, Apr 1, 2015 at 3:04 PM, Tobias Oberstein wrote: > Is this vmstat after the test ? >> > > No, it wasn't (I ran vmstat hours after the test). > > Here is right after test (shortened test duration, otherwise exactly the > same FIO config): > > https://github.com/oberstet/scratchbox/blob/master/ > freebsd/cruncher/results/freebsd_vmstat.md#nvd7 > > Somewhat funny is that nvme does not use MSI(X). >> >> >> Yes - this is exactly the problem. >> >> nvme does use MSI-X if it can allocate the vectors (one per core). With >> 48 cores, >> I suspect we are quickly running out of vectors, so NVMe is reverting to >> INTx. >> >> Could you actually send vmstat -ia (I left off the 'a' previously) - >> just so we can >> see all allocated interrupt vectors. >> >> As an experiment, can you try disabling hyperthreading - this will >> reduce the >> > > The CPUs in this box > > root@s4l-zfs:~/src/sys/amd64/conf # sysctl hw.model > hw.model: Intel(R) Xeon(R) CPU E7-8857 v2 @ 3.00GHz > > don't have hyperthreading (we deliberately selected CPU model for max. > clock rather than HT) > > http://ark.intel.com/products/75254/Intel-Xeon-Processor-E7- > 8857-v2-30M-Cache-3_00-GHz > > number of cores and should let you get MSI-X vectors allocated for at >> least >> the first couple of NVMe controllers. Then please re-run your performance >> test on one of those controllers. >> >> > You mean I should run against nvdN where N is a controller that still got > MSI-X while other controllers did not? > > How would I find out which controller N? I don't know which nvdN is > mounted in a PCIe slot directly assigned to which CPU socket, and I don't > know which one's still got MSI-X and which not. > vmstat -ia should show you which controllers were assigned per-core vectors - you'll see all of them in the irq256+ range instead of the single vector per controller you see now in the lower irq index range. > > I could arrange for disabling all but 1 CPU and retest. Would that help? > Yes - that would help. Depending on how your system is configured, and which CPU socket the NVMe controllers are attached to, you may need to keep 2 CPU sockets enabled. You can also try a debug tunable that is in the nvme driver. hw.nvme.per_cpu_io_queues=0 This would just try to allocate a single MSI-X vector per controller - so all cores would still share a single I/O queue pair, but it would be MSI-X instead of INTx. (This actually should be the first fallback if we cannot allocate per-core vectors). Would at least show we are able to allocate some number of MSI-X vectors for NVMe. > > === > > Right after running against nvd7 > > irq56: nvme0 6440 0 > ... > irq106: nvme7 145056 3 > > > Then, immediately thereafter, running against nvd0 > > https://github.com/oberstet/scratchbox/blob/master/ > freebsd/cruncher/results/freebsd_vmstat.md#nvd0 > > irq56: nvme0 9233 0 > ... > irq106: nvme7 145056 3 > > === > > Earlier this day, I ran multiple longer tests .. all against nvd7. So if > these are cumulative numbers since last boot, that would make sense. > > From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 09:02:46 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4C0BCC18; Thu, 2 Apr 2015 09:02:46 +0000 (UTC) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 08FB7F3; Thu, 2 Apr 2015 09:02:45 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 9451E1FE022; Thu, 2 Apr 2015 11:02:42 +0200 (CEST) Message-ID: <551D05DE.3070200@selasky.org> Date: Thu, 02 Apr 2015 11:03:26 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: Eitan Adler , FreeBSD Hackers , freebsd-current Current Subject: Re: Bazaaring the cathedral (Lowering the Barrier to Entry) References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 09:02:46 -0000 On 04/01/15 18:55, Eitan Adler wrote: > One of the key reasons for the lack of people is the high barrier of > entry to joining the FreeBSD project. While every modern project uses > git (usually hosted on github), FreeBSD uses self-hosted subversion. > The use of git goes beyond just the choice of version control. It > allows for workflows that FreeBSD can't even dream of. The linux > kernel has no concept of a committer. Instead anyone can clone the > git tree, build a kernel, and call themselves a Linux distribution. Hi Eitan, Before you speak so nicely about how Linux is doing things, have you ever tried to submit a patch to Linux yourself? I have a bunch of candidates in /usr/ports/multimedia/webcamd/work/webcamd-3.18.0.1/patches (Use this latest tarball: http://home.selasky.org:8192/distfiles/webcamd-4.0.0.2.tar.bz2) which you can start with as a fun experiment ! And then write back when your done. I'm starting counting right now. I have ported a lot of Linux USB drivers to userspace in FreeBSD through the webcamd project, and quite frequently I need to make patches to make the code compile which really should be up-streamed. Sometimes I also find real bugs. Sending the patch to Linux-USB is easy. Getting attention to the patch is hard. Frequent roadblocks in the Linux-USB: - patch must be styled correctly - patch must be send using a certain e-mail program - patch must apply cleanly to the Linux GIT - patch must have a signed-off-by before it can be committed Speaking about USB I don't want FreeBSD-USB to become what Linux-USB is. There are so many mails flowing into Linux-USB every day that no-one is caring to read it all. Getting a decent reply from someone can take months, because of the huge amount of e-mails. --HPS From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 09:33:24 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6B6C61B2; Thu, 2 Apr 2015 09:33:24 +0000 (UTC) Received: from m.saper.info (m.saper.info [IPv6:2a01:4f8:a0:7383::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "m.saper.info", Issuer "Marcin Cieslak 2011" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D47355EF; Thu, 2 Apr 2015 09:33:23 +0000 (UTC) Received: from m.saper.info (saper@m.saper.info [IPv6:2a01:4f8:a0:7383::]) by m.saper.info (8.14.9/8.14.9) with ESMTP id t329XJwg013591 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 2 Apr 2015 09:33:19 GMT (envelope-from saper@saper.info) Received: from localhost (saper@localhost) by m.saper.info (8.14.9/8.14.9/Submit) with ESMTP id t329XIVQ013578; Thu, 2 Apr 2015 09:33:19 GMT (envelope-from saper@saper.info) X-Authentication-Warning: m.saper.info: saper owned process doing -bs Date: Thu, 2 Apr 2015 09:33:18 +0000 From: Marcin Cieslak To: Eitan Adler Subject: Re: Bazaaring the cathedral (Lowering the Barrier to Entry) In-Reply-To: Message-ID: References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: FreeBSD Hackers , freebsd-current Current X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 09:33:24 -0000 On Wed, 1 Apr 2015, Eitan Adler wrote: > Hi all, > > One of the key reasons for the lack of people is the high barrier of > entry to joining the FreeBSD project. While every modern project uses > git (usually hosted on github), As much as I love github - please try to go to http://github.com/freebsd/freebsd-ports and try to view history of one particular port. Example task I am facing right now: "I need to compile iojs 1.0.4 using older version of FreeBSD port". Github web interface says to me: > > Sorry, this commit history is taking too long to generate. > > Refresh the page to try again, or view this history locally using the following command: > > git log master -- www/iojs/Makefile Sure one might argue every ports should be a separate repository and we should use git submodules. No, thank you. > Our wiki and code review tools are in an even worse position than our > repository. In order to contribute you need to send an email to > clusteradm@ and hope they deem you worthy enough of access. The bug > tracking system is in a better shape but isn't perfect: while any user > can create an account, their privileges are limited: they can't help > close out bad PRs, reclassify good ones, or do other generally useful > work. I am pretty frustrated I can't login to Phabricator without @freebsd.org address. This is something that should be addressed _ReallySoonNow_ I don't follow FreeBSD intrastructure discussions but I don't understand while we jumed from gnats onto bugzilla instead of going straight to Phabricator Maniphest. Also from my Phabricator experience it is still a bit better suited for svn-centralized model than git (although it supports git as well). > Today I hope we turn ourselves from the cathedral into a truly open > and democratic project! I'd love to have a Phabricator account but despite using FreeBSD since I think version 2.1 and contributing very little I never aspired to get a commit bit. //Marcin From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 10:08:32 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 734739FC; Thu, 2 Apr 2015 10:08:32 +0000 (UTC) Received: from mail-wi0-x231.google.com (mail-wi0-x231.google.com [IPv6:2a00:1450:400c:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 14CBC96E; Thu, 2 Apr 2015 10:08:32 +0000 (UTC) Received: by wiaa2 with SMTP id a2so99077410wia.0; Thu, 02 Apr 2015 03:08:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Wcl6KZVwlp4a9+XEpcLrICdfNbGsUTNFBtay2xd8YDw=; b=VeG3MuS0ioQt9IFs4VaGMI++JoEHY+b8I2CamKKHgUrBaalNvd9B4cwir5daedrQ0Y gjvOta8V/YkDmkuFxLuscGHmQYalNvhAOS+6f8/NB7Kahq5aabykwIlbvXhkW3kvjdhh wpfweAxbgNDDkT9GtPzhX1o2dZ4shIrCZOVPi8603/FR1ulQqfwpbixow5xOLiqc5hCj hbSBN3RztWoq81RLJ+8BlvIlLWv9egofPUMCZx1lmTtIRpeIQFCYGv2euqaBMqmwf6Zz MuMH6p6op36LJiGZ6LDPKhSPnJm8OB3FhGWAAzt/4CowlALTqKc39/ul7fmqNQYIXSzT tfaA== MIME-Version: 1.0 X-Received: by 10.180.208.46 with SMTP id mb14mr23036258wic.31.1427969310423; Thu, 02 Apr 2015 03:08:30 -0700 (PDT) Received: by 10.180.4.41 with HTTP; Thu, 2 Apr 2015 03:08:29 -0700 (PDT) Received: by 10.180.4.41 with HTTP; Thu, 2 Apr 2015 03:08:29 -0700 (PDT) In-Reply-To: <551D05DE.3070200@selasky.org> References: <551D05DE.3070200@selasky.org> Date: Thu, 2 Apr 2015 12:08:29 +0200 Message-ID: Subject: Re: Bazaaring the cathedral (Lowering the Barrier to Entry) From: =?UTF-8?Q?Fernando_Apestegu=C3=ADa?= To: Hans Petter Selasky Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Eitan Adler , freebsd-current@freebsd.org, FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 10:08:32 -0000 El 02/04/2015 11:03, "Hans Petter Selasky" escribi=C3=B3: > > On 04/01/15 18:55, Eitan Adler wrote: >> >> One of the key reasons for the lack of people is the high barrier of >> entry to joining the FreeBSD project. While every modern project uses >> git (usually hosted on github), FreeBSD uses self-hosted subversion. >> The use of git goes beyond just the choice of version control. It >> allows for workflows that FreeBSD can't even dream of. The linux >> kernel has no concept of a committer. Instead anyone can clone the >> git tree, build a kernel, and call themselves a Linux distribution. > > > Hi Eitan, > > Before you speak so nicely about how Linux is doing things, have you ever tried to submit a patch to Linux yourself? I have a bunch of candidates in /usr/ports/multimedia/webcamd/work/webcamd-3.18.0.1/patches (Use this latest tarball: http://home.selasky.org:8192/distfiles/webcamd-4.0.0.2.tar.bz2) which you can start with as a fun experiment ! And then write back when your done. I'm starting counting right now. > > I have ported a lot of Linux USB drivers to userspace in FreeBSD through the webcamd project, and quite frequently I need to make patches to make the code compile which really should be up-streamed. Sometimes I also find real bugs. Sending the patch to Linux-USB is easy. Getting attention to the patch is hard. Frequent roadblocks in the Linux-USB: > - patch must be styled correctly > - patch must be send using a certain e-mail program > - patch must apply cleanly to the Linux GIT > - patch must have a signed-off-by before it can be committed > I suppose no project is perfect, but all of the above make sense to me. The only thing I disagree is about the mail client. I've never seen that restriction in the documents in the documentation directory of the kernel. I've read restrictions about the format of the mails though. > Speaking about USB I don't want FreeBSD-USB to become what Linux-USB is. There are so many mails flowing into Linux-USB every day that no-one is caring to read it all. Getting a decent reply from someone can take months, because of the huge amount of e-mails. > Getting attention to the patch being hard is probably because of the amount of patches sent every day, but with a fairly smaller stream of patches in FreeBSD, we have some of them sitting in bugzilla for a really long time. Cheers. > --HPS > > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 10:15:27 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 37BA3CC6; Thu, 2 Apr 2015 10:15:27 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B40DDA52; Thu, 2 Apr 2015 10:15:26 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t32AFKqv073289 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 2 Apr 2015 13:15:20 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t32AFKqv073289 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t32AFKe8073284; Thu, 2 Apr 2015 13:15:20 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 2 Apr 2015 13:15:20 +0300 From: Konstantin Belousov To: Tobias Oberstein Subject: Re: NVMe performance 4x slower than expected Message-ID: <20150402101520.GC2379@kib.kiev.ua> References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <551C702D.2070009@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <551C702D.2070009@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: "freebsd-hackers@freebsd.org" , Michael Fuckner , Jim Harris , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 10:15:27 -0000 On Thu, Apr 02, 2015 at 12:24:45AM +0200, Tobias Oberstein wrote: > Am 01.04.2015 um 23:23 schrieb Konstantin Belousov: > > On Wed, Apr 01, 2015 at 10:52:18PM +0200, Tobias Oberstein wrote: > >>> > FreeBSD 11 Current with patches (DMAR and ZFS patches, otherwise the box > >>> > doesn't boot at all .. because of 3TB RAM and the amount of periphery). > >>> > >>> Do you still have WITNESS and INVARIANTS turned on in your kernel > >>> config? They're turned on by default for Current, but they do have > >>> some performance impact. To turn them off, just build a > >>> GENERIC-NODEBUG kernel . > >> > >> WITNESS is off, INVARIANTS is still on. > > INVARIANTS are costly. > > ah, ok. will rebuild without this option. > > > I have the following patch for a long time, it allowed to increase pps > > in iperf and similar tests when DMAR is enabled. In your case it could > > reduce the rate of the DMAR interrupts. > > You mean these lines from vmstat? > > irq257: dmar0:qi 22312 0 > irq259: dmar1:qi 22652 0 > irq261: dmar2:qi 261874194 6911 > irq263: dmar3:qi 124939 3 > > So these dmar2 interrupts come from DMAR region 2 which is used by nvd7? Dmar unit 2. In modern machines, there is one (or two, sometimes) translation units per CPU package, which handle devices from the PCIe buses rooted in the socket. Interrupt stats above mean that the load on your machine is unbalanced WRT PCIe buses, most of the DMA transfers were performed by devices attached to the bus(es) on socket where DMAR 2 is located. > > From dmesg: > > dmar0: iomem 0xc7ffc000-0xc7ffcfff on acpi0 > dmar1: iomem 0xe3ffc000-0xe3ffcfff on acpi0 > dmar2: iomem 0xfbffc000-0xfbffcfff on acpi0 > dmar3: iomem 0xabffc000-0xabffcfff on acpi0 > > mpr0: dmar3 pci0:4:0:0 rid 400 domain 4 mgaw 48 agaw 48 re-mapped > mpr1: dmar2 pci0:195:0:0 rid c300 domain 2 mgaw 48 agaw 48 re-mapped > > nvme0: dmar0 pci0:65:0:0 rid 4100 domain 0 mgaw 48 agaw 48 re-mapped > nvme1: dmar0 pci0:67:0:0 rid 4300 domain 1 mgaw 48 agaw 48 re-mapped > nvme2: dmar0 pci0:69:0:0 rid 4500 domain 2 mgaw 48 agaw 48 re-mapped > nvme3: dmar1 pci0:129:0:0 rid 8100 domain 0 mgaw 48 agaw 48 re-mapped > nvme4: dmar1 pci0:131:0:0 rid 8300 domain 1 mgaw 48 agaw 48 re-mapped > nvme5: dmar1 pci0:132:0:0 rid 8400 domain 2 mgaw 48 agaw 48 re-mapped > nvme6: dmar2 pci0:193:0:0 rid c100 domain 0 mgaw 48 agaw 48 re-mapped > nvme7: dmar2 pci0:194:0:0 rid c200 domain 1 mgaw 48 agaw 48 re-mapped > > unknown: dmar3 pci0:0:29:0 rid e8 domain 0 mgaw 48 agaw 48 re-mapped > unknown: dmar3 pci0:0:26:0 rid d0 domain 1 mgaw 48 agaw 48 re-mapped > > ix0: dmar3 pci0:1:0:0 rid 100 domain 2 mgaw 48 agaw 48 re-mapped > ix1: dmar3 pci0:1:0:1 rid 101 domain 3 mgaw 48 agaw 48 re-mapped > > ix0: Using MSIX interrupts with 49 vectors > ix1: Using MSIX interrupts with 49 vectors > > -- > > So the LSI HBAs, Intel NICs and NVMe are all using DMAR, but only the > NICs use MSI-X? MSI-X is the method of reporting interrupt requests to CPUs. DMARs are some engines to translate addresses of DMA requests (and also to translate interrupts). > > But 2 * 49 = 98, and that is smaller than the 191 which Jim mentions. > > And what are those "unknown" devices on dmar3? 0:26:0 and 0:29:0 are USB controllers, most likely, the b/d/f numbers are typical for the Intel PCH. "unknown" is displayed when pci device does not have driver attached, you probably do not have USB loaded. DMAR still has to enable the translation context for USB controllers, since BIOS performs transfers behind the OS, and instructs DMAR driver to enable mappings. From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 10:26:28 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C476C18B for ; Thu, 2 Apr 2015 10:26:28 +0000 (UTC) Received: from mail.embedded-brains.de (host-82-135-62-35.customer.m-online.net [82.135.62.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4EDFABB6 for ; Thu, 2 Apr 2015 10:26:27 +0000 (UTC) Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id B62F92A1930 for ; Thu, 2 Apr 2015 12:26:52 +0200 (CEST) Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 2Z3SH0EG9c8C for ; Thu, 2 Apr 2015 12:26:52 +0200 (CEST) Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id 19F3A2A194E for ; Thu, 2 Apr 2015 12:26:52 +0200 (CEST) X-Virus-Scanned: amavisd-new at zimbra.eb.localhost Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id s4heCwig_PKW for ; Thu, 2 Apr 2015 12:26:52 +0200 (CEST) Received: from [192.168.96.129] (unknown [192.168.96.129]) by mail.embedded-brains.de (Postfix) with ESMTPSA id F1E132A1930 for ; Thu, 2 Apr 2015 12:26:51 +0200 (CEST) Message-ID: <551D1948.3090805@embedded-brains.de> Date: Thu, 02 Apr 2015 12:26:16 +0200 From: Sebastian Huber User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: A tcp_do_segment() question Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 10:26:28 -0000 Hello, I do currently some tests with a port of the FreeBSD 9.3 network stack=20 to the RTEMS real-time operating system. I do an FTP transfer from my=20 development machine to the target (curl -T /dev/zero=20 ftp://anonymous@192.168.96.157/dev/null). So this is a one-sided TCP=20 transfer. In tcp_do_segment() there is one path as explained here: /* * Header prediction: check for the two common cases * of a uni-directional data xfer. If the packet has * no control flags, is in-sequence, the window didn't * change and we're not retransmitting, it's a * candidate. If the length is zero and the ack moved * forward, we're the sender side of the xfer. Just * free the data acked & wake any higher level process * that was blocked waiting for space. If the length * is non-zero and the ack didn't move, we're the * receiver side. If we're getting packets in-order * (the reassembly queue is empty), add the data to * the socket buffer and note that we need a delayed ack. * Make sure that the hidden state-flags are also off. * Since we check for TCPS_ESTABLISHED first, it can only * be TH_NEEDSYN. */ if (tp->t_state =3D=3D TCPS_ESTABLISHED && th->th_seq =3D=3D tp->rcv_nxt && (thflags & (TH_SYN|TH_FIN|TH_RST|TH_URG|TH_ACK)) =3D=3D TH_ACK &= & tp->snd_nxt =3D=3D tp->snd_max && tiwin && tiwin =3D=3D tp->snd_wnd && ((tp->t_flags & (TF_NEEDSYN|TF_NEEDFIN)) =3D=3D 0) && LIST_EMPTY(&tp->t_segq) && ((to.to_flags & TOF_TS) =3D=3D 0 || TSTMP_GEQ(to.to_tsval, tp->ts_recent)) ) { It seems that after some initial setup phase I end up always in this=20 branch. The problem is now that the retransmit timer is ticking (TT_REXMT= ). In this branch there are three alternatives: 1. if (tlen =3D=3D 0) { Since I transfer to the target, tlen !=3D 0. 2. } else if (th->th_ack =3D=3D tp->snd_una && tlen <=3D sbspace(&so->so_rcv)) { It seems I always end up here. 3. Otherwise. In the (1) alternative we end up in: if (tp->snd_una =3D=3D tp->snd_max) tcp_timer_activate(tp, TT_REXMT, 0); else if (!tcp_timer_active(tp, TT_PERSIST)) tcp_timer_activate(tp, TT_REXMT, tp->t_rxtcur); sowwakeup(so); if (so->so_snd.sb_cc) (void) tcp_output(tp); goto check_delack; So here we reset the retransmit timer if possible. In the (2) alternative we don't reset the retransmit timer! All my=20 connections stop after approx. 2:30min via tcp_timer_rexmt(). If I apply=20 the following hack @@ -1852,6 +1866,13 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th,=20 struct socket *so, } /* NB: sorwakeup_locked() does an implicit=20 unlock. */ sorwakeup_locked(so); + + if (tp->snd_una =3D=3D tp->snd_max) + tcp_timer_activate(tp, TT_REXMT, 0); + else if (!tcp_timer_active(tp, TT_PERSIST)) + tcp_timer_activate(tp, TT_REXMT, + tp->t_rxtcur); + if (DELAY_ACK(tp)) { tp->t_flags |=3D TF_DELACK; } else { in the (2) alternative, then I can transfer for hours. Does anyone know why the retransmit timer is not reset in the (2)=20 alternative? --=20 Sebastian Huber, embedded brains GmbH Address : Dornierstr. 4, D-82178 Puchheim, Germany Phone : +49 89 189 47 41-16 Fax : +49 89 189 47 41-09 E-Mail : sebastian.huber@embedded-brains.de PGP : Public key available on request. Diese Nachricht ist keine gesch=C3=A4ftliche Mitteilung im Sinne des EHUG= . From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 10:28:20 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5246300; Thu, 2 Apr 2015 10:28:20 +0000 (UTC) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 91421BD6; Thu, 2 Apr 2015 10:28:20 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id CC29A1FE022; Thu, 2 Apr 2015 12:28:10 +0200 (CEST) Message-ID: <551D19E6.8080405@selasky.org> Date: Thu, 02 Apr 2015 12:28:54 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: =?UTF-8?B?RmVybmFuZG8gQXBlc3RlZ3XDrWE=?= Subject: Re: Bazaaring the cathedral (Lowering the Barrier to Entry) References: <551D05DE.3070200@selasky.org> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Eitan Adler , freebsd-current@freebsd.org, FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 10:28:20 -0000 I hope this is not one more of those April fools :-) --HPS From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 10:33:09 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 370C051B for ; Thu, 2 Apr 2015 10:33:09 +0000 (UTC) Received: from mail-ie0-x233.google.com (mail-ie0-x233.google.com [IPv6:2607:f8b0:4001:c03::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F39B4C99 for ; Thu, 2 Apr 2015 10:33:08 +0000 (UTC) Received: by ierf6 with SMTP id f6so65196266ier.2 for ; Thu, 02 Apr 2015 03:33:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=pqgGlUUleu6uIbYDGotzNXscp+kuYSQPVL2YKfpzSzs=; b=stiwx3capJ9YpOnFN6uMWEPB4U27b+S0lFKTVyG86oXYQEbZPuXk0fciAmdoaPamAN Q9wcLWIkK/JmR/0aSGPdLC7hK2eQnrLo7bg1RR9+UC735lhMGCqJd5pNFvIXnIu8iu1R TSwyAugbgC4KUg4RY8HxQgJ9Yu5dvVu6rMu4XSH7N0RnkY6UCG5Xx5ujSSvCZwo7rUVp aLIg9v75aNIPktugGoKhjnJT7IjiVZEs8bTz+OxjPXWzhi920oCdcmwSJVvwX1gFUr9f MZZyp06oD1oAKxbr6L8n/ZtrXqN6ijESW6S8TUvdjm+VfwEfNnUNyRPV+2zf+4EcihUp rz8g== MIME-Version: 1.0 X-Received: by 10.50.7.34 with SMTP id g2mr19043894iga.36.1427970788422; Thu, 02 Apr 2015 03:33:08 -0700 (PDT) Received: by 10.64.78.105 with HTTP; Thu, 2 Apr 2015 03:33:08 -0700 (PDT) Date: Thu, 2 Apr 2015 03:33:08 -0700 Message-ID: Subject: Free BSD Kernel Documentation From: Rakesh Sharma To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 10:33:09 -0000 Can somebody please tell me link where i can find best documentation for freeBSD ARM Kernel From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 13:54:21 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5EBA9791; Thu, 2 Apr 2015 13:54:21 +0000 (UTC) Received: from slim.berklix.org (slim.berklix.org [94.185.90.68]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DF23B88C; Thu, 2 Apr 2015 13:54:16 +0000 (UTC) Received: from mart.js.berklix.net (p5DCBEFBF.dip0.t-ipconnect.de [93.203.239.191]) (authenticated bits=128) by slim.berklix.org (8.14.5/8.14.5) with ESMTP id t32Dx06w058153; Thu, 2 Apr 2015 15:59:00 +0200 (CEST) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by mart.js.berklix.net (8.14.3/8.14.3) with ESMTP id t32Drvmq010120; Thu, 2 Apr 2015 15:53:57 +0200 (CEST) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.7/8.14.7) with ESMTP id t32DrKd1075099; Thu, 2 Apr 2015 15:53:38 +0200 (CEST) (envelope-from jhs@berklix.com) Message-Id: <201504021353.t32DrKd1075099@fire.js.berklix.net> To: Hans Petter Selasky Subject: Re: Bazaaring the cathedral (Lowering the Barrier to Entry) From: "Julian H. Stacey" Organization: http://berklix.com BSD Unix Linux Consultants, Munich Germany User-agent: EXMH on FreeBSD http://berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Thu, 02 Apr 2015 12:28:54 +0200." <551D19E6.8080405@selasky.org> Date: Thu, 02 Apr 2015 15:53:20 +0200 Cc: Eitan Adler , freebsd-current@freebsd.org, FreeBSD Hackers , =?UTF-8?B?RmVybmFuZG8gQXBlc3RlZ3XDrWE=?= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 13:54:21 -0000 Hans Petter Selasky wrote: > I hope this is not one more of those April fools :-) I've been thinking that since Eitan's first post of Wed, 1 Apr 2015 09:55:11 -0700 (18:55 CEST) "self-serve commit access" I kept wondering what would keep looneys out ? :-) Your experience feeding back to Linux was interesting, I suppose we assume "the grass is greener" till we hear someone tried it :-) Fernando, Your mailer software auto fold mechanism is bad, best turn it off. It corrupted quote from Hans Petter. It inserted repeated "\n" not "\n> " Cheers, Julian -- Julian Stacey, BSD Linux Unix C Sys Eng Consultant Munich http://berklix.com Indent previous with "> ". Reply Below as a play script. Send plain text, Not quoted-printable, HTML, or base64. From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 14:12:52 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7AD22F2E; Thu, 2 Apr 2015 14:12:52 +0000 (UTC) Received: from mail-wi0-x230.google.com (mail-wi0-x230.google.com [IPv6:2a00:1450:400c:c05::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 04E31AB2; Thu, 2 Apr 2015 14:12:52 +0000 (UTC) Received: by wixm2 with SMTP id m2so56813743wix.0; Thu, 02 Apr 2015 07:12:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=lP3U3n8N6n/PwWkdivTiMAgDcWYjbF/TX/IOEYFNHN0=; b=fp0kSviDgo0hjVWTDQlEHhqQGDRWAvGcDE31plecXZ0ZRRQaTqIDKwF62Up/jX2vfU XJwP1q/OP1onyEUFNZMYTlbeOjf4I0pHp7Q2lLCk/z879A0aVIffJbNjuO/5gxbO+aPs jc5czE+oXOZz+ugRmv77BfpCV5xDAH/OziYZAoN3U/F64b5y2/mi+0JJ0CSQq0MbXu4e 8UUSXD0eYeV5JK1M18zru11ZyRRIookurdRcXyXh43cC9/oE0VFzrFbjefMmTw8Gg+c9 T+CyT5Qq5VLMMtgRMhaFjOjWZutKzYiI6dj7td1deFispkoQZsWazt9rR8vCERoDhbq7 RU7g== X-Received: by 10.194.187.236 with SMTP id fv12mr95977834wjc.131.1427983969519; Thu, 02 Apr 2015 07:12:49 -0700 (PDT) Received: from [192.168.43.73] ([89.15.239.102]) by mx.google.com with ESMTPSA id dj4sm7443820wjc.13.2015.04.02.07.12.48 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 02 Apr 2015 07:12:48 -0700 (PDT) Message-ID: <551D4E5F.9090400@gmail.com> Date: Thu, 02 Apr 2015 16:12:47 +0200 From: Tobias Oberstein User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Jim Harris Subject: Re: NVMe performance 4x slower than expected References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <551C6B62.7080205@gmail.com> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Konstantin Belousov , "freebsd-hackers@freebsd.org" , Michael Fuckner , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 14:12:52 -0000 > You can also try a debug tunable that is in the nvme driver. > > hw.nvme.per_cpu_io_queues=0 I have rerun tests with kernel that has INVARIANTS off, and above sysctl in loader.conf. Results are the same. vmstat now: root@s4l-zfs:~/oberstet # vmstat -ia | grep nvme irq371: nvme0 8 0 irq372: nvme0 7478 0 irq373: nvme1 8 0 irq374: nvme1 7612 0 irq375: nvme2 8 0 irq376: nvme2 7695 0 irq377: nvme3 7 0 irq378: nvme3 7716 0 irq379: nvme4 8 0 irq380: nvme4 7622 0 irq381: nvme5 7 0 irq382: nvme5 7561 0 irq383: nvme6 8 0 irq384: nvme6 7609 0 irq385: nvme7 7 0 irq386: nvme7 15373417 1174 === I was advised (off list) to run tests against a pure ramdisk. Here are results from a single socket E3: https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_ramdisk.md#xeon-e3-machine and here are results for the 48 core box https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_ramdisk.md#48-core-big-machine Performance with this box is 1/10 on this test compared to single socket E3! Something is severely wrong. It seems, there might be multiple issues (not only NVMe). And this is after already running with 3 patches to make it even boot. Well. I'm running out of ideas to try, and also patience with the users waiting for this box =( /Tobias From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 14:44:16 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BEB6DCE for ; Thu, 2 Apr 2015 14:44:16 +0000 (UTC) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.pix.net", Issuer "Pix.Com Technologies, LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 65164E8B for ; Thu, 2 Apr 2015 14:44:16 +0000 (UTC) Received: from torb.pix.net (verizon.pix.net [71.178.232.3]) (authenticated bits=0) by hydra.pix.net (8.15.1/8.15.1) with ESMTPA id t32EiEQ4087995; Thu, 2 Apr 2015 10:44:14 -0400 (EDT) (envelope-from lidl@pix.net) X-Authentication-Warning: hydra.pix.net: Host verizon.pix.net [71.178.232.3] claimed to be torb.pix.net Message-ID: <551D55BE.3020904@pix.net> Date: Thu, 02 Apr 2015 10:44:14 -0400 From: Kurt Lidl User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: NVMe performance 4x slower than expected References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <551C6B62.7080205@gmail.com> <551D4E5F.9090400@gmail.com> In-Reply-To: <551D4E5F.9090400@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 14:44:16 -0000 On 4/2/15 10:12 AM, Tobias Oberstein wrote: > I was advised (off list) to run tests against a pure ramdisk. > > Here are results from a single socket E3: > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_ramdisk.md#xeon-e3-machine > > > and here are results for the 48 core box > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_ramdisk.md#48-core-big-machine > > > Performance with this box is 1/10 on this test compared to single socket > E3! > > Something is severely wrong. It seems, there might be multiple issues > (not only NVMe). And this is after already running with 3 patches to > make it even boot. Offhand, I'd guess the performance difference between the single-socket machine and the quad-socket machine has to do with the NUMA effects of the memory in the multi-socket system. FreeBSD does not have per-socket memory allocation/affliation at this time. So, some of the memory allocated to your ramdisk might be accessible to your process only over the QPI interconnect between the different CPU sockets. You could install the latest and greatest intel-pcm tools from /usr/ports and see what that says about the memory while you are running your randomio/fio tests. -Kurt From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 15:19:13 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C1729A36; Thu, 2 Apr 2015 15:19:13 +0000 (UTC) Received: from dmz-mailsec-scanner-3.mit.edu (dmz-mailsec-scanner-3.mit.edu [18.9.25.14]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 33FDF234; Thu, 2 Apr 2015 15:19:12 +0000 (UTC) X-AuditID: 1209190e-f79a76d000000d1b-53-551d5cba7445 Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by dmz-mailsec-scanner-3.mit.edu (Symantec Messaging Gateway) with SMTP id 9E.E0.03355.ABC5D155; Thu, 2 Apr 2015 11:14:02 -0400 (EDT) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id t32FE1qB010441; Thu, 2 Apr 2015 11:14:02 -0400 Received: from multics.mit.edu (system-low-sipb.mit.edu [18.187.2.37]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id t32FDxxb013987 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 2 Apr 2015 11:14:01 -0400 Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id t32FDxjb022363; Thu, 2 Apr 2015 11:13:59 -0400 (EDT) Date: Thu, 2 Apr 2015 11:13:59 -0400 (EDT) From: Benjamin Kaduk X-X-Sender: kaduk@multics.mit.edu To: freebsd-hackers@freebsd.org Subject: Last week to submit FreeBSD 2015Q1 (January-March) Status Reports Message-ID: User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrAIsWRmVeSWpSXmKPExsUixCmqrLsrRjbU4MRqPYtd106zW8x584HJ Yvvmf4wOzB4zPs1nCWCM4rJJSc3JLEst0rdL4Mp4fOcQY8F67oq3i+6wNjDu5Oxi5OSQEDCR mHvgKiOELSZx4d56ti5GLg4hgcVMEi+u72GBcDYwSvxtmscM4Rxkkvi9vIcVpEVIoF7i15fn 7CA2i4CWxKnlf8FGsQmoSTze28wKMVZRYvOpSUDNHBwiAvISC87bg4SZgcz/Vy4zgdjCAl4S k79fB2vlFXCU6Jj2DMwWFdCRWL1/CgtEXFDi5MwnLBC9WhLLp29jmcAoMAtJahaS1AJGplWM sim5Vbq5iZk5xanJusXJiXl5qUW6xnq5mSV6qSmlmxjBASnJt4Px60GlQ4wCHIxKPLwZe2RC hVgTy4orcw8xSnIwKYny3o6UDRXiS8pPqcxILM6ILyrNSS0+xCjBwawkwisSApTjTUmsrEot yodJSXOwKInzbvrBFyIkkJ5YkpqdmlqQWgSTleHgUJLg/RoF1ChYlJqeWpGWmVOCkGbi4AQZ zgM0/A5IDW9xQWJucWY6RP4Uo6KUOO9rkIQASCKjNA+uF5YwXjGKA70izOsZDVTFA0w2cN2v gAYzAQ12mCcNMrgkESEl1cC40W5bunH7rGdLjgnWaWUW9HD8sygruZX5S/1+yb2F4albegwn iWfcMdH5smt+nttZ65Me+Uuu2KYWTrg8q153b2vvf67NclLpk5bz95ZZ1eioL9Yykpd8lhne scPw/bWDXyJiX8i+SVi8bRN38APvwE9duqIz17yYrFgkPXGGvzRr+T09tx4lluKMREMt5qLi RAAuOUlA8wIAAA== Cc: freebsd-current@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 15:19:14 -0000 Dear FreeBSD Community, The first quarter of 2015 has come and passed, and the deadline for the FreeBSD Quarterly Status Report for the first quarter is fast approaching -- the deadline is April 7, 2015, for work done in January through March. Status report submissions do not have to be very long. They may be about anything happening in the FreeBSD project and community, and provide a great way to inform FreeBSD users and developers about what you're working on. Submission of reports is not restricted to committers. Anyone doing anything interesting and FreeBSD-related can -- and should -- write one! The preferred and easiest submission method is to use the XML generator [1] with the results emailed to the status report team at monthly at freebsd.org . There is also an XML template [2] which can be filled in manually and attached if preferred. For the expected content and style, please study our guidelines on how to write a good status report [3]. You can also review previous issues [4,5] for ideas on the style and format. We are looking forward to all of your 2015Q1 reports! Thanks, Ben (on behalf of monthly@) [1] http://www.freebsd.org/cgi/monthly.cgi [2] http://www.freebsd.org/news/status/report-sample.xml [3] http://www.freebsd.org/news/status/howto.html [4] http://www.freebsd.org/news/status/report-2014-07-2014-09.html [5] http://www.freebsd.org/news/status/report-2014-10-2014-12.html From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 15:26:25 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04AE0178 for ; Thu, 2 Apr 2015 15:26:25 +0000 (UTC) Received: from mail-la0-f44.google.com (mail-la0-f44.google.com [209.85.215.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9C353367 for ; Thu, 2 Apr 2015 15:26:24 +0000 (UTC) Received: by lahf3 with SMTP id f3so62087414lah.2 for ; Thu, 02 Apr 2015 08:26:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=4ySAkDzwS5/dYtWY7a+ayjwKlBMrNVLqv5ivFpWiKWY=; b=enl0MF1xm0L8RS81wl5Cb4WXeL8Vdnn3vzcQP+H4+YBgu5n3CnOgo8dAqlVgufytTq M8O+IO6CVy6ng7cCg1WZ3nFOtHx8n0wb9XDrBAsnF0zmvL82fRvYc39r9BC0pRCtIWhu Y0JDPPlV9H7ecAsdSJUAjqj69Sb7pnKhvNeGzZidkamaGVvv9j30yEKtCnagG9xodpyZ cweBhQwB9r7dQ4C4qiAetQPA79krS4OvSrDytjPxj5XqVhU8FY0T1WAlBFYZWL5d3V4h xqDURu1es8uFCY9FHDoZslajDxQFAKpuwbWjViXy16Wgv+DFUB5J3yz7UzEaht1avg2y LoOw== X-Gm-Message-State: ALoCoQkNcueEL9ZebepvBExuQcXNpPTDoo+xvqn+yoBMHQN4r0k1bZcIH1TNrnYanWXsr6iCwT1w MIME-Version: 1.0 X-Received: by 10.112.198.40 with SMTP id iz8mr27623398lbc.101.1427988057277; Thu, 02 Apr 2015 08:20:57 -0700 (PDT) Received: by 10.25.19.83 with HTTP; Thu, 2 Apr 2015 08:20:57 -0700 (PDT) X-Originating-IP: [8.25.222.10] In-Reply-To: References: Date: Thu, 2 Apr 2015 08:20:57 -0700 Message-ID: Subject: Re: Bazaaring the cathedral (Lowering the Barrier to Entry) From: Samuel Cassiba To: Eitan Adler Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: FreeBSD Hackers , freebsd-current Current X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 15:26:25 -0000 Eitan, This being posted on April 1 sets off my BS-o-meter, but I'll bite since it's a topic worth shaving a yak or two over. WARNING: there be perceptions and opinions here Having been ephemerally associated with FreeBSD since early 4.x, I never really saw FreeBSD as being cathedral-like, but the barrier to entry today feels even higher than it did 15 years ago. I view FreeBSD as being very much bazaar-like in terms of the actual code that comprises the OS and how it's treated, but like I must reiterate: the wall is harder to scale today. I appreciate the work that everyone has done over the years to bring the project's infrastructure up to modern times, such as the work done bringing in CI, a modern bug tracker and an honest to goodness code review tool, but the general workflow that I learned all those years ago for getting a change to go live *feels* more or less still the same: - find/fix bug - send-pr (okay... but you get the idea) - pester a committer with the proper access - ??? - profit!!! The arbitrary nature of how commit bits have historically been allocated have been more of a detractor than anything, making the the barrier appear even higher than it may actually be. The general wisdom I learned was "when you've bugged people often enough to commit your code, they'll burden you with that ability", which is great in terms of a pure meritocracy, but we're people and things aren't so black and white. Commit access thresholds shouldn't be measured in SLOC. Yes, the meritocratic way of handling commit access has worked well enough so far, but it has been at a cost (see FreeBSD's standing in the mindshare/overall market, as well as number of active FreeBSD committers vs your favorite big name Linux flavor). Access to the tools that the project has gravitated towards can be difficult if not impossible to obtain without a long slog through commit hell to get that sweet @FreeBSD.org spiff. Is a "free commit access for anyone!" approach really the answer? It feels like going from one extreme to the other, and we all know how extremes work. I feel that commit access should be more transparent, so that if someone really wants to be able to push code or act as a representative of the project, they can learn how to work towards that, instead of some nebulous "push enough code until we get tired of committing it". Again, I'm pretty sure this is just a April 1 troll, but it's all worth saying since the topic was brought up. Lowering the barrier to entry is good, but the act of doing so mustn't come at a detriment to quality. -Samuel On Wed, Apr 1, 2015 at 9:55 AM, Eitan Adler wrote: > Hi all, > > FreeBSD today has a significant problem of attracting new developers. > The lack of new developers manifests itself in a dearth of driver > support, inadequate documentation, and trailing third party packages. > > One of the key reasons for the lack of people is the high barrier of > entry to joining the FreeBSD project. While every modern project uses > git (usually hosted on github), FreeBSD uses self-hosted subversion. > The use of git goes beyond just the choice of version control. It > allows for workflows that FreeBSD can't even dream of. The linux > kernel has no concept of a committer. Instead anyone can clone the > git tree, build a kernel, and call themselves a Linux distribution. > > Our wiki and code review tools are in an even worse position than our > repository. In order to contribute you need to send an email to > clusteradm@ and hope they deem you worthy enough of access. The bug > tracking system is in a better shape but isn't perfect: while any user > can create an account, their privileges are limited: they can't help > close out bad PRs, reclassify good ones, or do other generally useful > work. > > To solve this problem I propose a simple solution: self-serve commit > access. We create a web service on accounts.freebsd.org via which > users can create themselves a freefall account. In addition to a > freefall account, the identical username would be created for the wiki > and phabricator, bugzilla, and any other service we might provide. > > This will allow us to quickly gain large number of contributors, who > will be able to make better progress on our missing features, and > rectify some of the drawbacks listed above. > > Today I hope we turn ourselves from the cathedral into a truly open > and democratic project! > > > -- > Eitan Adler > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 16:15:14 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1EB62A83; Thu, 2 Apr 2015 16:15:14 +0000 (UTC) Received: from mail-qg0-x232.google.com (mail-qg0-x232.google.com [IPv6:2607:f8b0:400d:c04::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C42D0C01; Thu, 2 Apr 2015 16:15:13 +0000 (UTC) Received: by qgdy78 with SMTP id y78so11804165qgd.0; Thu, 02 Apr 2015 09:15:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=nc/5m8SgKpy8XDQ7l8bAuuZ0ox+mkebzMyCyPq9+GHU=; b=FUhB6tP275t6s7CiH+/++4P+s2ejr4YlE0ESLS0QfNfrymbuw2c4bB79QNAOJ0g9do 0M4iqQ7nRpOvb7k56SZB+U605LzEnZuqwV2d7XIo69fWteCYBFepjyguwBc9eBvXWwus IA44MWBY8DOROqfCH8DoR1/rdxk3IV9cVaDaeo7JhV441dunl9gaigkgrJg5jYafkAW4 9F/BGKt1dYeD9q/jQxp3TkqCwBAHwBBb01j2bh65P5eCuw4LYmc6sJfGrZXuerLgj97F 3w7Oh2gdcaTtFUsYWNbp+Y/PDUgs9Sv2SMdjH+L4Wkk5qMwuU996mjrE5nQHeas4zy9w jKaA== X-Received: by 10.141.18.208 with SMTP id u199mr62774193qhd.47.1427991312834; Thu, 02 Apr 2015 09:15:12 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.42.200 with HTTP; Thu, 2 Apr 2015 09:14:51 -0700 (PDT) In-Reply-To: References: From: "B. Estrade" Date: Thu, 2 Apr 2015 11:14:51 -0500 Message-ID: Subject: Re: Bazaaring the cathedral (Lowering the Barrier to Entry) To: Samuel Cassiba Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Eitan Adler , freebsd-current Current , FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 16:15:14 -0000 Joke or not, it's worth pointing out that while DragonFlyBSD's approach seems to be a fairly sane hybrid; there is no reason this can't be done within FreeBSD right now. https://www.dragonflybsd.org/docs/developer/TypicalGitUsage/ Anyone can clone, but if you can't commit to the repo then you're asked to register at http://bugs.dragonflybsd.org/ and send your patch over email. DFBSD's GItub repo is just a mirror and FreeBSD has something similar available, including a "GitWorkFlow" document: https://wiki.freebsd.org/GitWorkflow So I don't see why *now* someone couldn't basically follow the same workflow if they wanted to contribute to FreeBSD; namely: * clone mirrored repo/branch from GH * make changes * create a problem report at bugs.freebsd.org * submit patches And if this is not possible, it is likely a procedural impediment and not a technical one. Cheers, Brett On Thu, Apr 2, 2015 at 10:20 AM, Samuel Cassiba wrote: > Eitan, > > This being posted on April 1 sets off my BS-o-meter, but I'll bite since > it's a topic worth shaving a yak or two over. > > WARNING: there be perceptions and opinions here > > Having been ephemerally associated with FreeBSD since early 4.x, I never > really saw FreeBSD as being cathedral-like, but the barrier to entry today > feels even higher than it did 15 years ago. I view FreeBSD as being very > much bazaar-like in terms of the actual code that comprises the OS and how > it's treated, but like I must reiterate: the wall is harder to scale today. > > I appreciate the work that everyone has done over the years to bring the > project's infrastructure up to modern times, such as the work done bringing > in CI, a modern bug tracker and an honest to goodness code review tool, but > the general workflow that I learned all those years ago for getting a > change to go live *feels* more or less still the same: > - find/fix bug > - send-pr (okay... but you get the idea) > - pester a committer with the proper access > - ??? > - profit!!! > > The arbitrary nature of how commit bits have historically been allocated > have been more of a detractor than anything, making the the barrier appear > even higher than it may actually be. The general wisdom I learned was "when > you've bugged people often enough to commit your code, they'll burden you > with that ability", which is great in terms of a pure meritocracy, but > we're people and things aren't so black and white. Commit access thresholds > shouldn't be measured in SLOC. > > Yes, the meritocratic way of handling commit access has worked well enough > so far, but it has been at a cost (see FreeBSD's standing in the > mindshare/overall market, as well as number of active FreeBSD committers vs > your favorite big name Linux flavor). Access to the tools that the project > has gravitated towards can be difficult if not impossible to obtain without > a long slog through commit hell to get that sweet @FreeBSD.org spiff. > > Is a "free commit access for anyone!" approach really the answer? It feels > like going from one extreme to the other, and we all know how extremes > work. I feel that commit access should be more transparent, so that if > someone really wants to be able to push code or act as a representative of > the project, they can learn how to work towards that, instead of some > nebulous "push enough code until we get tired of committing it". > > Again, I'm pretty sure this is just a April 1 troll, but it's all worth > saying since the topic was brought up. Lowering the barrier to entry is > good, but the act of doing so mustn't come at a detriment to quality. > > > -Samuel > > On Wed, Apr 1, 2015 at 9:55 AM, Eitan Adler wrote: > > > Hi all, > > > > FreeBSD today has a significant problem of attracting new developers. > > The lack of new developers manifests itself in a dearth of driver > > support, inadequate documentation, and trailing third party packages. > > > > One of the key reasons for the lack of people is the high barrier of > > entry to joining the FreeBSD project. While every modern project uses > > git (usually hosted on github), FreeBSD uses self-hosted subversion. > > The use of git goes beyond just the choice of version control. It > > allows for workflows that FreeBSD can't even dream of. The linux > > kernel has no concept of a committer. Instead anyone can clone the > > git tree, build a kernel, and call themselves a Linux distribution. > > > > Our wiki and code review tools are in an even worse position than our > > repository. In order to contribute you need to send an email to > > clusteradm@ and hope they deem you worthy enough of access. The bug > > tracking system is in a better shape but isn't perfect: while any user > > can create an account, their privileges are limited: they can't help > > close out bad PRs, reclassify good ones, or do other generally useful > > work. > > > > To solve this problem I propose a simple solution: self-serve commit > > access. We create a web service on accounts.freebsd.org via which > > users can create themselves a freefall account. In addition to a > > freefall account, the identical username would be created for the wiki > > and phabricator, bugzilla, and any other service we might provide. > > > > This will allow us to quickly gain large number of contributors, who > > will be able to make better progress on our missing features, and > > rectify some of the drawbacks listed above. > > > > Today I hope we turn ourselves from the cathedral into a truly open > > and democratic project! > > > > > > -- > > Eitan Adler > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to " > freebsd-current-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 17:44:34 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E839646F; Thu, 2 Apr 2015 17:44:33 +0000 (UTC) Received: from udns.ultimatedns.net (unknown [IPv6:2602:d1:b4d6:e600:4261:86ff:fef6:aa2a]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B43DB964; Thu, 2 Apr 2015 17:44:33 +0000 (UTC) Received: from ultimatedns.net (localhost [127.0.0.1]) by udns.ultimatedns.net (8.14.9/8.14.9) with ESMTP id t32HiYX2058636; Thu, 2 Apr 2015 10:44:40 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) To: Eitan Adler , Samuel Cassiba In-Reply-To: References: , From: "Chris H" Subject: Re: Bazaaring the cathedral (Lowering the Barrier to Entry) Date: Thu, 02 Apr 2015 10:44:40 -0700 Content-Type: text/plain; charset=UTF-8; format=fixed MIME-Version: 1.0 Message-id: Content-Transfer-Encoding: 8bit Cc: FreeBSD Hackers , freebsd-current Current X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 17:44:34 -0000 On Thu, 2 Apr 2015 08:20:57 -0700 Samuel Cassiba wrote > Eitan, > > This being posted on April 1 sets off my BS-o-meter, but I'll bite since > it's a topic worth shaving a yak or two over. > > WARNING: there be perceptions and opinions here > > Having been ephemerally associated with FreeBSD since early 4.x, I never .. > Again, I'm pretty sure this is just a April 1 troll, but it's all worth > saying since the topic was brought up. Lowering the barrier to entry is > good, but the act of doing so mustn't come at a detriment to quality. OK. I'll bite... IMHO I believe that the height of the bar, is directly proportionate to the quality of the product. --Chris > > > -Samuel > > On Wed, Apr 1, 2015 at 9:55 AM, Eitan Adler wrote: > > > Hi all, > > > > FreeBSD today has a significant problem of attracting new developers. > > The lack of new developers manifests itself in a dearth of driver > > support, inadequate documentation, and trailing third party packages. > > > > One of the key reasons for the lack of people is the high barrier of > > entry to joining the FreeBSD project. While every modern project uses > > git (usually hosted on github), FreeBSD uses self-hosted subversion. > > The use of git goes beyond just the choice of version control. It > > allows for workflows that FreeBSD can't even dream of. The linux > > kernel has no concept of a committer. Instead anyone can clone the > > git tree, build a kernel, and call themselves a Linux distribution. > > > > Our wiki and code review tools are in an even worse position than our > > repository. In order to contribute you need to send an email to > > clusteradm@ and hope they deem you worthy enough of access. The bug > > tracking system is in a better shape but isn't perfect: while any user > > can create an account, their privileges are limited: they can't help > > close out bad PRs, reclassify good ones, or do other generally useful > > work. > > > > To solve this problem I propose a simple solution: self-serve commit > > access. We create a web service on accounts.freebsd.org via which > > users can create themselves a freefall account. In addition to a > > freefall account, the identical username would be created for the wiki > > and phabricator, bugzilla, and any other service we might provide. > > > > This will allow us to quickly gain large number of contributors, who > > will be able to make better progress on our missing features, and > > rectify some of the drawbacks listed above. > > > > Today I hope we turn ourselves from the cathedral into a truly open > > and democratic project! > > > > > > -- > > Eitan Adler > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 20:04:10 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5024571; Thu, 2 Apr 2015 20:04:10 +0000 (UTC) Received: from mail-ob0-x236.google.com (mail-ob0-x236.google.com [IPv6:2607:f8b0:4003:c01::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 98CCDC41; Thu, 2 Apr 2015 20:04:10 +0000 (UTC) Received: by obbgh1 with SMTP id gh1so138254638obb.1; Thu, 02 Apr 2015 13:04:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=VmJ222OcUfN53aXPDEjDiRFsKEQQUr3MJOECKUTCmv8=; b=P9bUPWTul5g1KXMj+jITpAPqEAwsM+eY3wIKmXlEPZN3Z1RM6hrlicCRo49RtUXNCl GtQFDZcCN1HX0V2jcIAVjAYxlhwFjHKOO9ZlULf7q9gse0nVlNfrOHwpJBMJKtWsTh+Y nZVkeliIekxl/flrm1mFiF0NGSLFrU1ZFq2yVVjSZ7mJl6APSoaPfzvy2tItQYMPCopi fZLiYe8uh/+GKc1kmAJAMDLI38HHidWSu9Wfr5GVtR+6vyCOm8zZmG8fPudcTcD58nxM SisgDMu/8UG0IzNdSirPxICk3JGQZoif38kR9ATEFhwXD3IYxcX40sp9RxzlZGIYA9qs Azhg== MIME-Version: 1.0 X-Received: by 10.60.33.106 with SMTP id q10mr48119403oei.67.1428005049501; Thu, 02 Apr 2015 13:04:09 -0700 (PDT) Sender: royce.williams@gmail.com Received: by 10.202.79.205 with HTTP; Thu, 2 Apr 2015 13:04:09 -0700 (PDT) Received: by 10.202.79.205 with HTTP; Thu, 2 Apr 2015 13:04:09 -0700 (PDT) In-Reply-To: References: Date: Thu, 2 Apr 2015 12:04:09 -0800 X-Google-Sender-Auth: 3SvuA2SZRB_hTc29khbmhF_7LiE Message-ID: Subject: Re: Bazaaring the cathedral (Lowering the Barrier to Entry) From: Royce Williams To: Chris H Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Eitan Adler , Samuel Cassiba , freebsd-current@freebsd.org, FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 20:04:10 -0000 On Apr 2, 2015 9:44 AM, "Chris H" wrote: > > IMHO I believe that the height of the bar, is directly proportionate > to the quality of the product. We were all new once. There are many reasons - language, social fluidity, economic background, etc. - for which a too-high initial hurdle can make a high bar so insurmountable as to prevent valuable contributors from getting in at all. Mentorship needs to take this into account. The trick is maintaining quality control at higher volumes, without quashing newbie enthusiasm. :) There are tools for managing this somewhat, but open source needs to grow more in this area. Royce From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 21:47:26 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4EABA50A; Thu, 2 Apr 2015 21:47:26 +0000 (UTC) Received: from mail-wg0-x22d.google.com (mail-wg0-x22d.google.com [IPv6:2a00:1450:400c:c00::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C3725AE4; Thu, 2 Apr 2015 21:47:25 +0000 (UTC) Received: by wgdm6 with SMTP id m6so97501086wgd.2; Thu, 02 Apr 2015 14:47:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=3uxCzqixpdiB9nm9RNHqpCSDc5Qq5g1DFfzAZYDAeeM=; b=wfC/SAt5XtZmCgIp4HXN6v+yr1KsDUtTrnTuURm1b8MYpQxBO/7i1cwCXYYRtgA67t MmeAmvjmPsyljyTalTie64+2nbSpB0bGXzJ1hy1qW5lJp75GSl6wa9zCI8qbOmtnGqLz t+1dvZn1jOuQ6zKZVfrnQk1PH6x7bTmzMppCBuwrRFnoH5YbvTFVztLINEnSxUWC5wuq GgsiD21nRw3V1d/WqNdNYMyVkjXIpy+GVPkp5VJ3kx/jZ/Lp9eAxKAq+AqoLxfzmNed8 n2+R71tVLpgk69sfeRoNoMCYm4Iq/vXORn7jGxhRYqPnE5PbmPI9Z12Vfa9WUWVRKvTv AuzA== X-Received: by 10.194.177.167 with SMTP id cr7mr93303731wjc.19.1428011244242; Thu, 02 Apr 2015 14:47:24 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id l1sm120318wiy.20.2015.04.02.14.47.22 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Thu, 02 Apr 2015 14:47:22 -0700 (PDT) Date: Thu, 2 Apr 2015 23:47:20 +0200 From: Mateusz Guzik To: Yue Chen Subject: Re: How to traverse kernel threads? Message-ID: <20150402214720.GE549@dft-labs.eu> Mail-Followup-To: Mateusz Guzik , Yue Chen , Oliver Pinter , Benjamin Kaduk , "freebsd-hackers@freebsd.org" , HardenedBSD Core , PaX Team References: <20150321232622.GF14650@dft-labs.eu> <20150327194920.GB18158@dft-labs.eu> <20150330173358.GB9095@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: PaX Team , Benjamin Kaduk , HardenedBSD Core , Oliver Pinter , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 21:47:26 -0000 On Mon, Mar 30, 2015 at 10:59:16PM -0400, Yue Chen wrote: > > This would be a performance hit. > > The cost is to traverse the .text area and some other data in memory. It's > linear and only happens after a given time interval, like several minutes, > and does not put overhead on the normal execution. So the performance > overhead won't be high. > This was a remark about a runtime cost of executing kernel code with with an extra jump in every function. > > The only somewhat workable solution I see would require leaving function > entry points at constant addresses and only insert jumps to new locations. > > Yes, we do this. For the PIC direct jump/call/"instructions including > %RIP", we just need to update them. > I have trouble parsing what you wrote here. What I wrote here suggests taking existing foo_func, placing a copy at a new location and putting a jump to that address as the very first instruction of foo_func. (and zeroing/whatever the rest of old foo_func). Quite terrible. Also quite irrelevant to how you arrive at given func. Also see below. > > Also runtime relocation of everything definitely has a lot of unknown unknowns, > so all in all I would say this is a non-starter. > > FreeBSD kernel itself only has 1500+ indirect jumps. There are two types: > 1. jump table. Like switch-case. 2. Tail call. We have not found other > types like manual assembly code, or context restoration like that in > longjmp in libc. Maybe there are some other but we have not found. > Unknown unknowns have the problem of not being known. You have to be quite skilled in the area to come up with a working solution and I highly doubt either of us is such a person. > > Leave a lot of data at constant addresses, defeating the point to some > extent. > > One could consider a different approach where kernel data is randomly shuffled > around with some granularity and > > relevant symbol relocated prior to booting. > > Why do you think the data address exposure could be a serious problem? It > is non-executable and the code is randomized. Although for data itself, it > is really hard to protect from tampering and may cause other problems. > I see I wrote 'data', where I should have written '.text' or 'code', although from the context it should have been clear this refers to function entry points left behind. if you can get away with one function call, you don't need any infoleaks - you know the address of the function. That's one of the problems with leaving func entry points at known addresses. > Why the randomized data leads to kernel panic? I think if the data goes > somewhere else, the instruction addressing would follow, or page fault may > happen. If randomizing code, the exploit attempt may hit an illegal > instruction and the kernel panics. > What? Anyway I still don't understand how are you trying to achieve randomisation in the kernel. Previous mail suggests you just want to move funcs around and update all callsites and structs which store addresses to them. As outlined with 'struct meh' example (which can be found below) I do not believe this can work on real-world systems without a major surgery simply because there is no way to tell what looking like a pointer should be changed and what should not. I'm happy to be proved wrong. Kernel already leaks tons of addresses to userspace in documented ways, and I'm sure there are tons of situations where it does so as a result of a bug. I believe an alternative implementation I proposed which randomizes *all* kernel pages prior to booting it would be doable, but given aforementioned leaks not worth it. (well, striclty speaking you would want to move functions around with respect to each other, but keep the kernel physically contiguous so that you can still use superpages) Your previous mail in this thread suggests you are just starting any kind of work on kernel-level, which makes this very hard task even harder. Exploitation mitigation is a complicated subject on its own, I recommend reading what people were doing it have to say. Here is a piece about KASLR: https://forums.grsecurity.net/viewtopic.php?f=7&t=3367 In conclusion I strongly recommend dropping this project for the time being and focusing on something easier. > > return-oriented > > > > programming (ROP). It is basically a stronger form of ASLR. > > > > After each randomization procedure, the function return addresses > > saved in > > > > the stack are the ``old'' addresses before randomization, so we need to > > > > update them to the new addresses. > > > > That's why we need to get all the stack ranges to find those addresses. > > > > > > > > Also, in kernel, we believe that not only the return addresses in > > stacks > > > > need to be updated, there may exist other ``old'' saved instruction > > (PC) > > > > addresses in memory. Like in exception handling (maybe, do not know), > > > > debugging-purpose code and restartable atomic sequences (RAS) > > > > implementation. That's why I asked how to traverse all the kernel > > pages and > > > > get their virtual addresses here: > > > > > > https://lists.freebsd.org/pipermail/freebsd-hackers/2015-March/047336.html > > > > > > > > Now we found that it seems needed to traverse the ``pv_entry'' > > structure for > > > > x86_64 MMU. > > > > > > > > Another problem is that we do not know if FreeBSD has any form of > > special > > > > encodings or offset form for 64-bit instruction addresses (e.g., saved > > %RIP) > > > > on X86_64, instead of hard-coded addresses. For example, using a 32-bit > > > > offset instead of the 64-bit full address; and doing what glibc does > > for the > > > > setjmp/longjmp jmp_buf (special encodings (PTR_MANGLE) for the saved > > > > register values). > > > > > > > > Any suggestion or correction are highly appreciated. > > > > > > > > Best, > > > > Yue > > > > > > (Added HardenedBSD core and PaXTeam to CC.) > > > > > > Until you can not fixed all of the infoleaks from kernel (try sysctl > > > -a | grep 0x or similar command) the KASLR and other kernel address > > > space randomization techniques are easily bypass-able... > > > > > > > I do not believe this can be implemented reliably without some serious > > tinkering around the kernel. Surely I'm not a live patching expert (not > > any other kind of expert), but hear my out. > > > > It seems proposed approach is to move the kernel around and then updated > > all relevant pointers in all pages. > > > > I do not believe this can be done reliably without corrupting data, > > unless a major surgery is performed on the kernel. > > > > Consider: > > struct meh { > > func_t *m_func; > > size_t m_len; > > data *m_buf; > > }; > > > > Here we have 'm_func' which needs to be updated, but we don't know if > > that's the only pointer so we have to scan the entire struct. But m_buf > > can have data which looks like kernel pointers (i.e. matches some kernel > > funcs) and how will you know not to modify it? What if this is some sort > > of a hack and in fact you *should* modify it? > > > > CTF or some other solutions like that don't help if you just traverse > > all allocated buffers since you have no idea what the object you found > > really is. > > > > So, assuming this has to be done in runtime, the only somewhat workable > > solution I see would require leaving function entry points at contant > > addresses and only insert jumps to new locations. > > > > This would be a performance hit and would leave a lot of data at > > constant addresses, defeating the point to some extent. > > > > Also runtime relocation of everything definitely has a lot of unknown > > unknowns, so all in all I would say this is a non-starter. > > > > One could consider a different approach where kernel data is randomly > > shuffled around with some granularity and relevant symbol relocated > > prior to booting. > > > > This should provide unique enough layout, which paired with big > > likelyhood of a kernel panic on first bad exploit attempt may be an > > acceptable solution. > > > > But then the kernel may still have enough info leaks for this to not > > matter, so I would not be so eager to implement it. > > > > That said, what prompted this entire effort? Is there an operating > > system which got this to work or what? > > > > > > > > > > > > > > > > > > On Fri, Mar 27, 2015 at 3:49 PM, Mateusz Guzik > > wrote: > > > >> > > > >> On Fri, Mar 27, 2015 at 02:35:55PM -0400, Yue Chen wrote: > > > >> > When using the following code on kernel module loading: > > > >> > > > > >> > > > ------------------------------------------------------------------------------------------ > > > >> > struct thread *td = kdb_thr_first(); > > > >> > td = kdb_thr_next(td); > > > >> > > > > >> > > > ------------------------------------------------------------------------------------------ > > > >> > The kernel panics. > > > >> > > > > >> > > > >> Panics how? > > > >> > > > >> Also you can easily see these functions don't lock anything, so it > > would > > > >> be assumed you took appropriate locks. > > > >> > > > >> Except it seems there routines are supposed to be only used when > > > >> execution is 'frozen' (e.g. when escaped to the debugger). > > > >> > > > >> > > > > >> > And when printing all threads in proc0 (all kernel threads?): > > > >> > > > > >> > > > ------------------------------------------------------------------------------------------ > > > >> > struct proc *p = pfind(0); > > > >> > FOREACH_THREAD_IN_PROC(p, td) { > > > >> > uprintf("td: %x\n", td); > > > >> > } > > > >> > > > > >> > > > >> proc0 is an exported symbol, no need to pfind. > > > >> > > > >> > td = curthread; > > > >> > uprintf("cur td: %x\n", td); > > > >> > > > > >> > > > ------------------------------------------------------------------------------------------ > > > >> > The ``curthread'' (from this kernel module running the above code) > > is > > > >> > not > > > >> > in the 0 process group. > > > >> > > > > >> > > > >> There is no 'curthread from kernel module'. > > > >> > > > >> My guess is you do this work from module initializator, and in that > > case > > > >> curthread is the thread which loads the module, and such a thread is > > > >> definitely not linked into proc0. > > > >> > > > >> Still nobody knows what you are trying to do. > > > >> > > > >> -- > > > >> Mateusz Guzik > > > > > > > > > > > > -- > > Mateusz Guzik > > -- Mateusz Guzik From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 2 22:10:40 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 56928A2C; Thu, 2 Apr 2015 22:10:40 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ED0D2DB5; Thu, 2 Apr 2015 22:10:39 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t32MATC3059242 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 3 Apr 2015 01:10:29 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t32MATC3059242 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t32MAT9l059241; Fri, 3 Apr 2015 01:10:29 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 3 Apr 2015 01:10:29 +0300 From: Konstantin Belousov To: Tobias Oberstein Subject: Re: NVMe performance 4x slower than expected Message-ID: <20150402221029.GE2379@kib.kiev.ua> References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <551C6B62.7080205@gmail.com> <551D4E5F.9090400@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <551D4E5F.9090400@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: "freebsd-hackers@freebsd.org" , Michael Fuckner , Jim Harris , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 22:10:40 -0000 On Thu, Apr 02, 2015 at 04:12:47PM +0200, Tobias Oberstein wrote: > > You can also try a debug tunable that is in the nvme driver. > > > > hw.nvme.per_cpu_io_queues=0 > > I have rerun tests with kernel that has INVARIANTS off, and above sysctl > in loader.conf. > > Results are the same. > > vmstat now: > > root@s4l-zfs:~/oberstet # vmstat -ia | grep nvme > irq371: nvme0 8 0 > irq372: nvme0 7478 0 > irq373: nvme1 8 0 > irq374: nvme1 7612 0 > irq375: nvme2 8 0 > irq376: nvme2 7695 0 > irq377: nvme3 7 0 > irq378: nvme3 7716 0 > irq379: nvme4 8 0 > irq380: nvme4 7622 0 > irq381: nvme5 7 0 > irq382: nvme5 7561 0 > irq383: nvme6 8 0 > irq384: nvme6 7609 0 > irq385: nvme7 7 0 > irq386: nvme7 15373417 1174 > > === > > I was advised (off list) to run tests against a pure ramdisk. > > Here are results from a single socket E3: > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_ramdisk.md#xeon-e3-machine > > and here are results for the 48 core box > > https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_ramdisk.md#48-core-big-machine > > Performance with this box is 1/10 on this test compared to single socket E3! > > Something is severely wrong. It seems, there might be multiple issues > (not only NVMe). And this is after already running with 3 patches to > make it even boot. The speed of dd already looks wrong. Check the CPU frequency settings in BIOS, and check what sysctl dev.cpu reports. Ensure that cpufreq.ko is loaded from loader.conf. > > Well. I'm running out of ideas to try, and also patience with the users > waiting for this box =( > > /Tobias From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 3 03:25:28 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D948A529 for ; Fri, 3 Apr 2015 03:25:28 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id C3B84826 for ; Fri, 3 Apr 2015 03:25:28 +0000 (UTC) Received: from AlfredMacbookAir.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id A4F4D341F948 for ; Thu, 2 Apr 2015 20:25:28 -0700 (PDT) Message-ID: <551E0828.80305@mu.org> Date: Thu, 02 Apr 2015 20:25:28 -0700 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: Bazaaring the cathedral (Lowering the Barrier to Entry) References: , In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Apr 2015 03:25:28 -0000 On 4/2/15 10:44 AM, Chris H wrote: > On Thu, 2 Apr 2015 08:20:57 -0700 Samuel Cassiba wrote > >> Eitan, >> >> This being posted on April 1 sets off my BS-o-meter, but I'll bite since >> it's a topic worth shaving a yak or two over. >> >> WARNING: there be perceptions and opinions here >> >> Having been ephemerally associated with FreeBSD since early 4.x, I never > .. >> Again, I'm pretty sure this is just a April 1 troll, but it's all worth >> saying since the topic was brought up. Lowering the barrier to entry is >> good, but the act of doing so mustn't come at a detriment to quality. > OK. I'll bite... > > IMHO I believe that the height of the bar, is directly proportionate > to the quality of the product. > Depends on if that bar is based on quality of the submission or just bad process. -Alfred From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 3 07:25:37 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E973BFA2 for ; Fri, 3 Apr 2015 07:25:37 +0000 (UTC) Received: from st11p02mm-asmtp001.mac.com (st11p02mm-asmtp001.mac.com [17.172.220.236]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BD5CDE7D for ; Fri, 3 Apr 2015 07:25:37 +0000 (UTC) Received: from fukuyama.hsd1.ca.comcast.net (c-73-162-13-215.hsd1.ca.comcast.net [73.162.13.215]) by st11p02mm-asmtp001.mac.com (Oracle Communications Messaging Server 7.0.5.35.0 64bit (built Dec 4 2014)) with ESMTPSA id <0NM700CSWZAMFR10@st11p02mm-asmtp001.mac.com> for freebsd-hackers@freebsd.org; Fri, 03 Apr 2015 07:25:36 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.13.68,1.0.33,0.0.0000 definitions=2015-04-03_03:2015-04-03,2015-04-03,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1412110000 definitions=main-1504030067 Content-type: text/plain; charset=us-ascii MIME-version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: Free BSD Kernel Documentation From: Rui Paulo In-reply-to: Date: Fri, 03 Apr 2015 00:25:33 -0700 Content-transfer-encoding: quoted-printable Message-id: <4FAC738A-902D-4F18-968F-1732F9806463@me.com> References: To: Rakesh Sharma X-Mailer: Apple Mail (2.2070.6) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Apr 2015 07:25:38 -0000 On Apr 2, 2015, at 03:33, Rakesh Sharma wrote: >=20 > Can somebody please tell me link where i can find best documentation = for > freeBSD ARM Kernel There's no documentation specifically about FreeBSD/arm, but there are = lots of technical details about the kernel in The Design and = Implementation of the FreeBSD Operating System book. -- Rui Paulo From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 4 02:28:57 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 68661E2F; Sat, 4 Apr 2015 02:28:57 +0000 (UTC) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0070.outbound.protection.outlook.com [65.55.169.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F32092EB; Sat, 4 Apr 2015 02:28:56 +0000 (UTC) Received: from DM2PR0801MB0944.namprd08.prod.outlook.com (25.160.131.27) by DM2PR0801MB0941.namprd08.prod.outlook.com (25.160.131.24) with Microsoft SMTP Server (TLS) id 15.1.125.19; Sat, 4 Apr 2015 02:28:46 +0000 Received: from DM2PR0801MB0944.namprd08.prod.outlook.com ([25.160.131.27]) by DM2PR0801MB0944.namprd08.prod.outlook.com ([25.160.131.27]) with mapi id 15.01.0125.002; Sat, 4 Apr 2015 02:28:46 +0000 From: "Pokala, Ravi" To: "freebsd-hardware@freebsd.org" , "freebsd-hackers@freebsd.org" Subject: Booting off NVMe using traditional bootstrap? Thread-Topic: Booting off NVMe using traditional bootstrap? Thread-Index: AQHQbn8TNrsUb917tEKV87L2Z+DBHQ== Date: Sat, 4 Apr 2015 02:28:46 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.8.150116 x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [64.80.217.3] authentication-results: freebsd.org; dkim=none (message not signed) header.d=none; x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DM2PR0801MB0941; x-microsoft-antispam-prvs: x-forefront-antispam-report: BMV:1; SFV:NSPM; SFS:(10009020)(6009001)(164054003)(87936001)(2656002)(86362001)(83506001)(229853001)(107886001)(54356999)(50986999)(106116001)(99286002)(450100001)(92566002)(62966003)(77156002)(2501003)(2900100001)(102836002)(66066001)(46102003)(40100003)(558084003)(36756003); DIR:OUT; SFP:1101; SCL:1; SRVR:DM2PR0801MB0941; H:DM2PR0801MB0944.namprd08.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(601004)(5005006)(5002010); SRVR:DM2PR0801MB0941; BCL:0; PCL:0; RULEID:; SRVR:DM2PR0801MB0941; x-forefront-prvs: 0536638EAC Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: panasas.com X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Apr 2015 02:28:46.2585 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: acf01c9d-c699-42af-bdbb-44bf582e60b0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM2PR0801MB0941 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Apr 2015 02:28:57 -0000 Hi folks, Does anyone know off-hand if it's possible to boot (amd64) off of an NVMe device using the traditional bootstrap code (i.e. *not* UEFI)? Thanks, Ravi From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 4 11:15:03 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D89793BE; Sat, 4 Apr 2015 11:15:03 +0000 (UTC) Received: from mail-wi0-x22d.google.com (mail-wi0-x22d.google.com [IPv6:2a00:1450:400c:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7439DA9E; Sat, 4 Apr 2015 11:15:03 +0000 (UTC) Received: by widjs5 with SMTP id js5so39372630wid.1; Sat, 04 Apr 2015 04:15:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=cJ0Kb0MOGy4rnKd4QrO+T8rFYmRVyAn0KYkOvBkB8F0=; b=RMzBZSqjFKBI5NYIpYMoqsr7eWazACPGLw7g192AW90olI9Fe7zjyPd52cEsWGjxfq 9idKOixfkeQh/i7H5UHZ2Bte+FTO+TrvJ+U0Uj2g9kojdkOFYSGB6atQ1kEa1o/oWsjH hM9kdnJ3RflgLK+HJTGMz1MFUBFCz/zlQz+Iu/MlRkNQHaPkga5+LWCeqqKpgi5rOsu7 Led8FDqSuiVXmntxwiSYsqq4pxTyqWazJph9ZqodVZebbQ1xyy1jpLiyTmNLtOyX9HxP bEYamT33DDb1ex7CconJn9JJnAalVw0NObUBshc5wx3GWQsmcd2kOZxgSxOo7WwnTGel 0m3Q== X-Received: by 10.180.75.233 with SMTP id f9mr41263061wiw.5.1428146100937; Sat, 04 Apr 2015 04:15:00 -0700 (PDT) Received: from [192.168.1.141] (ppp-46-244-230-77.dynamic.mnet-online.de. [46.244.230.77]) by mx.google.com with ESMTPSA id i10sm15232145wja.40.2015.04.04.04.14.59 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 04 Apr 2015 04:15:00 -0700 (PDT) Message-ID: <551FC7B3.80203@gmail.com> Date: Sat, 04 Apr 2015 13:14:59 +0200 From: Tobias Oberstein User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: NVMe performance 4x slower than expected References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <551C6B62.7080205@gmail.com> <551D4E5F.9090400@gmail.com> <20150402221029.GE2379@kib.kiev.ua> In-Reply-To: <20150402221029.GE2379@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , Michael Fuckner , Jim Harris , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Apr 2015 11:15:03 -0000 >> I was advised (off list) to run tests against a pure ramdisk. >> >> Here are results from a single socket E3: >> >> https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_ramdisk.md#xeon-e3-machine >> >> and here are results for the 48 core box >> >> https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_ramdisk.md#48-core-big-machine >> >> Performance with this box is 1/10 on this test compared to single socket E3! >> >> Something is severely wrong. It seems, there might be multiple issues >> (not only NVMe). And this is after already running with 3 patches to >> make it even boot. > The speed of dd already looks wrong. Yes. > > Check the CPU frequency settings in BIOS, and check what sysctl dev.cpu > reports. Ensure that cpufreq.ko is loaded from loader.conf. It's loaded now, and CPU clock is at maximum: dev.cpu.0.freq: 3000 Unfortuanetly, performance (randomio/fio) did not change .. From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 4 20:25:13 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 407083C6; Sat, 4 Apr 2015 20:25:13 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E869E82B; Sat, 4 Apr 2015 20:25:12 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1YeUcq-0008wq-F0; Sat, 04 Apr 2015 23:25:04 +0300 Date: Sat, 4 Apr 2015 23:25:04 +0300 From: Slawa Olhovchenkov To: Adrian Chadd Subject: Re: irq cpu binding Message-ID: <20150404202504.GG74532@zxy.spb.ru> References: <20150328181026.GB23643@zxy.spb.ru> <20150328183147.GC23643@zxy.spb.ru> <20150328192505.GD23643@zxy.spb.ru> <20150331225947.GC23643@zxy.spb.ru> <20150401062259.GD23643@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Apr 2015 20:25:13 -0000 On Tue, Mar 31, 2015 at 11:57:27PM -0700, Adrian Chadd wrote: > I'll go take a look over the weekend at the non-RSS case. It shouldn't > be scheduling on the wrong CPU(s) if you've pinned things. Did you find the time for look? From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 4 22:09:57 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 95E06BEF; Sat, 4 Apr 2015 22:09:57 +0000 (UTC) Received: from mail-pd0-x234.google.com (mail-pd0-x234.google.com [IPv6:2607:f8b0:400e:c02::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 40E3F1A2; Sat, 4 Apr 2015 22:09:57 +0000 (UTC) Received: by pdbni2 with SMTP id ni2so1252888pdb.1; Sat, 04 Apr 2015 15:09:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=gLk2lt+UCmj2jB8DDtRaRY+aZIAqvKo+LoI0co80Ep0=; b=zROoRM/jeOULYo7INbmNxyskai/Au44DIhVqpqN6f5MNJpyhG+La3wv4loAWy5RIlu 1gNIqKjQFnhImRqD8mvnSS+lSXEMsxiRQDrMdEacVWYw2y63A2yM3pHklIjzxKN8lits Zo2a+XgkvpYenfIQkK+ZymH92UXajtDSoWkWnxKP95zTXcT3xtRK7zX49tDXB5HhoGoZ EQGiAJ6UynO+6wL+T89xIhpaEf8mjz3UsjTsoq4sqOv1r6J+lz+hkUvcr2fSSnTHPMqB XB0S39iCeHsAeoq/GFbXhu4k6X1VcszQ5XGqycbXwfhWsW5awDNNjXG3JvSmNUBd8qs8 h5Dg== X-Received: by 10.66.160.197 with SMTP id xm5mr15181566pab.51.1428185396598; Sat, 04 Apr 2015 15:09:56 -0700 (PDT) MIME-Version: 1.0 Received: by 10.67.2.42 with HTTP; Sat, 4 Apr 2015 15:09:26 -0700 (PDT) In-Reply-To: <20150402214720.GE549@dft-labs.eu> References: <20150321232622.GF14650@dft-labs.eu> <20150327194920.GB18158@dft-labs.eu> <20150330173358.GB9095@dft-labs.eu> <20150402214720.GE549@dft-labs.eu> From: Yue Chen Date: Sat, 4 Apr 2015 18:09:26 -0400 Message-ID: Subject: Re: How to traverse kernel threads? To: Mateusz Guzik , Yue Chen , Oliver Pinter , Benjamin Kaduk , "freebsd-hackers@freebsd.org" , HardenedBSD Core , PaX Team Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Apr 2015 22:09:57 -0000 Thank you so much for the suggestions and kind help. On Thu, Apr 2, 2015 at 5:47 PM, Mateusz Guzik wrote: > On Mon, Mar 30, 2015 at 10:59:16PM -0400, Yue Chen wrote: > > > This would be a performance hit. > > > > The cost is to traverse the .text area and some other data in memory. > It's > > linear and only happens after a given time interval, like several > minutes, > > and does not put overhead on the normal execution. So the performance > > overhead won't be high. > > > > This was a remark about a runtime cost of executing kernel code with > with an extra jump in every function. > > > > The only somewhat workable solution I see would require leaving > function > > entry points at constant addresses and only insert jumps to new > locations. > > > > Yes, we do this. For the PIC direct jump/call/"instructions including > > %RIP", we just need to update them. > > > > I have trouble parsing what you wrote here. > > What I wrote here suggests taking existing foo_func, placing a copy at a > new location and putting a jump to that address as the very first > instruction of foo_func. (and zeroing/whatever the rest of old > foo_func). Quite terrible. > > Also quite irrelevant to how you arrive at given func. > > Also see below. > > > > Also runtime relocation of everything definitely has a lot of unknown > unknowns, > > so all in all I would say this is a non-starter. > > > > FreeBSD kernel itself only has 1500+ indirect jumps. There are two types: > > 1. jump table. Like switch-case. 2. Tail call. We have not found other > > types like manual assembly code, or context restoration like that in > > longjmp in libc. Maybe there are some other but we have not found. > > > > Unknown unknowns have the problem of not being known. You have to be > quite skilled in the area to come up with a working solution and I > highly doubt either of us is such a person. > > > > Leave a lot of data at constant addresses, defeating the point to some > > extent. > > > One could consider a different approach where kernel data is randomly > shuffled > > around with some granularity and > > > relevant symbol relocated prior to booting. > > > > Why do you think the data address exposure could be a serious problem? It > > is non-executable and the code is randomized. Although for data itself, > it > > is really hard to protect from tampering and may cause other problems. > > > > I see I wrote 'data', where I should have written '.text' or 'code', > although from the context it should have been clear this refers to > function entry points left behind. > > if you can get away with one function call, you don't need any infoleaks > - you know the address of the function. That's one of the problems with > leaving func entry points at known addresses. > > > Why the randomized data leads to kernel panic? I think if the data goes > > somewhere else, the instruction addressing would follow, or page fault > may > > happen. If randomizing code, the exploit attempt may hit an illegal > > instruction and the kernel panics. > > > > What? > > Anyway I still don't understand how are you trying to achieve > randomisation in the kernel. > > Previous mail suggests you just want to move funcs around and update all > callsites and structs which store addresses to them. > > As outlined with 'struct meh' example (which can be found below) I do > not believe this can work on real-world systems without a major surgery > simply because there is no way to tell what looking like a pointer > should be changed and what should not. I'm happy to be proved wrong. > > Kernel already leaks tons of addresses to userspace in documented ways, > and I'm sure there are tons of situations where it does so as a result > of a bug. > > I believe an alternative implementation I proposed which randomizes > *all* kernel pages prior to booting it would be doable, but given > aforementioned leaks not worth it. > > (well, striclty speaking you would want to move functions around with > respect to each other, but keep the kernel physically contiguous so > that you can still use superpages) > > Your previous mail in this thread suggests you are just starting any > kind of work on kernel-level, which makes this very hard task even > harder. > > Exploitation mitigation is a complicated subject on its own, I recommend > reading what people were doing it have to say. Here is a piece about > KASLR: https://forums.grsecurity.net/viewtopic.php?f=7&t=3367 > > In conclusion I strongly recommend dropping this project for the time > being and focusing on something easier. > > > > return-oriented > > > > > programming (ROP). It is basically a stronger form of ASLR. > > > > > After each randomization procedure, the function return addresses > > > saved in > > > > > the stack are the ``old'' addresses before randomization, so we > need to > > > > > update them to the new addresses. > > > > > That's why we need to get all the stack ranges to find those > addresses. > > > > > > > > > > Also, in kernel, we believe that not only the return addresses in > > > stacks > > > > > need to be updated, there may exist other ``old'' saved instruction > > > (PC) > > > > > addresses in memory. Like in exception handling (maybe, do not > know), > > > > > debugging-purpose code and restartable atomic sequences (RAS) > > > > > implementation. That's why I asked how to traverse all the kernel > > > pages and > > > > > get their virtual addresses here: > > > > > > > > > https://lists.freebsd.org/pipermail/freebsd-hackers/2015-March/047336.html > > > > > > > > > > Now we found that it seems needed to traverse the ``pv_entry'' > > > structure for > > > > > x86_64 MMU. > > > > > > > > > > Another problem is that we do not know if FreeBSD has any form of > > > special > > > > > encodings or offset form for 64-bit instruction addresses (e.g., > saved > > > %RIP) > > > > > on X86_64, instead of hard-coded addresses. For example, using a > 32-bit > > > > > offset instead of the 64-bit full address; and doing what glibc > does > > > for the > > > > > setjmp/longjmp jmp_buf (special encodings (PTR_MANGLE) for the > saved > > > > > register values). > > > > > > > > > > Any suggestion or correction are highly appreciated. > > > > > > > > > > Best, > > > > > Yue > > > > > > > > (Added HardenedBSD core and PaXTeam to CC.) > > > > > > > > Until you can not fixed all of the infoleaks from kernel (try sysctl > > > > -a | grep 0x or similar command) the KASLR and other kernel address > > > > space randomization techniques are easily bypass-able... > > > > > > > > > > I do not believe this can be implemented reliably without some serious > > > tinkering around the kernel. Surely I'm not a live patching expert (not > > > any other kind of expert), but hear my out. > > > > > > It seems proposed approach is to move the kernel around and then > updated > > > all relevant pointers in all pages. > > > > > > I do not believe this can be done reliably without corrupting data, > > > unless a major surgery is performed on the kernel. > > > > > > Consider: > > > struct meh { > > > func_t *m_func; > > > size_t m_len; > > > data *m_buf; > > > }; > > > > > > Here we have 'm_func' which needs to be updated, but we don't know if > > > that's the only pointer so we have to scan the entire struct. But m_buf > > > can have data which looks like kernel pointers (i.e. matches some > kernel > > > funcs) and how will you know not to modify it? What if this is some > sort > > > of a hack and in fact you *should* modify it? > > > > > > CTF or some other solutions like that don't help if you just traverse > > > all allocated buffers since you have no idea what the object you found > > > really is. > > > > > > So, assuming this has to be done in runtime, the only somewhat workable > > > solution I see would require leaving function entry points at contant > > > addresses and only insert jumps to new locations. > > > > > > This would be a performance hit and would leave a lot of data at > > > constant addresses, defeating the point to some extent. > > > > > > Also runtime relocation of everything definitely has a lot of unknown > > > unknowns, so all in all I would say this is a non-starter. > > > > > > One could consider a different approach where kernel data is randomly > > > shuffled around with some granularity and relevant symbol relocated > > > prior to booting. > > > > > > This should provide unique enough layout, which paired with big > > > likelyhood of a kernel panic on first bad exploit attempt may be an > > > acceptable solution. > > > > > > But then the kernel may still have enough info leaks for this to not > > > matter, so I would not be so eager to implement it. > > > > > > That said, what prompted this entire effort? Is there an operating > > > system which got this to work or what? > > > > > > > > > > > > > > > > > > > > > > > On Fri, Mar 27, 2015 at 3:49 PM, Mateusz Guzik > > > wrote: > > > > >> > > > > >> On Fri, Mar 27, 2015 at 02:35:55PM -0400, Yue Chen wrote: > > > > >> > When using the following code on kernel module loading: > > > > >> > > > > > >> > > > > > ------------------------------------------------------------------------------------------ > > > > >> > struct thread *td = kdb_thr_first(); > > > > >> > td = kdb_thr_next(td); > > > > >> > > > > > >> > > > > > ------------------------------------------------------------------------------------------ > > > > >> > The kernel panics. > > > > >> > > > > > >> > > > > >> Panics how? > > > > >> > > > > >> Also you can easily see these functions don't lock anything, so it > > > would > > > > >> be assumed you took appropriate locks. > > > > >> > > > > >> Except it seems there routines are supposed to be only used when > > > > >> execution is 'frozen' (e.g. when escaped to the debugger). > > > > >> > > > > >> > > > > > >> > And when printing all threads in proc0 (all kernel threads?): > > > > >> > > > > > >> > > > > > ------------------------------------------------------------------------------------------ > > > > >> > struct proc *p = pfind(0); > > > > >> > FOREACH_THREAD_IN_PROC(p, td) { > > > > >> > uprintf("td: %x\n", td); > > > > >> > } > > > > >> > > > > > >> > > > > >> proc0 is an exported symbol, no need to pfind. > > > > >> > > > > >> > td = curthread; > > > > >> > uprintf("cur td: %x\n", td); > > > > >> > > > > > >> > > > > > ------------------------------------------------------------------------------------------ > > > > >> > The ``curthread'' (from this kernel module running the above > code) > > > is > > > > >> > not > > > > >> > in the 0 process group. > > > > >> > > > > > >> > > > > >> There is no 'curthread from kernel module'. > > > > >> > > > > >> My guess is you do this work from module initializator, and in > that > > > case > > > > >> curthread is the thread which loads the module, and such a thread > is > > > > >> definitely not linked into proc0. > > > > >> > > > > >> Still nobody knows what you are trying to do. > > > > >> > > > > >> -- > > > > >> Mateusz Guzik > > > > > > > > > > > > > > > > -- > > > Mateusz Guzik > > > > > -- > Mateusz Guzik > From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 4 23:27:21 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 81FFD41B for ; Sat, 4 Apr 2015 23:27:21 +0000 (UTC) Received: from mail-ie0-x22b.google.com (mail-ie0-x22b.google.com [IPv6:2607:f8b0:4001:c03::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41D31BC4 for ; Sat, 4 Apr 2015 23:27:21 +0000 (UTC) Received: by iedfl3 with SMTP id fl3so1190808ied.1 for ; Sat, 04 Apr 2015 16:27:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=F1QhfSbxo/qVLrNwSl1vG639mwfxi84guH6HttFgOFo=; b=ilOuXOBsZudkvmOky/XduR1QdHMmvY6kQ8ZYP5xVFm5ncZCmEyAtJgqzcbCVeOXUpT qZerIq2tZIhwKZx5NOVOjfJnRHtUarYeAMnXIOAsNaqHdAkZ9lgwtv1MIe+taqvl9dMR 8eQiNJe3112M2VkBvUmbKZqJU9gJ2XZl0AGqlKeZy1w90HV9OeTtT+xvsPP7i059LoQo UP1q+DzOS3HyH2RY+mpbtz2Uf/Jbm5Vq8JCbNMCgweoazQDpwsfPcUs7YhBCkThbVR7Z Wfj1V2ltCVjKSyuAvYo5pp6cT8mc63om+hRfKncx6OFldWKxoFfyNN/ZYb2oPnREvPTr EaCg== MIME-Version: 1.0 X-Received: by 10.42.137.202 with SMTP id z10mr1677634ict.37.1428190040515; Sat, 04 Apr 2015 16:27:20 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Sat, 4 Apr 2015 16:27:20 -0700 (PDT) In-Reply-To: <20150404202504.GG74532@zxy.spb.ru> References: <20150328181026.GB23643@zxy.spb.ru> <20150328183147.GC23643@zxy.spb.ru> <20150328192505.GD23643@zxy.spb.ru> <20150331225947.GC23643@zxy.spb.ru> <20150401062259.GD23643@zxy.spb.ru> <20150404202504.GG74532@zxy.spb.ru> Date: Sat, 4 Apr 2015 16:27:20 -0700 X-Google-Sender-Auth: agI5eP8ABmczG2Fl5q5XbyxgsQY Message-ID: Subject: Re: irq cpu binding From: Adrian Chadd To: Slawa Olhovchenkov Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Apr 2015 23:27:21 -0000 On 4 April 2015 at 13:25, Slawa Olhovchenkov wrote: > On Tue, Mar 31, 2015 at 11:57:27PM -0700, Adrian Chadd wrote: > >> I'll go take a look over the weekend at the non-RSS case. It shouldn't >> be scheduling on the wrong CPU(s) if you've pinned things. > > Did you find the time for look? Not yet; I've been validating things and adding prefetching/batching to my testing setup, which uses ixgbe + netmap. I'll get to it soon. Thanks -adrian From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 4 23:48:46 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 11B40A70; Sat, 4 Apr 2015 23:48:46 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 717D5DA2; Sat, 4 Apr 2015 23:48:45 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t34NmdYk083083 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 5 Apr 2015 02:48:39 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t34NmdYk083083 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t34NmdkV083082; Sun, 5 Apr 2015 02:48:39 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 5 Apr 2015 02:48:39 +0300 From: Konstantin Belousov To: Tobias Oberstein Subject: Re: NVMe performance 4x slower than expected Message-ID: <20150404234839.GM2379@kib.kiev.ua> References: <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <551C6B62.7080205@gmail.com> <551D4E5F.9090400@gmail.com> <20150402221029.GE2379@kib.kiev.ua> <551FC7B3.80203@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <551FC7B3.80203@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: "freebsd-hackers@freebsd.org" , Michael Fuckner , Jim Harris , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Apr 2015 23:48:46 -0000 On Sat, Apr 04, 2015 at 01:14:59PM +0200, Tobias Oberstein wrote: > >> I was advised (off list) to run tests against a pure ramdisk. > >> > >> Here are results from a single socket E3: > >> > >> https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_ramdisk.md#xeon-e3-machine > >> > >> and here are results for the 48 core box > >> > >> https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_ramdisk.md#48-core-big-machine > >> > >> Performance with this box is 1/10 on this test compared to single socket E3! > >> > >> Something is severely wrong. It seems, there might be multiple issues > >> (not only NVMe). And this is after already running with 3 patches to > >> make it even boot. > > The speed of dd already looks wrong. > > Yes. Try some simple checks for the performance anomalies. E.g., verify the raw memory bandwidth, check that it is in the expected range. Quick search popped up the following tool, e.g.: https://zsmith.co/bandwidth.html > > > > > Check the CPU frequency settings in BIOS, and check what sysctl dev.cpu > > reports. Ensure that cpufreq.ko is loaded from loader.conf. > > It's loaded now, and CPU clock is at maximum: dev.cpu.0.freq: 3000 > > Unfortuanetly, performance (randomio/fio) did not change ..