Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 15 Apr 2017 07:00:57 -0700 (PDT)
From:      "Rodney W. Grimes" <freebsd@pdx.rh.CN85.dnsmgr.net>
To:        Slawa Olhovchenkov <slw@zxy.spb.ru>
Cc:        Mark Johnston <markj@freebsd.org>, Xin LI <delphij@gmail.com>, "svn-src-head@freebsd.org" <svn-src-head@freebsd.org>, "svn-src-all@freebsd.org" <svn-src-all@freebsd.org>, "src-committers@freebsd.org" <src-committers@freebsd.org>, Ngie Cooper <ngie@freebsd.org>, Alan Somers <asomers@freebsd.org>
Subject:   Re: svn commit: r316938 - head/sbin/savecore
Message-ID:  <201704151400.v3FE0vXk012250@pdx.rh.CN85.dnsmgr.net>
In-Reply-To: <20170415083952.GA83631@zxy.spb.ru>

next in thread | previous in thread | raw e-mail | index | archive | help
> On Fri, Apr 14, 2017 at 03:05:25PM -0700, Mark Johnston wrote:
> 
> > > And with textdumps available, the benefit
> > > of having compression is limited because we can request for minidump
> > > or full dumps only when the textdumps are not good enough for
> > > diagnosing the kernel bug.
> > 
> > Sure, but in this case the compression may be vital.
> > 
> > > 
> > > I don't think security (e.g. leaking information because of the use of
> > > compression) is a very big concern in this context because in order
> > > for the potential attacker to read the raw material needs a
> > > compromised system (unlike an attack from the network, where someone
> > > who controls the network would have access to the raw material); the
> > > dump is usually quite large, and measuring downtime would be hard at
> > > that scale.
> > 
> > Ok.
> > 
> > > 
> > > By the way (not meant to bikeshed) if I was to do this I'd prefer
> > > using lz4 or something that compresses faster than zlib.
> > 
> > I agree, but I think the existing lz4 implementation in the kernel is
> > not so well suited to running after a panic. It seems fixable, but
> > compression speed also isn't hugely important here IMO.
> 
> On production system this is downtime.
> For may case, dumped about 32GB (from 256GB RAM). This is take several
> minutes. Can compression increase this to hour?

On productions systems the compression layer of dump may very well
be a win situation depending on choosen algorith (you want something
fairly fast, but still effective).  If your rate to compress bytes
is close to the disk write bandwith you have an over all win caused
by writting less to disk.

Someone who enjoys math should write an equation for given
compression bandwidth cb and given disk bandwidth db and
compression ratio cr what do the curves look like? 

IIRC we measure cpu/memory bandwidth in the 10'sG bytes/sec
range, compression should be some place under that, and
our disk bandwidth even on SSD is in the 500MB range,
even the fastest 15k rpm spinning rust is in the <200MB
range, we should be able to compress at a higher rate than
this.

-- 
Rod Grimes                                                 rgrimes@freebsd.org



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201704151400.v3FE0vXk012250>