From owner-freebsd-embedded@freebsd.org Tue Jun 25 22:25:05 2019 Return-Path: Delivered-To: freebsd-embedded@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E638E15D8B67 for ; Tue, 25 Jun 2019 22:25:04 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound3d.ore.mailhop.org (outbound3d.ore.mailhop.org [54.186.57.195]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 27F69816FC for ; Tue, 25 Jun 2019 22:25:03 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1561500530; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=MUPvQrTVNUnJlMC/AOEXIMNA7p5jmoja+OucGJX8jBWLo6E2bjpUiKL10IN+r/xiHhrIQsYGjEqU0 dWSWQ/m5A3IVr9NuRjNWgzLxdR0e9vxL+RqeVsmzPfuUlUeNQegmlCzdqGDChtx0PgBqjJWn70d34t Z8Ili4uBEGOtjXhERd36R35AC5LtMmc8pLMmVTnen4jx1xC3n9fqSTaYTugkWY+Biq4aDpcRPauNir fPdzBJIHj66m1Uj6PV3qyPH/Q7gno6rUzd15FAJVXoRdPXMkqgFv18xtpHrGQns+iCJDfKOQIgzuUX f+OlI2zNx8PXXF1UjJg85AidJ7CEXRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:date:to:from:subject: message-id:dkim-signature:from; bh=WJG2hMVzBnzmYNSC3bOxg2T5h6pcX7Vnf9WMM+YKJuI=; b=MNt0SuCFkih/FZyCMMx/jp/jKcV3s9vGZz/we2ebxopvmhIXIWnlqXhgZZRlAKJnAxS3KUiSzFw4o 5nIsimUz/4KU1gdv7Ty82cupEej3OBhltFbjytUXZaxSVR6Aeepr474iKj4XbxFxAf4VGzg425k1qk H5xhCEOvRUbGnbsBYhYkuuODr7036TF8JJCMxy11t5jWyvzVp2SUg+NdmUG1Tr4zEQ8//TsELNT12H cT3NnhxZYySP0CcqkbYX+s3ayfjPEfx+AsxEvJy27svOhxdciXiDId0nghhV7A3tb863W52GMg5HbB Tg1kXefUBAX8+1NUz6P6Iafh6tf0VRg== ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:date:to:from:subject: message-id:from; bh=WJG2hMVzBnzmYNSC3bOxg2T5h6pcX7Vnf9WMM+YKJuI=; b=B3wGS8Ov5w/XQwbzrOf8BiHk/t8s+8/Slb+ts8yiQDN9uqmcYxzuUxjjvT3AYZaFw2jwvMIp+fAf+ LZint79JNvJrcq67135xdMpOGHIRbIBlKLcCtO7EqxLD5tkJcu9sRdBrIV/ihdrL4o8wj00KV5Nrys XHo5p+OCfRC4i16p5ajxbxAf2TiFYVZYYO9yIM8WoJPWBEHYoCS/Vl9hmC6cKnNSqWVUWz8ETOVIVJ W78bnrei8c9Dh9izbOaQ2Z4KFRfgaSUfPvWrx8XSTZYnnYqo+R6lWMm50GeRWDni9fOP51skBdrG1U OgP2Ald1sq9qukmdPGSZ+Ezxofahtzg== X-MHO-RoutePath: aGlwcGll X-MHO-User: ce720a12-9795-11e9-a4fc-d9033e875444 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound3.ore.mailhop.org (Halon) with ESMTPSA id ce720a12-9795-11e9-a4fc-d9033e875444; Tue, 25 Jun 2019 22:08:49 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x5PM8lPF048202; Tue, 25 Jun 2019 16:08:47 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: Subject: Limiting the size of syslogd output files using options in syslog.conf From: Ian Lepore To: freebsd-embedded@freebsd.org, freebsd-hackers@freebsd.org Date: Tue, 25 Jun 2019 16:08:47 -0600 Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 27F69816FC X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.985,0]; ASN(0.00)[asn:16509, ipnet:54.186.0.0/15, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jun 2019 22:25:05 -0000 I've posted a review of a small syslogd change which lets you set a limit on the size of syslogd output files in /etc/syslog.conf. The idea is to prevent filling up a filesystem on emmc or sdcard or other small storage device on an embedded system with unexpected logging triggered by some error or failing hardware. If you're interested, please see the review at https://reviews.freebsd.org/D20764 -- Ian From owner-freebsd-embedded@freebsd.org Tue Jun 25 23:07:09 2019 Return-Path: Delivered-To: freebsd-embedded@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50B3D15D94B4 for ; Tue, 25 Jun 2019 23:07:09 +0000 (UTC) (envelope-from kudzu@tenebras.com) Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com [IPv6:2607:f8b0:4864:20::736]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 26FEB827FF for ; Tue, 25 Jun 2019 23:07:08 +0000 (UTC) (envelope-from kudzu@tenebras.com) Received: by mail-qk1-x736.google.com with SMTP id m14so102427qka.10 for ; Tue, 25 Jun 2019 16:07:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tenebras-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=kjt6kvVJSeUdqdD6RxnC2je+HannTYuOBNf0LXQnnbU=; b=1P8p+UCs0FP3ETzCEjmoth8LTXn/5uZJoS06KHpl5iwPjQyctJ5KUkqI+H2qEd9eMj 4oU2CFgn6FQvzbc9hmst+WmS7CvUQZeXtqvVfuwScubrGAnwedDUH/LIpPBqgiou2swe PIT4bGm36WRQbJko4xqvQkc+jzXcCm9jhxX62+ZZRaUFpewxDEM8T7JqlPUdSeEAhnvY NdwHB3SGNcjTIqTK9Ag37mXP39oXpidVlEaz8vWa6YllKEw6TGd6/qu6BwQFYDqonntT PfZ3Xk72o75CClDYWXhEwtYVZvl5O2unu/hDkrEGRSEQR7Ovzhhz3b8EBnvobrDsjzIv nKjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=kjt6kvVJSeUdqdD6RxnC2je+HannTYuOBNf0LXQnnbU=; b=qfBvgUy5FWanyyBVcLt5EGSAsRaDpuot/z3VeWjfmoFQ0srl6TlTgcMAUQQUsOCKlp CxT0sVM2K5duzZMtB3r3gBTEPk2RfRzrBdYeKsD6xGKIbA2mRSbWTuaaTnp5cZdy3ZEh Zlur1T/Fo4m7v1eUpAE0UHbzVX+cjUyna91Zbh8XApwN58WD43uD5O8GWVPZhWOxbVkH V6XnY1QkQoppNI/2zzGrdbSSdTN08uRYkgZWG5EM0J9TGBz2dMss+UGRFCgUsSfeFf+M cGtpARbUDKHK7HYAldRRHnaaHO/UQfB5jhBTr1u0I1vqevLtdE0Kptf+DCCQA8K+I5sL b0IA== X-Gm-Message-State: APjAAAWZ5evX2q+czzTtB49fQ7ndHAeIjXjdDzvsn5GT5BckHS12sweP KLAnrty8AMnT4sHMYFWfqIn8YB92/+48FgVReV2fug== X-Google-Smtp-Source: APXvYqwyvGf0P7x6P16tl0G3i6C276ps2PYG6fsRLCC6SdfK1F6/kfICj8xcvlalXQHXmSMGpeEwNTA+HL74yBDyD7o= X-Received: by 2002:ae9:ee17:: with SMTP id i23mr1074901qkg.399.1561504027270; Tue, 25 Jun 2019 16:07:07 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Michael Sierchio Date: Tue, 25 Jun 2019 16:06:30 -0700 Message-ID: Subject: Re: Limiting the size of syslogd output files using options in syslog.conf To: Ian Lepore Cc: freebsd-embedded@freebsd.org, freebsd-hackers@freebsd.org X-Rspamd-Queue-Id: 26FEB827FF X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=tenebras-com.20150623.gappssmtp.com header.s=20150623 header.b=1P8p+UCs X-Spamd-Result: default: False [-6.28 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[tenebras-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-embedded@freebsd.org]; DMARC_NA(0.00)[tenebras.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: alt1.aspmx.l.google.com]; DKIM_TRACE(0.00)[tenebras-com.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[6.3.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-2.99)[ip: (-9.43), ipnet: 2607:f8b0::/32(-3.14), asn: 15169(-2.33), country: US(-0.06)] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jun 2019 23:07:09 -0000 Apologies, but I don't understand the point of this at all. I object especially to idea of patching syslogd when EVERYTHING you seek can be accomplished with appropriate settings in /etc/newsyslog.conf and changing its entry in /etc/crontab to run more frequently. You can control the size and number of files there. You can also do remote syslog. Do you need to rotate logs more than once per minute? On Tue, Jun 25, 2019 at 3:25 PM Ian Lepore wrote: > I've posted a review of a small syslogd change which lets you set a > limit on the size of syslogd output files in /etc/syslog.conf. The > idea is to prevent filling up a filesystem on emmc or sdcard or other > small storage device on an embedded system with unexpected logging > triggered by some error or failing hardware. > > If you're interested, please see the review at > > https://reviews.freebsd.org/D20764 > > -- Ian > > _______________________________________________ > freebsd-embedded@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-embedded > To unsubscribe, send any mail to "freebsd-embedded-unsubscribe@freebsd.or= g > " > --=20 "Well," Brahm=C4=81 said, "even after ten thousand explanations, a fool is = no wiser, but an intelligent person requires only two thousand five hundred." - The Mah=C4=81bh=C4=81rata From owner-freebsd-embedded@freebsd.org Wed Jun 26 05:07:12 2019 Return-Path: Delivered-To: freebsd-embedded@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBE7C15B34D5; Wed, 26 Jun 2019 05:07:12 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 542568D511; Wed, 26 Jun 2019 05:07:10 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (v-critter.freebsd.dk [192.168.55.3]) by phk.freebsd.dk (Postfix) with ESMTP id EC9042025698; Wed, 26 Jun 2019 05:07:02 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.15.2/8.15.2) with ESMTPS id x5Q572RL030084 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 26 Jun 2019 05:07:02 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.15.2/8.15.2/Submit) id x5Q572Zk030083; Wed, 26 Jun 2019 05:07:02 GMT (envelope-from phk) To: Ian Lepore cc: freebsd-embedded@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: Limiting the size of syslogd output files using options in syslog.conf In-reply-to: From: "Poul-Henning Kamp" References: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <30081.1561525622.1@critter.freebsd.dk> Content-Transfer-Encoding: quoted-printable Date: Wed, 26 Jun 2019 05:07:02 +0000 Message-ID: <30082.1561525622@critter.freebsd.dk> X-Rspamd-Queue-Id: 542568D511 X-Spamd-Bar: +++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [3.73 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(0.96)[0.958,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.dk]; AUTH_NA(1.00)[]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_MEDIUM(0.46)[0.462,0]; RCVD_COUNT_THREE(0.00)[4]; IP_SCORE(0.06)[ip: (0.11), ipnet: 130.225.0.0/16(0.06), asn: 1835(0.16), country: EU(-0.00)]; MX_GOOD(-0.01)[phk.freebsd.dk]; NEURAL_SPAM_LONG(0.96)[0.956,0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[phk@phk.freebsd.dk,phk@critter.freebsd.dk]; RCVD_NO_TLS_LAST(0.10)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:1835, ipnet:130.225.0.0/16, country:EU]; FROM_NEQ_ENVFROM(0.00)[phk@phk.freebsd.dk,phk@critter.freebsd.dk] X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 05:07:13 -0000 -------- In message , I= an Le pore writes: >I've posted a review of a small syslogd change which lets you set a >limit on the size of syslogd output files in /etc/syslog.conf. The >idea is to prevent filling up a filesystem on emmc or sdcard or other >small storage device on an embedded system with unexpected logging >triggered by some error or failing hardware. You should consider fifolog(1) in such environments. -- = Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe = Never attribute to malice what can adequately be explained by incompetence= . From owner-freebsd-embedded@freebsd.org Wed Jun 26 14:50:10 2019 Return-Path: Delivered-To: freebsd-embedded@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 76CE315CA162 for ; Wed, 26 Jun 2019 14:50:10 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1a.eu.mailhop.org (outbound1a.eu.mailhop.org [52.58.109.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AF4CD70AEE for ; Wed, 26 Jun 2019 14:50:09 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1561560600; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=n3Sp9Bz2Q8BtuPXKP//iQT4ilGneVrf0awnoLzeZPd8ojYB175v27paIbbEsZE+xcGf8Qs0UuZoDF yZg1tQJDHiHSYBt6N8APqD6GsQwpvwKzTt89EUGkBxbBlrDMDmAdQTzDjbdJnYJWQgYr+TcdZeK7LX Q7u+SX5aP2jo/SftH88olSlle8xr84MQu+q8RZnx9Icc/aSwoPZREVtfoetRjN5rh3ObG2xWqm54Cg o+xww7NjpLCAVc/Tg2zMAj0I9seCutHWaQZ9E/5pCKmLsptuRkd4tn5KqX0lzy1mwE87dt0uqtFs5/ t21HUHtOH4657gAnSa2BP7snJC6VXyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=mShiKk6bGlJpZdxfTAZKtDSSxUYY1w81EfM2mEQkUAY=; b=GCpaLpSjFi/S+B7powMn4wQRm/0BE9/Z3slJwnhlTKmYtrkgAzM45EMlOKRmrZLT4ZcYgHrRWFWbV 37+N5rIfw3DchNIQQsDvKDb6bjdLPcf8a1fpGCLIN5ZI20pUyzS91PwrvVFKbuZ4/1T0Z2ztnRGB8U iOp6wT5dqWYjRFKMHjLJLiyb3XbEXpvTpqMGxaOgfCde6hgtsp4sw54C8H6dBQI0w24GRvsbjPhOQf nIFP6JuCVUCVWVWxN92icCC+vAxVJsmlhUk5I03KHwfiyxGJHFZxe1M4rJPCEQKsXIAg+wohgr4jzv 5Kd0FGZeyFXGpKVJqcXpNhohPiAOFZg== ARC-Authentication-Results: i=1; outbound2.eu.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=mShiKk6bGlJpZdxfTAZKtDSSxUYY1w81EfM2mEQkUAY=; b=tiISJ2kQw+SP4ikJCjZ7Hyu/EtBlfgNiE2bB/Y/meuIm/94qKjoL/JE8Q3CuBd2PM4TSB46rjarZo HsnnFoSXMPPHw2htQbIyQv+1FdDcZ9J54LLwY8uHz3qNx2onL0hkA9nHmnALvWWzDNyK6dKQ50GMJu KzXX8GQ4IFv4qQh3UgqfyGUTrDQ6hiy2vGn2h4Yeubqt0NNRGA28nGpcZga4lN4FBogajQVCT1+ogi erGmYmWY5rC5ntNWcDYEDyHSjcFhmiJzmIPStlMMVA5TjCibItxpoleTnaKOCf8oFsqN8Qr7WBy171 8aMI++i14OZW+2nhAExldAk6zSDtpJA== X-MHO-RoutePath: aGlwcGll X-MHO-User: aa1e52f3-9821-11e9-bbea-3f55ba53af12 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound2.eu.mailhop.org (Halon) with ESMTPSA id aa1e52f3-9821-11e9-bbea-3f55ba53af12; Wed, 26 Jun 2019 14:49:58 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x5QEnuVK051004; Wed, 26 Jun 2019 08:49:56 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: Subject: Re: Limiting the size of syslogd output files using options in syslog.conf From: Ian Lepore To: Michael Sierchio Cc: freebsd-hackers@freebsd.org, freebsd-embedded@freebsd.org Date: Wed, 26 Jun 2019 08:49:56 -0600 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: AF4CD70AEE X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.978,0]; ASN(0.00)[asn:16509, ipnet:52.58.0.0/15, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 14:50:10 -0000 On Tue, 2019-06-25 at 16:06 -0700, Michael Sierchio wrote: > Apologies, but I don't understand the point of this at all. I object > especially to idea of patching syslogd when EVERYTHING you seek can > be > accomplished with appropriate settings in /etc/newsyslog.conf and > changing > its entry in /etc/crontab to run more frequently. You can control the > size > and number of files there. > > You can also do remote syslog. > > Do you need to rotate logs more than once per minute? > > Newsyslog handles expected log growth. I guess you've never had a hardware device or other problem unexpectedly trigger the writing of hundreds or thousands of lines of output per minute to syslog? It happens. You write apps that try to strike a sensible balance between logging important events and not logging so much it leads to problems, then something unexpected happens and your carefully crafted logging starts happening at 200hz and actually contributes to the problem. We've been using this code at $work since 2003; it appears in dozens of different products ranging from lab instruments to national timescale systems. People using the products experiencing such problems just want the device to keep working as best it can. They don't want repeated mysterious reboots because of things like /var filling up. (In the case of products like lab instruments, they don't really perceive the thing to be a computer running an OS at all.) And most of all, when something does go wrong and they get our support engineers to look into it, they don't want to hear "We can't figure out what happened originally to cause this problem because all the logging got rotated out of existence." Newsyslog does exactly the opposite of what you want in that situation... it purposely destroys the valuable information about the original problem while carefully preserving the useless fallout that follows. Over the years we've upstreamed every freebsd customization we've ever done at $work except this one. The only reason I've been carrying this one local mod is because it seemed too specialized to be useful to anyone else. Then yesterday someone working on an embedded-system product asked me on irc how to solve the problem of unexpected log spewage, so I offered the solution we've been using. -- Ian > > On Tue, Jun 25, 2019 at 3:25 PM Ian Lepore wrote: > > > I've posted a review of a small syslogd change which lets you set a > > limit on the size of syslogd output files in /etc/syslog.conf. The > > idea is to prevent filling up a filesystem on emmc or sdcard or > > other > > small storage device on an embedded system with unexpected logging > > triggered by some error or failing hardware. > > > > If you're interested, please see the review at > > > > https://reviews.freebsd.org/D20764 > > > > -- Ian > > > > _______________________________________________ > > freebsd-embedded@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-embedded > > To unsubscribe, send any mail to " > > freebsd-embedded-unsubscribe@freebsd.org > > " > > > > From owner-freebsd-embedded@freebsd.org Wed Jun 26 14:54:33 2019 Return-Path: Delivered-To: freebsd-embedded@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4F3E15CA45F for ; Wed, 26 Jun 2019 14:54:32 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound2m.ore.mailhop.org (outbound2m.ore.mailhop.org [54.149.155.156]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2387570F49 for ; Wed, 26 Jun 2019 14:54:31 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1561560865; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=MCsNhefG3iLH5NGDVzFhLaLfu7jXc9Ou7SzgzmD7+wvvO2bvMSTh5fInrnSrl34P0EHdGBAQiW0Y+ XkNbHzF7VeI3gkJ5adUy107kBzmz6ZosjcdIzpxoeJvh5yqnCuMvsVNrtGNgYp1xgvDoT2d8yeP3rP VIv8GmBDRBAUHhdfs6Txbb8o6ALqCWDgu4GWT6PLpGkD8PUulygecurF+9wHhtagMbuoidTYtby8hm 6kBiRfU2FTLxbdA8tTqJBGENzLvInxdMqNJtitoaWHJZqHCUJ5AeqDVugbj9WdZHSrgzuXzyypT3vz uEf3zjLyHw8cP08d0zq9Y0paBYsW2VA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:dkim-signature:from; bh=3nL4t5dWWVna4zBaWNEvII39UnwWgBYtk1N5wXDFIWo=; b=f9G8c4OoDwEy5wnk8+y5g4IpQes9alwHgNsxsQ8ctXFKDP5P5cmP0AX+nf2y2zprDMcrDEQGNfVba V0mzXNC2C3Lp2II932jFahJeyciT+tYXcIYA+GDWqy5TTMAv/FzpfxVAQTEaAS5XRrCApjsKG824V0 TfoEJ8Pbr9fWQWC/zNR6gPK9Jusaxrg+xDT9LkHe6N9GltPWZaqFSwdUPbBU11uuZll84sH20VR3iJ ov5yOhwvzepYVd+95quPnVkKakgZ53miCbZGBPQVeDY6F6Y89brzUtCZs1M1eKI4+0/tkoJ+YMBJyj M/TD5BP4ge7jywhWP8qUhmIMQ8p8jCg== ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:cc:to:from:subject:message-id:from; bh=3nL4t5dWWVna4zBaWNEvII39UnwWgBYtk1N5wXDFIWo=; b=vBUEhpdWH4ighJPiaY0YVcdmgla1A63UjT3bdm1ckDHw0m2PprXi1F2M4ODJSJMRxeyeO6FabrBOa Gyuv4mTrEo8CLWSK2nfQR8oJD8fQT4WyI5PuqO3xKRdGgpoxyVVtk+0otLP3S/m7Q1Z0uYoydNkMoL X9BxD87oHaWQnmEdaGzJSn0yrC8Mn8NAXFq17G7N7Ha79gy9IIb6KhLP3/Vm3CT3umh9mJ3Kwamw05 mx7PKVzFK0fP1l8jfM88Alyx9YPJqsdt+fGtmIuRWv01+kQTByiNyJbMGtels4Sw9kGf6lfmfVCD3t 0u/PuJABTwFksRFNeJXf00RRp6riL/A== X-MHO-RoutePath: aGlwcGll X-MHO-User: 49388d28-9822-11e9-8d98-0b538189c249 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound4.ore.mailhop.org (Halon) with ESMTPSA id 49388d28-9822-11e9-8d98-0b538189c249; Wed, 26 Jun 2019 14:54:24 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id x5QEsNN1051017; Wed, 26 Jun 2019 08:54:23 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: Subject: Re: Limiting the size of syslogd output files using options in syslog.conf From: Ian Lepore To: Poul-Henning Kamp Cc: freebsd-hackers@freebsd.org, freebsd-embedded@freebsd.org Date: Wed, 26 Jun 2019 08:54:23 -0600 In-Reply-To: <30082.1561525622@critter.freebsd.dk> References: <30082.1561525622@critter.freebsd.dk> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 2387570F49 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.992,0]; ASN(0.00)[asn:16509, ipnet:54.148.0.0/15, country:US] X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 14:54:33 -0000 On Wed, 2019-06-26 at 05:07 +0000, Poul-Henning Kamp wrote: > -------- > In message < > bf597c3830ebabbef6ac422a5d648e1eed13fac5.camel@freebsd.org>, Ian Le > pore writes: > > I've posted a review of a small syslogd change which lets you set a > > limit on the size of syslogd output files in /etc/syslog.conf. The > > idea is to prevent filling up a filesystem on emmc or sdcard or > > other > > small storage device on an embedded system with unexpected logging > > triggered by some error or failing hardware. > > You should consider fifolog(1) in such environments. > > fifolog looks pretty cool and I had no idea it existed, so thanks for the pointer; I may find useful things to do with it. But for the problem of unexpected syslog spewage it has exactly the same problem as running newsyslog very frequently: by design it discards the information you most want preserved (the triggering event that led to unexpected log spewage) while carefully preserving all the following noise which is typically more about symptoms rather than causes of the problem. -- Ian From owner-freebsd-embedded@freebsd.org Wed Jun 26 15:07:59 2019 Return-Path: Delivered-To: freebsd-embedded@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6FC215CA9DB for ; Wed, 26 Jun 2019 15:07:59 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 62BE07168B for ; Wed, 26 Jun 2019 15:07:53 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id 3009F211089 for ; Wed, 26 Jun 2019 11:07:47 -0400 (EDT) Received: from [192.168.10.24] (D14.Denninger.Net [192.168.10.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id A78EEE1B9D for ; Wed, 26 Jun 2019 10:07:46 -0500 (CDT) Subject: Re: Limiting the size of syslogd output files using options in syslog.conf To: freebsd-embedded@freebsd.org References: From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: <3feff28d-0b91-a6fc-b0e9-161d3d315a78@denninger.net> Date: Wed, 26 Jun 2019 10:07:45 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms000609070701040603050202" X-Rspamd-Queue-Id: 62BE07168B X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.52 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[px.denninger.net]; NEURAL_HAM_SHORT(-0.69)[-0.690,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-embedded@freebsd.org]; DMARC_NA(0.00)[denninger.net]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; R_SPF_NA(0.00)[]; IP_SCORE(-2.62)[ip: (-9.86), ipnet: 104.236.64.0/18(-4.30), asn: 14061(1.12), country: US(-0.06)] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 15:08:00 -0000 This is a cryptographically signed message in MIME format. --------------ms000609070701040603050202 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 6/26/2019 09:49, Ian Lepore wrote: > > Over the years we've upstreamed every freebsd customization we've ever > done at $work except this one. The only reason I've been carrying this= > one local mod is because it seemed too specialized to be useful to > anyone else. Then yesterday someone working on an embedded-system > product asked me on irc how to solve the problem of unexpected log > spewage, so I offered the solution we've been using. > > -- Ian Yep -- this is a small corner case /but if it happens /in the embedded world (and I've had it happen) it's a five-alarm pain in the neck. I'm in favor for that reason, and I can't see how having it there hurts someone that doesn't use it. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms000609070701040603050202 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwNjI2MTUwNzQ2 WjBPBgkqhkiG9w0BCQQxQgRAYwqivhzFOYCC+b3ujFwupapKgrj/BsFcYePE13C1wDgC5HS3 u+iXBXY5rcTD0d4u2ZyddC+SayE+hy+ZeS6WnjBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgBOxr/d81yHI0ld9iwMkOEHaYAhNmTpU2FSn/q+L8cZNNtmgpCvKqaZGIWlCUJxSHHX RuzNB71E5O4M1/h5lmhowHKF8cmFX9JSh5ECJNTmLzOMUlyPj5ysWYMb10dpF0BRnYXrnlb4 l1f39THYraRkqKVEA3vylLzwpusrfUR+HzqYaxejWUlBeeKCN7XJFQGkhFU/F2bYD45BOaRX PW4HRA2ye+kMVX9gJebiPt244mLncUkF/FCj5AmIICCGjgSZpAk3T7kXjY/LzgSiVEU6KJ3R YRQibT7vkz8L4lInsUdjaoLLZqayPIA7A+d569lSz16Fp5GO+9UE18By8y667tWoNSfFRMU6 /BdYMmXtVSFCTa44Bm8IguvBbs7MON+p67tWerqflvgFeKZ6nTQKr6O5E6IYWRdQAhq8aJpF 316Ar01qRtbmZ+4EoHQxj8ko78NEEPKWlRm3NETItTeZ2l0tV8VOlz0IS+gIuEODK2U/HHcd D4Wul74JpU65TKn9vZn5r/DlWrFZqbhO9N7ybDFw/R/0ha0VMugDGzz94VQwroEMyaxh/0mr jCFBex9LvH2DqzkPPifdWXtKpDSy15lK3kszS7HUj4GWTS44czurU9VHc5uQB17vko8Ijm7B HPwmawBqI0jAN0ZT2oMRX2XZQrMKBpsQOZweCtyGEgAAAAAAAA== --------------ms000609070701040603050202-- From owner-freebsd-embedded@freebsd.org Wed Jun 26 15:15:38 2019 Return-Path: Delivered-To: freebsd-embedded@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DDB6215CAC2F; Wed, 26 Jun 2019 15:15:37 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-so.shaw.ca (smtp-out-so.shaw.ca [64.59.136.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2A3A071AE7; Wed, 26 Jun 2019 15:15:35 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA id g9dxh2vO1Gusjg9dyhhqjy; Wed, 26 Jun 2019 09:15:32 -0600 X-Authority-Analysis: v=2.3 cv=fOdHIqSe c=1 sm=1 tr=0 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=IkcTkHD0fZMA:10 a=dq6fvYVFJ5YA:10 a=6I5d2MoRAAAA:8 a=YxBL1-UpAAAA:8 a=6rM6nHsMst5jgpFo25wA:9 a=QEXdDO2ut3YA:10 a=IjZwj45LgO3ly-622nXo:22 a=Ia-lj3WSrqcvXOmTRaiG:22 Received: from android-9b917f0ce39da6e6.esitwifi.local (S0106788a207e2972.gv.shawcable.net [70.66.154.233]) by spqr.komquats.com (Postfix) with ESMTPSA id 5707541D; Wed, 26 Jun 2019 08:15:28 -0700 (PDT) Date: Wed, 26 Jun 2019 08:15:05 -0700 User-Agent: K-9 Mail for Android In-Reply-To: References: <30082.1561525622@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: Limiting the size of syslogd output files using options in syslog.conf To: freebsd-hackers@freebsd.org, Ian Lepore , Poul-Henning Kamp CC: freebsd-embedded@freebsd.org From: Cy Schubert Message-ID: <79F61714-D4EC-4A15-90AF-0E48BE684244@cschubert.com> X-CMAE-Envelope: MS4wfGWx19XwILVjfMnqA7v0iw8Kzt1APITuDSB9n3Ts2A4Uq/U0YbC4CcchZmanBLsw6sPLE7ZHz91vTJKStVOX8ziqnfCqGrqvs8rhUSN6LctsUwdv9DT1 dhW4cxx+9LdyTH3kTLeRrrXruRBSxX6vNf5GohbTUitfF/4rMUmuUUX/75+IOD1ORWSytPwb45EsPFRXEF0ZYGnT72eWWtmmBFrNX01Q/6VUS3yVNVMFQVER w+lt30DsmGnszSpdF+MCDLUc/b+oC+3+TH/lBuUAI2j+d8n5pJLGXuCUhJhj5NTp X-Rspamd-Queue-Id: 2A3A071AE7 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-5.66 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; IP_SCORE(-2.47)[ip: (-6.45), ipnet: 64.59.128.0/20(-3.28), asn: 6327(-2.54), country: CA(-0.09)]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: spqr.komquats.com]; NEURAL_HAM_SHORT(-0.98)[-0.979,0]; R_SPF_NA(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[138.136.59.64.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.zen.spamhaus.org : 127.0.0.11,233.154.66.70.zen.spamhaus.org : 127.0.0.11]; ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 15:15:38 -0000 On June 26, 2019 7:54:23 AM PDT, Ian Lepore wrote: >On Wed, 2019-06-26 at 05:07 +0000, Poul-Henning Kamp wrote: >> -------- >> In message < >> bf597c3830ebabbef6ac422a5d648e1eed13fac5=2Ecamel@freebsd=2Eorg>, Ian Le >> pore writes: >> > I've posted a review of a small syslogd change which lets you set a >> > limit on the size of syslogd output files in /etc/syslog=2Econf=2E T= he >> > idea is to prevent filling up a filesystem on emmc or sdcard or >> > other >> > small storage device on an embedded system with unexpected logging >> > triggered by some error or failing hardware=2E >>=20 >> You should consider fifolog(1) in such environments=2E >>=20 >>=20 > >fifolog looks pretty cool and I had no idea it existed, so thanks for >the pointer; I may find useful things to do with it=2E > >But for the problem of unexpected syslog spewage it has exactly the >same problem as running newsyslog very frequently: by design it >discards the information you most want preserved (the triggering event >that led to unexpected log spewage) while carefully preserving all the >following noise which is typically more about symptoms rather than >causes of the problem=2E=20 > >-- Ian > >_______________________________________________ >freebsd-hackers@freebsd=2Eorg mailing list >https://lists=2Efreebsd=2Eorg/mailman/listinfo/freebsd-hackers >To unsubscribe, send any mail to >"freebsd-hackers-unsubscribe@freebsd=2Eorg" I suppose another knob to control the flow of syslog is ok but filling /va= r or however a person configures /var with multiple mount points, this is u= ltimately an AO problem=2E A monitor to invoke newsyslog, among other thing= s when a threshold is exceeded, whether %, free GB, or combination or slidi= ng scale, is a better job for an event handler=2E Devd is already an event = handler=2E Teaching devd and the kernel to trigger an % or GB event to subs= equently spawn a script or process to clean up, like calling newsyslog, cle= aning out /var/tmp, spool, cache, or elsewhere, let's say, a holistic appro= ach, might have broader application than a point solution=2E Just a thought=2E --=20 Pardon the typos and autocorrect, small keyboard in use=2E Cheers, Cy Schubert FreeBSD UNIX: Web: http://www=2EFreeBSD=2Eorg The need of the many outweighs the greed of the few=2E From owner-freebsd-embedded@freebsd.org Wed Jun 26 19:49:08 2019 Return-Path: Delivered-To: freebsd-embedded@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2FDE215D0651; Wed, 26 Jun 2019 19:49:08 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 234B18510B; Wed, 26 Jun 2019 19:49:06 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x5QJn3hT055974; Wed, 26 Jun 2019 12:49:03 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x5QJn3Po055973; Wed, 26 Jun 2019 12:49:03 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201906261949.x5QJn3Po055973@gndrsh.dnsmgr.net> Subject: Re: Limiting the size of syslogd output files using options in syslog.conf In-Reply-To: To: Ian Lepore Date: Wed, 26 Jun 2019 12:49:03 -0700 (PDT) CC: Michael Sierchio , freebsd-hackers@freebsd.org, freebsd-embedded@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 234B18510B X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [2.41 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.87)[0.869,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dnsmgr.net]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.19)[0.187,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: gndrsh.dnsmgr.net]; NEURAL_SPAM_LONG(0.42)[0.424,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.04)[ip: (0.14), ipnet: 69.59.192.0/19(0.07), asn: 13868(0.05), country: US(-0.06)] X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 19:49:08 -0000 > On Tue, 2019-06-25 at 16:06 -0700, Michael Sierchio wrote: > > Apologies, but I don't understand the point of this at all. I object > > especially to idea of patching syslogd when EVERYTHING you seek can > > be > > accomplished with appropriate settings in /etc/newsyslog.conf and > > changing > > its entry in /etc/crontab to run more frequently. You can control the > > size > > and number of files there. > > > > You can also do remote syslog. > > > > Do you need to rotate logs more than once per minute? It is not a matter of rapid log rotation it is that your rotate the very data you want out of existance that is the problem. > > Newsyslog handles expected log growth. I guess you've never had a > hardware device or other problem unexpectedly trigger the writing of > hundreds or thousands of lines of output per minute to syslog? It > happens. You write apps that try to strike a sensible balance between > logging important events and not logging so much it leads to problems, > then something unexpected happens and your carefully crafted logging > starts happening at 200hz and actually contributes to the problem. :-) Only 200hz? > We've been using this code at $work since 2003; it appears in dozens of > different products ranging from lab instruments to national timescale > systems. People using the products experiencing such problems just > want the device to keep working as best it can. They don't want > repeated mysterious reboots because of things like /var filling up. > (In the case of products like lab instruments, they don't really > perceive the thing to be a computer running an OS at all.) > > And most of all, when something does go wrong and they get our support > engineers to look into it, they don't want to hear "We can't figure out > what happened originally to cause this problem because all the logging > got rotated out of existence." Newsyslog does exactly the opposite of > what you want in that situation... it purposely destroys the valuable > information about the original problem while carefully preserving the > useless fallout that follows. Yes, and this is not only an issue on embeded, been in exactly that position trying to post diagnose server failures, all the logs had been rotated out and filled with a repeated changing sequence of error message that was a cascade effect of what had originally gone wrong. Syslog does very good with rate limiting and colapsing a single repeated line, but when it is getting 2 or more intertwined lines this no longer works. It may make an interesting project to create a coalescing buffer that holds some N messages for up to some t time and collapse them into a "Foo bar table top (repeated n times in t)" that would be a much smarter rate limiter. You toss anything from the coalesce buffer that is of some age > t if it only occured once. > Over the years we've upstreamed every freebsd customization we've ever > done at $work except this one. The only reason I've been carrying this > one local mod is because it seemed too specialized to be useful to > anyone else. Then yesterday someone working on an embedded-system > product asked me on irc how to solve the problem of unexpected log > spewage, so I offered the solution we've been using. I agree that this is a useful feature for exactly the reasons Ian sites. It is rarely the end of a fifo log that is useful, but rather some point back in time that occured before the cascade effect. This looks to be a simple, but reasonable solution to a long standing problem that is not solved in any other way. The 32k circ buffer size should probably be config file tweakable though. > -- Ian > > On Tue, Jun 25, 2019 at 3:25 PM Ian Lepore wrote: > > > > > I've posted a review of a small syslogd change which lets you set a > > > limit on the size of syslogd output files in /etc/syslog.conf. The > > > idea is to prevent filling up a filesystem on emmc or sdcard or > > > other > > > small storage device on an embedded system with unexpected logging > > > triggered by some error or failing hardware. > > > > > > If you're interested, please see the review at > > > > > > https://reviews.freebsd.org/D20764 > > > > > > -- Ian -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-embedded@freebsd.org Wed Jun 26 19:54:02 2019 Return-Path: Delivered-To: freebsd-embedded@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CEA4A15D0989 for ; Wed, 26 Jun 2019 19:54:02 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2B818855A7 for ; Wed, 26 Jun 2019 19:54:02 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x5QJrwrH055995; Wed, 26 Jun 2019 12:53:58 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x5QJrwgE055994; Wed, 26 Jun 2019 12:53:58 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201906261953.x5QJrwgE055994@gndrsh.dnsmgr.net> Subject: Re: Limiting the size of syslogd output files using options in syslog.conf In-Reply-To: <3feff28d-0b91-a6fc-b0e9-161d3d315a78@denninger.net> To: Karl Denninger Date: Wed, 26 Jun 2019 12:53:58 -0700 (PDT) CC: freebsd-embedded@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 2B818855A7 X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [2.13 / 15.00]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: gndrsh.dnsmgr.net]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.10)[-0.105,0]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.86)[0.860,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dnsmgr.net]; AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.45)[0.447,0]; R_SPF_NA(0.00)[]; IP_SCORE(0.04)[ip: (0.14), ipnet: 69.59.192.0/19(0.07), asn: 13868(0.05), country: US(-0.06)] X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 19:54:03 -0000 > On 6/26/2019 09:49, Ian Lepore wrote: > > > > Over the years we've upstreamed every freebsd customization we've ever > > done at $work except this one. The only reason I've been carrying this > > one local mod is because it seemed too specialized to be useful to > > anyone else. Then yesterday someone working on an embedded-system > > product asked me on irc how to solve the problem of unexpected log > > spewage, so I offered the solution we've been using. > > > > -- Ian > > Yep -- this is a small corner case /but if it happens /in the embedded > world (and I've had it happen) it's a five-alarm pain in the neck. > > I'm in favor for that reason, and I can't see how having it there hurts > someone that doesn't use it. I would not restrict this to an embedded system problem, our default sizes and rotate rates on even /var/log/messages require a tiny 600kbhyte log event and your good info is gone... I always tweak my newsyslog.conf file as our defaults are near useless for post failure analysys. > Karl Denninger -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-embedded@freebsd.org Wed Jun 26 21:54:20 2019 Return-Path: Delivered-To: freebsd-embedded@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EEECF15D3231 for ; Wed, 26 Jun 2019 21:54:19 +0000 (UTC) (envelope-from kudzu@tenebras.com) Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A66F89904 for ; Wed, 26 Jun 2019 21:54:18 +0000 (UTC) (envelope-from kudzu@tenebras.com) Received: by mail-qt1-x836.google.com with SMTP id w40so311581qtk.0 for ; Wed, 26 Jun 2019 14:54:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tenebras-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=8aYIJQjVjT7TohBYroHJ56nJZ8HGrQQjSPKvTE3llbc=; b=wL3Tn4JUpPlUK5LF7G1tHwr7ljgZHkZ29SblfenvYDhoZkv7wajZ0GezzlQoIe5h8F DqcDCbvaMxmuSCfvOZgd05XgxFQuQ1TUqqmCuyzzmWDAhu+ZJ8bRaLCo8yFrdf23HvkW R7qTy6HCwazDUQvqhqKHE/TdZieTiTB9WFpw1wBUEOVab5rVdoZPmVfyAQ2gv7l1h+Oz IdXcPJX8k8/H904SinsUmCgg4hPrJcU1aET0/iWQ90NWeq/H++F32BokLKudOCf4uWrs 4f82UIBExrbbfybM/lCt0wSLRN4H7YcFlhROEkvDzvSXUDn4o5yGZFpx2RHYu0eNk9eE zt/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=8aYIJQjVjT7TohBYroHJ56nJZ8HGrQQjSPKvTE3llbc=; b=OPX+v7d+kXLXDDOvCvn1M1yuib9SKhfA8tBfhMO2PjmaS7Qy4VUWZQGWjK9nHymcLJ 05TZjaTVTZevr3Pjq2gbGaILRGHwp8Zws5TkQubbbbkWgSQuYoAXv3PgtlhY1Uy/aVhW ZlRH1ATfb01XKsfZFew6iYbKuWEp+864T/oMvuqZB+TRz3kpkLGuw5SZ+ggUnkLEeN3N bGtgzqcMBQdOCGbz46JqTDsQ2eZNPzvzpHWa2DzzR3wPJQ/yvlnhIS0K9Z+XQLyIT4Fu 8y0FWISpY6ooeDyBaRoUYymzO7wS0pNwCtvGTPfe7Qc+PmFFDvhQOlbr3MGQn6o5e+hW m6yg== X-Gm-Message-State: APjAAAWoEkArfzGGeSXW0Dx1spTPAYILnWB61N+CL7UpeLuJl2m3iz85 UISKh8XD/WWo+ZU91JU2bZjfhQnXRxM1KLPp5OsIB7X+lkVIpg== X-Google-Smtp-Source: APXvYqxbpoSfzou+nCi+NHqdwGRqwB5/QSVM/fPdWljGdeJYOKcXZ/QXstHH9EC5cI/6BQYn/EFUoCxYe3ktYI0H61c= X-Received: by 2002:ac8:3014:: with SMTP id f20mr131815qte.69.1561586057565; Wed, 26 Jun 2019 14:54:17 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Michael Sierchio Date: Wed, 26 Jun 2019 14:53:41 -0700 Message-ID: Subject: Re: Limiting the size of syslogd output files using options in syslog.conf To: Ian Lepore Cc: freebsd-hackers@freebsd.org, freebsd-embedded@freebsd.org X-Rspamd-Queue-Id: 5A66F89904 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=tenebras-com.20150623.gappssmtp.com header.s=20150623 header.b=wL3Tn4JU X-Spamd-Result: default: False [-6.23 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[tenebras-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-embedded@freebsd.org]; DMARC_NA(0.00)[tenebras.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[tenebras-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com]; RCVD_IN_DNSWL_NONE(0.00)[6.3.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.96)[-0.958,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-2.97)[ip: (-9.30), ipnet: 2607:f8b0::/32(-3.14), asn: 15169(-2.34), country: US(-0.06)] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jun 2019 21:54:20 -0000 You can pipe any syslog facility to a program. My /etc/syslog.conf has auth.*;authpriv.* |/usr/local/bin/authlog because I do some processing, but also rate-limiting and alerting, before writing to /var/log/auth.log You could just as easily write auth.*;authpriv.* | fifolog_writer /var/log/auth.fifolog Again, I object to modifying syslogd to add a feature that already exists in the modular framework of pipes and utilities and specialized tools like fifolog. =E2=80=93 M On Wed, Jun 26, 2019 at 7:50 AM Ian Lepore wrote: > On Tue, 2019-06-25 at 16:06 -0700, Michael Sierchio wrote: > > Apologies, but I don't understand the point of this at all. I object > > especially to idea of patching syslogd when EVERYTHING you seek can > > be > > accomplished with appropriate settings in /etc/newsyslog.conf and > > changing > > its entry in /etc/crontab to run more frequently. You can control the > > size > > and number of files there. > > > > You can also do remote syslog. > > > > Do you need to rotate logs more than once per minute? > > > > > > Newsyslog handles expected log growth. I guess you've never had a > hardware device or other problem unexpectedly trigger the writing of > hundreds or thousands of lines of output per minute to syslog? It > happens. You write apps that try to strike a sensible balance between > logging important events and not logging so much it leads to problems, > then something unexpected happens and your carefully crafted logging > starts happening at 200hz and actually contributes to the problem. > > We've been using this code at $work since 2003; it appears in dozens of > different products ranging from lab instruments to national timescale > systems. People using the products experiencing such problems just > want the device to keep working as best it can. They don't want > repeated mysterious reboots because of things like /var filling up. > (In the case of products like lab instruments, they don't really > perceive the thing to be a computer running an OS at all.) > > And most of all, when something does go wrong and they get our support > engineers to look into it, they don't want to hear "We can't figure out > what happened originally to cause this problem because all the logging > got rotated out of existence." Newsyslog does exactly the opposite of > what you want in that situation... it purposely destroys the valuable > information about the original problem while carefully preserving the > useless fallout that follows. > > Over the years we've upstreamed every freebsd customization we've ever > done at $work except this one. The only reason I've been carrying this > one local mod is because it seemed too specialized to be useful to > anyone else. Then yesterday someone working on an embedded-system > product asked me on irc how to solve the problem of unexpected log > spewage, so I offered the solution we've been using. > > -- Ian > > > > > On Tue, Jun 25, 2019 at 3:25 PM Ian Lepore wrote: > > > > > I've posted a review of a small syslogd change which lets you set a > > > limit on the size of syslogd output files in /etc/syslog.conf. The > > > idea is to prevent filling up a filesystem on emmc or sdcard or > > > other > > > small storage device on an embedded system with unexpected logging > > > triggered by some error or failing hardware. > > > > > > If you're interested, please see the review at > > > > > > https://reviews.freebsd.org/D20764 > > > > > > -- Ian > > > > > > _______________________________________________ > > > freebsd-embedded@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-embedded > > > To unsubscribe, send any mail to " > > > freebsd-embedded-unsubscribe@freebsd.org > > > " > > > > > > > > > --=20 "Well," Brahm=C4=81 said, "even after ten thousand explanations, a fool is = no wiser, but an intelligent person requires only two thousand five hundred." - The Mah=C4=81bh=C4=81rata