From owner-trustedbsd-audit@freebsd.org Thu Jun 6 11:37:34 2019 Return-Path: Delivered-To: trustedbsd-audit@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D3FA15AFDBD for ; Thu, 6 Jun 2019 11:37:34 +0000 (UTC) (envelope-from b-spam@intraversal.de) Received: from mail.webgo24-server43.de (mail.webgo24-server43.de [37.17.227.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.webgo24.de", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5975F685D5 for ; Thu, 6 Jun 2019 11:37:32 +0000 (UTC) (envelope-from b-spam@intraversal.de) Received: from [172.20.10.2] (ip-109-41-130-246.web.vodafone.de [109.41.130.246]) by mail.webgo24-server43.de (Postfix) with ESMTPSA id 06C69BA40042; Thu, 6 Jun 2019 13:37:23 +0200 (CEST) From: b Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Date: Thu, 6 Jun 2019 13:37:22 +0200 Subject: Audit Pipe Drops Message-Id: <5247B21D-FB16-4949-85E5-9D0D8B37908C@intraversal.de> To: darwin-dev@lists.apple.com, trustedbsd-audit@freebsd.org X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 5975F685D5 X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [1.91 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; IP_SCORE(-0.00)[country: DE(-0.01)]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[intraversal.de]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.21)[0.210,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[mail2.intraversal.de,mail1.intraversal.de]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[244.227.17.37.list.dnswl.org : 127.0.10.0]; NEURAL_HAM_SHORT(-0.25)[-0.255,0]; NEURAL_SPAM_LONG(0.56)[0.561,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:44066, ipnet:37.17.224.0/21, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: trustedbsd-audit@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: TrustedBSD Audit Discussion List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jun 2019 11:37:34 -0000 Hello everyone, I am crossposting this message to the darwin dev list and the = trustedbsd-audit list since relevant knowledge might be shared among = members of both lists and I was not really sure how to properly present = the issue to both lists separately. Also, my post is quite long as it = bundles the outcome of several days of debugging. Please excuse. I am currently reading a cloned bsm audit pipe from a user space client = on macOS to retrieve information about process creation and operations = on the system (pc|ex). In the future I want to add other audit classes = as well. I am currently looking at bsmtrace as a reference = implementation of the read loop. There is one major issue, though, that took me a while to become aware = of. The audit pipe is allowed to drop audit records in the kernel, which = eventually is technically inevitable of course. The issue I am facing is = finding a reliable solution to avoid this. Drops can occur only in audit_pipe_append (audit_pipe.c) under two = conditions, (1) the queue is full or (2) the kernel was not able to = allocate memory without blocking. (1) can generally be managed in user space client code. There is only = one scenario I found (up to now) where even the maximum audit pipe = length is insufficient and that is system wake up procedure. Even before = the system emits kIOMessageSystemWillPowerOn there are lots and lots of = audit events that eventually max out the audit pipe. (2) can=E2=80=99t be influenced from a user space client, at least not = that I am aware of. This happens sporadically but reproducibly when the = system is under a lot of stress, e.g. when a lot of processes are = spawned at a time and start executing some audited code. All this leads to several questions of which I am not sure if they = qualify as potential bug reports or enhancement requests. I would be = more than happy if someone with a better understanding of things could = comment on them or even suggest possible solution approaches. - (1) could clearly be solved by increasing the allowed maximum audit = pipe length which currently is 1024. This maximum value is probably = several years old by now and I could imagine that in the meantime the = xnu kernel has become a lot more =E2=80=9Etalkative=E2=80=9C. Is there = some technical limitation that would prevent an increase? - I am wondering how the global audit trail that is written to disk is = able to perform better, i.e. does not drop (seemingly), than the cloned = audit pipe purely in memory? Is there some penalty associated with = passing memory from the kernel to user space? How does the global audit = trail not struggle with the M_NOWAIT policy? - Since the audit pipe tries to acquire memory by calling malloc with = the M_NOWAIT flag, which I assume happens for a reason, is there some = strategy or configuration available from user space to ease the = restraint on kernel memory allocation =E2=80=93 or am I simply out of = luck here? - I am surely not an expert on memory allocation and even less so in = kernel space. With that said, I imagine that allocating only one block = of memory for both, pointer (ape) and memory blob (ape->ape_record), = instead of two separate memory allocations would half the potential for = M_NOWAIT failure. - Potential for both (1) and (2) could at least be reduced by further = filtering events in the kernel. I am not interested in each and every = audit event type. The FreeBSD Handbook has the notion that =E2=80=9Eaudit = event classes may be customized by modifying the audit_class and = audit_event configuration files=E2=80=9C. I assume this could also mean = adding custom audit classes and associating them with the event types of = interest. How can this be done programmatically for local audit trails? = Are there any code examples available? I hope this does not come across (too) snarky but my expectation towards = a security mechanism is first and foremost reliability. Information loss = is definitely not supportive in that. Therefore I either hope for a = viable existing solution for my problem or am very eager for a fix to = the issues at hand. Thanks, Benjamin=