From owner-freebsd-security@FreeBSD.ORG Wed May 30 11:34:44 2007 Return-Path: X-Original-To: freebsd-security@freebsd.org Delivered-To: freebsd-security@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 00A4516A400 for ; Wed, 30 May 2007 11:34:43 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 93A7B13C44C for ; Wed, 30 May 2007 11:34:43 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 2F34146F3A; Wed, 30 May 2007 07:08:52 -0400 (EDT) Date: Wed, 30 May 2007 12:08:52 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: zhouyi zhou In-Reply-To: <20070529154654.2d7d12ce.zhouzhouyi@ercist.iscas.ac.cn> Message-ID: <20070530120106.B56059@fledge.watson.org> References: <20070528120029.DFCCB16A5BC@hub.freebsd.org> <20070529154654.2d7d12ce.zhouzhouyi@ercist.iscas.ac.cn> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="0-278346229-1180523332=:56059" Cc: freebsd-security@freebsd.org, trustedbsd-discuss@FreeBSD.org Subject: Re: (Security Regression Testsuites)Request for comments X-BeenThere: freebsd-security@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Security issues \[members-only posting\]" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2007 11:34:44 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-278346229-1180523332=:56059 Content-Type: TEXT/PLAIN; charset=GB2312; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE On Tue, 29 May 2007, zhouyi zhou wrote: > Where I am still confused: > > 1) Which area and direction should I focus. The security subsystem in=20 > FreeBSD is large, which area deserves a testsuite in higher priority. Off-hand, my feeling is I'd like us to consider three areas of testing: - Correctness testing, in which we test general and edge cases in access control to make sure both the infrastructure and policies operate correc= tly. For example, using test policies to validate that the MAC Framework is correctly implemented, and using tests on real policies to determine tha= t the real policies implement the desired access control. Likewise, for audit, making sure that the right records are generated with the right tokens for the right events, that the selection model works, etc. - Stability testing, in which we test general and edge cases to make sure t= he system is stable, especially under load, etc. - ABI/API life cycle testing, in which we confirm that key data structures = in the ABI remain stable over time, and that the interpretation remains consistent. For example, the persisting data structures in on-disk labe= ls, audit record formats, etc. Obviously, doing all this in one summer is well out of scope, but I think s= ome=20 useful in-roads can be made in testing key areas, such as making sure that= =20 file system protections with MLS and Biba are correct, tests for audit, and= so=20 on. You may want to look at Pawel's and my existing file system test tools= =20 (src/tools/regression in 7-CURRENT) to see some areas and approaches to=20 testing. > 2) The general structure of the testsuite: Will it be a userland applicat= ion=20 > package like stress2, or include a kernel module cooperation (like=20 > security/mac_test) 3) How to write a testsuite that will prevent the furt= hor=20 > violation of security instead of test the cases which are already correct= ed.=20 > PF=A1=A2IPFW and IPSec have already corrected their confliction with Mand= atory=20 > Access Control, I think the testcases for the already corrected problems= =20 > will not discover the newly generated problems, for example: test case fo= r=20 > the PF's synproxy state rule only verify PF have correctly add a correct = tag=20 > for Mandatory access control in function pf_send_tcp, how we discover a= =20 > problem which may create in the future by means of create a mbuf without = a=20 > correct tag for Mandatory access control in a new function? I would suggest starting with a small set of test projects to evaluate=20 approaches. For example: (1) Consider adding a new test policy couple with userland tools to make su= re that access control checks occur as required for each system call. (2) Add a set of user space tests that confirm that MLS is properly implemented for each system call. (3) Add a set of user space tests that confirm that, for each system call, = the right audit records are generated with the right tokens. (4) Add a set of user space tests to confirm that audit record preselection= is properly implemented. These are a bit more bounded in scope, and should start to bring out common= =20 aspects to testing across security functions (i.e., "foreach system call"). Robert N M Watson Computer Laboratory University of Cambridge --0-278346229-1180523332=:56059--