From owner-freebsd-arch@FreeBSD.ORG Sun Dec 11 14:23:00 2005 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C1D9216A41F for ; Sun, 11 Dec 2005 14:23:00 +0000 (GMT) (envelope-from flz@xbsd.org) Received: from smtp.xbsd.org (xbsd.org [82.233.2.192]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3CE7443D49 for ; Sun, 11 Dec 2005 14:23:00 +0000 (GMT) (envelope-from flz@xbsd.org) Received: from localhost (localhost.xbsd.org [127.0.0.1]) by smtp.xbsd.org (Postfix) with ESMTP id 627AD119B2 for ; Sun, 11 Dec 2005 15:22:59 +0100 (CET) Received: from smtp.xbsd.org ([127.0.0.1]) by localhost (srv1.xbsd.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 98045-06 for ; Sun, 11 Dec 2005 15:22:54 +0100 (CET) Received: from cream.xbsd.org (cream.xbsd.org [192.168.42.6]) by smtp.xbsd.org (Postfix) with ESMTP id 00762114B3 for ; Sun, 11 Dec 2005 15:22:53 +0100 (CET) From: Florent Thoumie To: freebsd-arch@freebsd.org Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-7v2NhW1MS4+TXJA20BqG" Date: Sun, 11 Dec 2005 15:21:56 +0000 Message-Id: <1134314516.648.12.camel@cream.xbsd.org> Mime-Version: 1.0 X-Mailer: Evolution 2.4.2 FreeBSD GNOME Team Port X-Virus-Scanned: amavisd-new at xbsd.org Subject: Latest changes to iwi(4) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2005 14:23:00 -0000 --=-7v2NhW1MS4+TXJA20BqG Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Now that iwi firmware is read from /boot/firmware, is there any=20 chance that someone adds this directory to mtree/BSD.root.dist ? --=20 Florent Thoumie flz@FreeBSD.org FreeBSD committer --=-7v2NhW1MS4+TXJA20BqG Content-Type: application/pgp-signature; name=signature.asc Content-Description: Ceci est une partie de message =?ISO-8859-1?Q?num=E9riquement?= =?ISO-8859-1?Q?_sign=E9e?= -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQBDnEQUMxEkbVFH3PQRAmGPAJ9LNmz0a+t6vfHkdeBIZVNdICjS9gCeIm++ 9+HtF9NEVRxjPy4vHFmZbT0= =Zb/a -----END PGP SIGNATURE----- --=-7v2NhW1MS4+TXJA20BqG-- From owner-freebsd-arch@FreeBSD.ORG Sun Dec 11 15:28:07 2005 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7DAAA16A41F for ; Sun, 11 Dec 2005 15:28:07 +0000 (GMT) (envelope-from delphij@gmail.com) Received: from zproxy.gmail.com (zproxy.gmail.com [64.233.162.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id CBB0343D5C for ; Sun, 11 Dec 2005 15:28:06 +0000 (GMT) (envelope-from delphij@gmail.com) Received: by zproxy.gmail.com with SMTP id 4so1258490nzn for ; Sun, 11 Dec 2005 07:28:06 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=cJkcUn1auwfzaugOc1sTNy5O8uGb04NeIvCtnhKwBKyQ4tgXogjLMh0pmkbgMzVwMYfMPrS8nZ/vS03uZCac0ppqpAnULtbVOmk0R+h1cz8cH775RoVjNOuJOSNHpQmr7Jj2TimGksvx0tM5nmN82UQ+ifAo6loC0I/gmEUv+po= Received: by 10.64.184.10 with SMTP id h10mr5208649qbf; Sun, 11 Dec 2005 07:28:06 -0800 (PST) Received: by 10.65.72.5 with HTTP; Sun, 11 Dec 2005 07:28:06 -0800 (PST) Message-ID: Date: Sun, 11 Dec 2005 23:28:06 +0800 From: Xin LI To: Florent Thoumie In-Reply-To: <1134314516.648.12.camel@cream.xbsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: base64 Content-Disposition: inline References: <1134314516.648.12.camel@cream.xbsd.org> Cc: freebsd-arch@freebsd.org Subject: Re: Latest changes to iwi(4) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: delphij@delphij.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2005 15:28:07 -0000 SGksCgpPbiAxMi8xMS8wNSwgRmxvcmVudCBUaG91bWllIDxmbHpAeGJzZC5vcmc+IHdyb3RlOgo+ ICAgICAgICBOb3cgdGhhdCBpd2kgZmlybXdhcmUgaXMgcmVhZCBmcm9tIC9ib290L2Zpcm13YXJl LCBpcyB0aGVyZSBhbnkKPiAgICAgICAgY2hhbmNlIHRoYXQgc29tZW9uZSBhZGRzIHRoaXMgZGly ZWN0b3J5IHRvIG10cmVlL0JTRC5yb290LmRpc3QgPwoKSSB0aGluayBJIGhhcyBwcm9wb3NlZCB0 aGlzIHNvbWUgd2Vla3MgYWdvIGFuZCByZWNlaXZlZCBubyBvYmplY3Rpb24Kc28gSSBoYXZlIGNv bW1pdHRlZCB0aGUgb2xkIHBhdGNoLi4uCgpDaGVlcnMsCi0tClhpbiBMSSA8ZGVscGhpakBkZWxw aGlqLm5ldD4gaHR0cDovL3d3dy5kZWxwaGlqLm5ldAo= From owner-freebsd-arch@FreeBSD.ORG Sun Dec 11 15:33:00 2005 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A16BA16A41F for ; Sun, 11 Dec 2005 15:33:00 +0000 (GMT) (envelope-from flz@xbsd.org) Received: from smtp.xbsd.org (xbsd.org [82.233.2.192]) by mx1.FreeBSD.org (Postfix) with ESMTP id 03B4443D5E for ; Sun, 11 Dec 2005 15:32:59 +0000 (GMT) (envelope-from flz@xbsd.org) Received: from localhost (localhost.xbsd.org [127.0.0.1]) by smtp.xbsd.org (Postfix) with ESMTP id 97FAF11A15; Sun, 11 Dec 2005 16:32:58 +0100 (CET) Received: from smtp.xbsd.org ([127.0.0.1]) by localhost (srv1.xbsd.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 98614-08; Sun, 11 Dec 2005 16:32:50 +0100 (CET) Received: from cream.xbsd.org (cream.xbsd.org [192.168.42.6]) by smtp.xbsd.org (Postfix) with ESMTP id 36CB51144D; Sun, 11 Dec 2005 16:32:50 +0100 (CET) From: Florent Thoumie To: delphij@delphij.net In-Reply-To: References: <1134314516.648.12.camel@cream.xbsd.org> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-IYWc2Ldl+n3lC9S7Ij0E" Date: Sun, 11 Dec 2005 16:31:52 +0000 Message-Id: <1134318712.648.19.camel@cream.xbsd.org> Mime-Version: 1.0 X-Mailer: Evolution 2.4.2 FreeBSD GNOME Team Port X-Virus-Scanned: amavisd-new at xbsd.org Cc: freebsd-arch@freebsd.org Subject: Re: Latest changes to iwi(4) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2005 15:33:00 -0000 --=-IYWc2Ldl+n3lC9S7Ij0E Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Le Dimanche 11 d=C3=A9cembre 2005 =C3=A0 23:28 +0800, Xin LI a =C3=A9crit : > Hi, >=20 > On 12/11/05, Florent Thoumie wrote: > > Now that iwi firmware is read from /boot/firmware, is there any > > chance that someone adds this directory to mtree/BSD.root.dist ? >=20 > I think I has proposed this some weeks ago and received no objection > so I have committed the old patch... Just seen that, thanks ! --=20 Florent Thoumie flz@FreeBSD.org FreeBSD committer --=-IYWc2Ldl+n3lC9S7Ij0E Content-Type: application/pgp-signature; name=signature.asc Content-Description: Ceci est une partie de message =?ISO-8859-1?Q?num=E9riquement?= =?ISO-8859-1?Q?_sign=E9e?= -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQBDnFR4MxEkbVFH3PQRAmUfAJ0Xka8Zm7gNM6eUMuN4rGB4FYgy3QCeI9kv BLJr8eBTnEWl0THyU1vfXE8= =fNuf -----END PGP SIGNATURE----- --=-IYWc2Ldl+n3lC9S7Ij0E-- From owner-freebsd-arch@FreeBSD.ORG Mon Dec 12 00:17:09 2005 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 88EDB16A41F for ; Mon, 12 Dec 2005 00:17:09 +0000 (GMT) (envelope-from mckusick@chez.mckusick.com) Received: from chez.mckusick.com (dsl081-247-049.sfo1.dsl.speakeasy.net [64.81.247.49]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0FDF043D46 for ; Mon, 12 Dec 2005 00:17:08 +0000 (GMT) (envelope-from mckusick@chez.mckusick.com) Received: from chez.mckusick.com (localhost [127.0.0.1]) by chez.mckusick.com (8.13.1/8.13.1) with ESMTP id jBC0HMZx061845 for ; Sun, 11 Dec 2005 16:17:22 -0800 (PST) (envelope-from mckusick@chez.mckusick.com) Message-Id: <200512120017.jBC0HMZx061845@chez.mckusick.com> To: arch@freebsd.org X-URL: http://WWW.McKusick.COM/ Date: Sun, 11 Dec 2005 16:17:22 -0800 From: Kirk McKusick Cc: Subject: FreeBSD 6.0 Code Reading Class X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Kirk McKusick List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2005 00:17:09 -0000 For those of you living in the Bay Area, I am organizing a FreeBSD code reading seminar to be run in Berkeley early next year (dates below). I would really appreciate having some of the committers attend and participate as I plan to debate architectural issues as well as discuss implementation details. Committers are eligible for a 50% discount, send me email for details. I do plan to video tape the lectures and make them available on DVD-R in June for those of you that are interested but unable to attend. Because of the colossal amount of time that it takes to prepare for and run this class, I only do it about once every six years. Kirk McKusick =-=-=-=-=-= Upcoming FreeBSD Kernel Code Reading Evening Course The ``FreeBSD Kernel Internals: An Intensive Code Walkthrough'' course will be taught during the Spring of 2006. The class will be held at the historic Hillside Club at 2286 Cedar Strett, Berkeley, CA 94709 just three blocks north of the Berkeley campus once per week from 6:30PM to 9:45PM starting Wednesday February 22nd and finishing Tuesday June 13th. You can sign up for the class at http://www.mckusick.com/courses/adveveclass.html Description This course will provide a detailed background in the FreeBSD kernel. The course will cover all the basic parts of the system including process managment, memory management, scheduling, I/O structure, local and remote filesystems, and networking. The main emphasis will be on the machine independent parts of the system; little time will be spent on the machine specific parts of the system such as device drivers. Where machine specific topics are covered, the Intel PC architecture will be used for illustration. Course Materials Each student receives a CD-ROM containing the FreeBSD 6.0 kernel sources with tags database, plus a printed copy of the discussed files from the FreeBSD 6.0 kernel sources (two volumes totalling approximately 1200 pages). Course Organization The evening course meets once per week for fifteen weeks. The majority of the lecture time is spent reading kernel source code. The fifteen weeks are structured as follows: 1) Weds February 22: Organization, overview of source layout 2) Weds March 1: Kernel header files 3) Weds March 8: System calls and file open 4) Weds March 15: Pathname translation and file creation 5) Weds March 22: Vnode interface mechanics, write to a local file 6) Weds March 29: Opening, using, and closing locally connected sockets One week break 7) Tues April 11: User datagram protocol and routing 8) Tues April 18: TCP Algorithms 9) Tues April 25: Fork, exit, and exec 10) Tues May 2: Signal generation and delivery, scheduling 11) Tues May 9: Virtual memory header files and file mapping 12) Tues May 16: Page fault service, pageout processing 13) Tues May 23: NFS client and server operation One week break 14) Tues June 6: Multiplexing with select, system startup 15) Tues June 13: Special topics: filesystem layering and soft updates From owner-freebsd-arch@FreeBSD.ORG Mon Dec 12 12:14:26 2005 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F378E16A41F for ; Mon, 12 Dec 2005 12:14:25 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.FreeBSD.org (Postfix) with ESMTP id A142643D62 for ; Mon, 12 Dec 2005 12:14:25 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id 3D930BC66 for ; Mon, 12 Dec 2005 12:14:23 +0000 (UTC) To: arch@freebsd.org From: Poul-Henning Kamp Date: Mon, 12 Dec 2005 13:14:23 +0100 Message-ID: <1023.1134389663@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: Subject: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2005 12:14:26 -0000 Obligatory bikeshed avoidance notice: >> Please read all the way to the bottom of this email before you reply << Given that illegal or malformed format strings to the printf family of functions mostly result in harmless misformatting but occationally in coredumps and maybe some times in security issues, what is the desired behaviour of libc's printf implementation ? A very good example is printf("%hf", 1.0); The 'h' modifier is not legal for %f format, and it is therefore a good bet that the programmer was confused and we know that the program contains at least one error. Our first line of defence against this kind of error is compile-time checking by GCC, but we cannot rely on the string being sane in libc, we still need to do error checking. The context for the above is that I'm working on adding extensibility to our printf, compatible with the GLIBC (see 12.13 in the glibc manual). Obviously, gcc cannot compile-time check such extensions for us, and therefore the question gains a bit more relevance. In an ideal world, the printf family of functions would have been defined to return EINVAL in this case. Almost nobody checks the return values of printf-like functions however and those few that do, all pressume that it is an I/O error so such an approach is unlikely to gain us much if anything. Another alternative is to spit out the format string unformatted, possibly with an attached notice, but this doesn't really seem to help anybody either, but at least indicates what the problem is. I'm leaning towards doing what phkmalloc has migrated to over time: Make a variable which can select between "normal/paranoia" and force it to paranoia for (uid==0 || gid==0 || setuid || setgid). If the variable is set, a bogus format string will result in abort(2). If it is not set, the format string will be output unformatted in the message "WARNING: Illegal printf() format string: \"...\". If you cannot possibly live with that, please state why, and describe what behaviour you would prefer instead. Thankyou, (and may the best bikeshed win). Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Dec 12 15:43:52 2005 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9BC8416A41F for ; Mon, 12 Dec 2005 15:43:52 +0000 (GMT) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id A5B7F43D5E for ; Mon, 12 Dec 2005 15:43:51 +0000 (GMT) (envelope-from max@love2party.net) Received: from [84.163.197.19] (helo=amd64.laiers.local) by mrelayeu.kundenserver.de (node=mrelayeu5) with ESMTP (Nemesis), id 0ML25U-1Elpq72UQx-0008I1; Mon, 12 Dec 2005 16:43:49 +0100 From: Max Laier Organization: FreeBSD To: freebsd-arch@freebsd.org Date: Mon, 12 Dec 2005 16:43:33 +0100 User-Agent: KMail/1.8.2 References: <1023.1134389663@critter.freebsd.dk> In-Reply-To: <1023.1134389663@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart11122301.lkS6gTjF6f"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200512121643.39236.max@love2party.net> X-Provags-ID: kundenserver.de abuse@kundenserver.de login:61c499deaeeba3ba5be80f48ecc83056 Cc: Poul-Henning Kamp Subject: Re: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2005 15:43:52 -0000 --nextPart11122301.lkS6gTjF6f Content-Type: text/plain; charset="iso-8859-6" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Monday 12 December 2005 13:14, Poul-Henning Kamp wrote: > Obligatory bikeshed avoidance notice: > >> Please read all the way to the bottom of this email before you reply << > > Given that illegal or malformed format strings to the printf family > of functions mostly result in harmless misformatting but occationally > in coredumps and maybe some times in security issues, what is the > desired behaviour of libc's printf implementation ? > > A very good example is > > printf("%hf", 1.0); > > The 'h' modifier is not legal for %f format, and it is therefore a good > bet that the programmer was confused and we know that the program > contains at least one error. > > > Our first line of defence against this kind of error is compile-time > checking by GCC, but we cannot rely on the string being sane in libc, > we still need to do error checking. > > The context for the above is that I'm working on adding extensibility > to our printf, compatible with the GLIBC (see 12.13 in the glibc > manual). Obviously, gcc cannot compile-time check such extensions > for us, and therefore the question gains a bit more relevance. > > In an ideal world, the printf family of functions would have been > defined to return EINVAL in this case. Almost nobody checks the > return values of printf-like functions however and those few that > do, all pressume that it is an I/O error so such an approach is > unlikely to gain us much if anything. > > Another alternative is to spit out the format string unformatted, > possibly with an attached notice, but this doesn't really seem to > help anybody either, but at least indicates what the problem is. > > > I'm leaning towards doing what phkmalloc has migrated to over time: > Make a variable which can select between "normal/paranoia" and force > it to paranoia for (uid=3D=3D0 || gid=3D=3D0 || setuid || setgid). > > If the variable is set, a bogus format string will result in abort(2). > > If it is not set, the format string will be output unformatted in > the message "WARNING: Illegal printf() format string: \"...\". I agree on principle but would like to ask if we need to revisit some of th= e=20 error cases. Especially with regard to 64bit porting there are some=20 "artifacts" that might cause serious pain for ported applications if the=20 above is adopted. Specifically, right now the following will warn "long long int format, int6= 4_t=20 arg (arg 2)" on our 64bit architectures while it is required on - at least = =2D=20 i386 int64_t i =3D 1; printf("%lld", i); Many other platforms allow it for 64bit architectures as well. As for all = our=20 64bit architectures sizeof(long) =3D=3D sizeof(long long) (as far as I am a= ware),=20 I am not convinced this should be a (fatal) error. There might be other=20 similar cases. So the question is, how strict should this check be? Are there cases where= we=20 are better off with a "just do it"-sollution? As a community service, there is a right way to do this (according to C99): int64_t i =3D 1; printf("%" PRIi64 "\n", i); but it's obvious this is not going to be adopted. The other often used=20 workaround is: int64_t i =3D 1; printf("%jd\n", (intmax_t)i); or: printf("%lld\n", (long long)i); which kind of reverts the idea behind useing C99-types. Note that: printf("%jd\n, i); seems to work as well, but I not sure this is correct. =2D-=20 /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --nextPart11122301.lkS6gTjF6f Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQBDnZqrXyyEoT62BG0RAlvXAJ9LxkWexHGT7ZC457d9690Gj6jNBwCdECb3 SnuQiz887BQ0tH0sSvZ/fgs= =iICx -----END PGP SIGNATURE----- --nextPart11122301.lkS6gTjF6f-- From owner-freebsd-arch@FreeBSD.ORG Mon Dec 12 15:49:18 2005 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A51E616A41F for ; Mon, 12 Dec 2005 15:49:18 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2DFC643D45 for ; Mon, 12 Dec 2005 15:49:18 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id 72269BC66; Mon, 12 Dec 2005 15:49:16 +0000 (UTC) To: Max Laier From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 12 Dec 2005 16:43:33 +0100." <200512121643.39236.max@love2party.net> Date: Mon, 12 Dec 2005 16:49:16 +0100 Message-ID: <2926.1134402556@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: freebsd-arch@freebsd.org Subject: Re: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2005 15:49:18 -0000 In message <200512121643.39236.max@love2party.net>, Max Laier writes: >I agree on principle but would like to ask if we need to revisit some of the >error cases. Especially with regard to 64bit porting there are some >"artifacts" that might cause serious pain for ported applications if the >above is adopted. > >Specifically, right now the following will warn "long long int format, int64_t >arg (arg 2)" on our 64bit architectures while it is required on - at least i386 > > int64_t i = 1; > printf("%lld", i); You misunderstood me. "%lld" is a legal formatting string, my printf implementation would never object to that. I'm talking about illegal/non-sensical formatting strings like "%lhd" or even "%!!!!d" and similar. The issue you raise is valid and important, but it is not for this bikeshed ;-) Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Dec 12 16:16:40 2005 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A81BA16A41F for ; Mon, 12 Dec 2005 16:16:40 +0000 (GMT) (envelope-from joseph.koshy@gmail.com) Received: from xproxy.gmail.com (xproxy.gmail.com [66.249.82.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id C6E3F43D5E for ; Mon, 12 Dec 2005 16:16:39 +0000 (GMT) (envelope-from joseph.koshy@gmail.com) Received: by xproxy.gmail.com with SMTP id i29so1010954wxd for ; Mon, 12 Dec 2005 08:16:39 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=Nq+XNihhlx0gKliEO4XYLDPlENm6Fc7FHhUiyyQ4cLd48HESQKDBCumcuHTczk/XhyI5JlM9GcoLZFguq1PdzF1l3XT9h9Uuv2f52jEFkyPp4NEQwz1s5DAccYW5776nMMrOmpfzs5es4xTL7Vdwf9BYA1zh2W6CEc89mNQCh5U= Received: by 10.70.60.2 with SMTP id i2mr9742618wxa; Mon, 12 Dec 2005 08:16:37 -0800 (PST) Received: by 10.70.105.13 with HTTP; Mon, 12 Dec 2005 08:16:37 -0800 (PST) Message-ID: <84dead720512120816t7c907c3aq9add32c5dc8b9a38@mail.gmail.com> Date: Mon, 12 Dec 2005 21:46:37 +0530 From: Joseph Koshy To: Poul-Henning Kamp In-Reply-To: <1023.1134389663@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <1023.1134389663@critter.freebsd.dk> Cc: arch@freebsd.org Subject: Re: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2005 16:16:40 -0000 phk> I'm leaning towards doing what phkmalloc has migrated to phk> over time: phk> Make a variable which can select between "normal/paranoia" phk> and force it to paranoia for (uid=3D=3D0 || gid=3D=3D0 || phk> setuid || setgid). phk> If the variable is set, a bogus format string will result phk> in abort(2). phk> If it is not set, the format string will be output phk> unformatted in the message "WARNING: Illegal printf() phk> format string: \"...\". Why not just print the warning for both cases, and stop interpreting the format string any further. What do we gain by having a uid 0 process dump core? -- FreeBSD Volunteer, http://people.freebsd.org/~jkoshy From owner-freebsd-arch@FreeBSD.ORG Mon Dec 12 16:19:03 2005 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BACCB16A41F for ; Mon, 12 Dec 2005 16:19:03 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.FreeBSD.org (Postfix) with ESMTP id 86A1E43D9D for ; Mon, 12 Dec 2005 16:18:54 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id 97B8DBC66; Mon, 12 Dec 2005 16:18:42 +0000 (UTC) To: Joseph Koshy From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 12 Dec 2005 21:46:37 +0530." <84dead720512120816t7c907c3aq9add32c5dc8b9a38@mail.gmail.com> Date: Mon, 12 Dec 2005 17:18:42 +0100 Message-ID: <3047.1134404322@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@freebsd.org Subject: Re: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2005 16:19:03 -0000 In message <84dead720512120816t7c907c3aq9add32c5dc8b9a38@mail.gmail.com>, Joseph Koshy writes: >Why not just print the warning for both cases, and >stop interpreting the format string any further. > >What do we gain by having a uid 0 process dump core? I'm not very keen on producing wrong output from uid0 programs without giving the programmer a way to handle the problem correctly. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Dec 12 19:19:08 2005 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 68CAE16A436 for ; Mon, 12 Dec 2005 19:19:08 +0000 (GMT) (envelope-from PeterJeremy@optushome.com.au) Received: from mail12.syd.optusnet.com.au (mail12.syd.optusnet.com.au [211.29.132.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 839A643D5D for ; Mon, 12 Dec 2005 19:18:45 +0000 (GMT) (envelope-from PeterJeremy@optushome.com.au) Received: from cirb503493.alcatel.com.au (c220-239-19-236.belrs4.nsw.optusnet.com.au [220.239.19.236]) by mail12.syd.optusnet.com.au (8.12.11/8.12.11) with ESMTP id jBCJIV74014232 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Tue, 13 Dec 2005 06:18:33 +1100 Received: from cirb503493.alcatel.com.au (localhost.alcatel.com.au [127.0.0.1]) by cirb503493.alcatel.com.au (8.12.10/8.12.10) with ESMTP id jBCJIVHh077100; Tue, 13 Dec 2005 06:18:31 +1100 (EST) (envelope-from pjeremy@cirb503493.alcatel.com.au) Received: (from pjeremy@localhost) by cirb503493.alcatel.com.au (8.12.10/8.12.9/Submit) id jBCJIUlP077099; Tue, 13 Dec 2005 06:18:30 +1100 (EST) (envelope-from pjeremy) Date: Tue, 13 Dec 2005 06:18:30 +1100 From: Peter Jeremy To: Poul-Henning Kamp Message-ID: <20051212191830.GD74684@cirb503493.alcatel.com.au> References: <1023.1134389663@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1023.1134389663@critter.freebsd.dk> User-Agent: Mutt/1.4.2.1i X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc Cc: arch@freebsd.org Subject: Re: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2005 19:19:08 -0000 On Mon, 2005-Dec-12 13:14:23 +0100, Poul-Henning Kamp wrote: >The context for the above is that I'm working on adding extensibility >to our printf, compatible with the GLIBC (see 12.13 in the glibc >manual). http://www.gnu.org/software/libc/manual/html_node/Customizing-Printf.html for anyone else wanting to see what it does. > Obviously, gcc cannot compile-time check such extensions >for us, and therefore the question gains a bit more relevance. There doesn't even appear to be a defined way of determining whether extensions are supported. Since not all libc's support printf extensions, an application that wants to use them has to confirm that they work and the only way to do this appears to be to try it and see what happens (either at runtime, or during a autoconf-style configuration process). >Another alternative is to spit out the format string unformatted, >possibly with an attached notice, but this doesn't really seem to >help anybody either, but at least indicates what the problem is. xterm does (or did) this if it is running as root and the format string contains conversion specification that it thinks are suspicious. >I'm leaning towards doing what phkmalloc has migrated to over time: >Make a variable which can select between "normal/paranoia" and force >it to paranoia for (uid==0 || gid==0 || setuid || setgid). > >If the variable is set, a bogus format string will result in abort(2). set{u,g}id programs won't dump core so just abort(2)ing leaves no trace of what went wrong. This makes finding the problem more difficult. Even for {u,g}id == 0 programs, it would be nice to have something reported (but see below). Note that this behaviour has implications for programs that are trying to determine if extensions are supported or not. >If it is not set, the format string will be output unformatted in >the message "WARNING: Illegal printf() format string: \"...\". Since this check presumably applies to the entire *printf() family, where do you report the error for {s,f}printf()? What do you define as an "illegal printf() format string"? I can think of four possible categories: 1) Using a nonsense value before '$', eg "%12345$d" 2) Having an invalid modifier on a builtin conversion specifier, eg "%hf" 3) Using an undefined conversion specified, eg '%W' 4) Having an invalid modifier on a user-specified conversion specifier The last category is particularly problematic because the glibc interface does not have any way to identify this error. The glibc documentation just states that the output handler conversion function (printf_function) should return a negative number if an error occurs so it's not possible to distinguish an invalid modifier from an I/O error or invalid argument. -- Peter Jeremy From owner-freebsd-arch@FreeBSD.ORG Mon Dec 12 19:35:48 2005 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3270816A420 for ; Mon, 12 Dec 2005 19:35:48 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.FreeBSD.org (Postfix) with ESMTP id 121E343D8C for ; Mon, 12 Dec 2005 19:35:41 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id 80474BC84; Mon, 12 Dec 2005 19:35:39 +0000 (UTC) To: Peter Jeremy From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 13 Dec 2005 06:18:30 +1100." <20051212191830.GD74684@cirb503493.alcatel.com.au> Date: Mon, 12 Dec 2005 20:35:39 +0100 Message-ID: <3879.1134416139@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@freebsd.org Subject: Re: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2005 19:35:48 -0000 In message <20051212191830.GD74684@cirb503493.alcatel.com.au>, Peter Jeremy writes: >>I'm leaning towards doing what phkmalloc has migrated to over time: >>Make a variable which can select between "normal/paranoia" and force >>it to paranoia for (uid==0 || gid==0 || setuid || setgid). >> >>If the variable is set, a bogus format string will result in abort(2). > >set{u,g}id programs won't dump core so just abort(2)ing leaves no >trace of what went wrong. That's one of the reason there is an "abort2(2)" system call in the works which allows the program to tell syslog why it comitted suicide. I have a patch in my inbox and I should really get it committed now. >>If it is not set, the format string will be output unformatted in >>the message "WARNING: Illegal printf() format string: \"...\". > >Since this check presumably applies to the entire *printf() family, >where do you report the error for {s,f}printf()? Whereever the strings was meant to go, what else can I do ? >What do you define as an "illegal printf() format string"? I can >think of four possible categories: >1) Using a nonsense value before '$', eg "%12345$d" >2) Having an invalid modifier on a builtin conversion specifier, eg "%hf" >3) Using an undefined conversion specified, eg '%W' >4) Having an invalid modifier on a user-specified conversion specifier Those are probably the primary suspects. >The last category is particularly problematic because the glibc >interface does not have any way to identify this error. My current plan is to provide a better API than GLIBC and make a couple of degraded glibc-api wrappers. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Dec 12 22:31:17 2005 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C4C5E16A41F for ; Mon, 12 Dec 2005 22:31:17 +0000 (GMT) (envelope-from geopapl@yahoo.com) Received: from web36306.mail.mud.yahoo.com (web36306.mail.mud.yahoo.com [209.191.84.236]) by mx1.FreeBSD.org (Postfix) with SMTP id B01EC43D49 for ; Mon, 12 Dec 2005 22:31:15 +0000 (GMT) (envelope-from geopapl@yahoo.com) Received: (qmail 46962 invoked by uid 60001); 12 Dec 2005 22:31:15 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=nGcRYV7fPllic2InMJMaYCpXrtxK9pSx26nMPtbSuCACKaKPflfLjcJHwKZ9QMBtCPKVfBctmVOLQIWzWo6gy8kZYfreGJHldWFaBsNPPe9CFH4J6/Uc69Jqp3gBBMNxlXgR+WlieTzlQd3zaL06z76AybvBt/j6QU8g1nZ75iU= ; Message-ID: <20051212223115.46960.qmail@web36306.mail.mud.yahoo.com> Received: from [83.171.211.247] by web36306.mail.mud.yahoo.com via HTTP; Mon, 12 Dec 2005 14:31:15 PST Date: Mon, 12 Dec 2005 14:31:15 -0800 (PST) From: George Paplas To: Poul-Henning Kamp , Peter Jeremy In-Reply-To: <3879.1134416139@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: arch@freebsd.org Subject: Re: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2005 22:31:17 -0000 --- Poul-Henning Kamp wrote: > >>If it is not set, the format string will be output unformatted in > >>the message "WARNING: Illegal printf() format string: \"...\". > > > >Since this check presumably applies to the entire *printf() family, > >where do you report the error for {s,f}printf()? > > Whereever the strings was meant to go, what else can I do ? And what if you are doing an sprintf to a buffer smaller than your warning message? -geop __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-arch@FreeBSD.ORG Tue Dec 13 01:29:08 2005 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ADC3616A423 for ; Tue, 13 Dec 2005 01:29:08 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.FreeBSD.org (Postfix) with ESMTP id AF70543D68 for ; Tue, 13 Dec 2005 01:28:58 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id 05BDABC66; Tue, 13 Dec 2005 01:28:56 +0000 (UTC) To: George Paplas From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 12 Dec 2005 14:31:15 PST." <20051212223115.46960.qmail@web36306.mail.mud.yahoo.com> Date: Tue, 13 Dec 2005 02:28:56 +0100 Message-ID: <7898.1134437336@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: Peter Jeremy , arch@freebsd.org Subject: Re: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2005 01:29:08 -0000 In message <20051212223115.46960.qmail@web36306.mail.mud.yahoo.com>, George Paplas writes: > > >--- Poul-Henning Kamp wrote: > >> >>If it is not set, the format string will be output unformatted in >> >>the message "WARNING: Illegal printf() format string: \"...\". >> > >> >Since this check presumably applies to the entire *printf() family, >> >where do you report the error for {s,f}printf()? >> >> Whereever the strings was meant to go, what else can I do ? > >And what if you are doing an sprintf to a buffer smaller than your >warning message? "Too f**king bad" It's the programmer who's made a mess in the first place, so he can't really make many demands in this situation... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Tue Dec 13 07:53:23 2005 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9BCAC16A41F for ; Tue, 13 Dec 2005 07:53:23 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailout1.pacific.net.au (mailout1.pacific.net.au [61.8.0.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id 57D1843D66 for ; Tue, 13 Dec 2005 07:53:22 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.0.87]) by mailout1.pacific.net.au (8.13.4/8.13.4/Debian-3) with ESMTP id jBD7rJTi012113; Tue, 13 Dec 2005 18:53:19 +1100 Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (8.13.4/8.13.4/Debian-3) with ESMTP id jBD7rFPW007222; Tue, 13 Dec 2005 18:53:16 +1100 Date: Tue, 13 Dec 2005 18:53:15 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Max Laier In-Reply-To: <200512121643.39236.max@love2party.net> Message-ID: <20051213175413.H80942@delplex.bde.org> References: <1023.1134389663@critter.freebsd.dk> <200512121643.39236.max@love2party.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Poul-Henning Kamp , freebsd-arch@freebsd.org Subject: Re: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2005 07:53:23 -0000 On Mon, 12 Dec 2005, Max Laier wrote: > On Monday 12 December 2005 13:14, Poul-Henning Kamp wrote: >> Obligatory bikeshed avoidance notice: >>>> Please read all the way to the bottom of this email before you reply << >> >> Given that illegal or malformed format strings to the printf family >> of functions mostly result in harmless misformatting but occationally >> in coredumps and maybe some times in security issues, what is the >> desired behaviour of libc's printf implementation ? It is to emit nasel daemons. Failing that, a core dump is best. >> A very good example is >> >> printf("%hf", 1.0); >> >> The 'h' modifier is not legal for %f format, and it is therefore a good >> bet that the programmer was confused and we know that the program >> contains at least one error. >> >> Our first line of defence against this kind of error is compile-time >> checking by GCC, but we cannot rely on the string being sane in libc, >> we still need to do error checking. There is also fmtcheck(3). >> The context for the above is that I'm working on adding extensibility >> to our printf, compatible with the GLIBC (see 12.13 in the glibc >> manual). Obviously, gcc cannot compile-time check such extensions >> for us, and therefore the question gains a bit more relevance. >> >> In an ideal world, the printf family of functions would have been >> defined to return EINVAL in this case. Almost nobody checks the >> return values of printf-like functions however and those few that >> do, all pressume that it is an I/O error so such an approach is >> unlikely to gain us much if anything. A core dump is best for these reasons. Also, no one uses fmtcheck(3), and there there is no way for standard-conforming programs to check for errors caused by undefined behaviour, since undefined behaviour includes undefined errors. I think most checking belongs in the compiler and in fmtcheck(3). printf() itself cannot detect most types errors without compiler support that would basically involve passing it the types of all args so that it could call fmtcheck(3). Extensions should rarely be needed for printf(), and it is not unreasonable to expect that applications to use the extensions to be more careful. Extensions might be more needed for printf-like interfaces that aren't really printf. >> Another alternative is to spit out the format string unformatted, >> possibly with an attached notice, but this doesn't really seem to >> help anybody either, but at least indicates what the problem is. >> >> I'm leaning towards doing what phkmalloc has migrated to over time: >> Make a variable which can select between "normal/paranoia" and force >> it to paranoia for (uid==0 || gid==0 || setuid || setgid). >> >> If the variable is set, a bogus format string will result in abort(2). This sometimes breaks defined behaviour. >> If it is not set, the format string will be output unformatted in >> the message "WARNING: Illegal printf() format string: \"...\". malloc()'s messages are better (": error ..."). > I agree on principle but would like to ask if we need to revisit some of the > error cases. Especially with regard to 64bit porting there are some > "artifacts" that might cause serious pain for ported applications if the > above is adopted. > > Specifically, right now the following will warn "long long int format, int64_t > arg (arg 2)" on our 64bit architectures while it is required on - at least - > i386 > > int64_t i = 1; > printf("%lld", i); Warning is the correct behaviour for this. Code like this isn't required on any arch, and is just wrong on arches where long long is longer than int64_t. Fortunately we have some arches where int64_t has a different type than long long, so code like this causes a warning so it doesn't get written so often. rms added the warning to gcc 15-18 years ago after I complained about the corresponding bad code for ints and long: int i = 1; printf("%ld", i); This is just wrong on arches where long is longer than int. > Many other platforms allow it for 64bit architectures as well. As for all our > 64bit architectures sizeof(long) == sizeof(long long) (as far as I am aware), > I am not convinced this should be a (fatal) error. There might be other > similar cases. int64_t = long != long long (although all the sizes are the same) is an artifact. We use the artificat mainly to detect errors so that many printf formats don't need to be "fixed" yet again for 128-bit machines. > So the question is, how strict should this check be? Are there cases where we > are better off with a "just do it"-sollution? Just doing it is the correct method. It requires the compiler to rewrite the string (or replace the call to printf by calls to special formatting functions). E.g., int64_t i64 = 1; long long ll = 1; int i = 1; printf("%I %I %I\n", i64, ll, i); should produce the same code as: int64_t i64 = 1; long long ll = 1; int i = 1; printf("%" PRIi64 "%ld %d\n", i64, ll, i); Message catalogs should probably use somthing like fmtcheck() to rewrite the strings. > As a community service, there is a right way to do this (according to C99): > > int64_t i = 1; > printf("%" PRIi64 "\n", i); This is a very wrong way to do this. It makes the programmer hande details that the compiler can handle better. > > but it's obvious this is not going to be adopted. The other often used Fortunately it is so ugly that no one uses it. > workaround is: > > int64_t i = 1; > printf("%jd\n", (intmax_t)i); This is not a workaround, but the only reasonable way to do things (unless %I exists). It is easier to use than PRI*, and doesn't need changing if the type of i is expanded. > or: > > printf("%lld\n", (long long)i); This is a wrong way to do this. It uses the long long mistake, and only works because i has type int64_t and long long is longer than int64_t (the latter is standard). > which kind of reverts the idea behind useing C99-types. > > Note that: > > printf("%jd\n, i); > > seems to work as well, but I not sure this is correct. It is incorrect. It assumes that int64_t has the same _size_ as intmax_t, and would cause a compile time error if int64_t doesn't have the same _type_ as intmax_t. We use intmax_t == int64_t on all arches, but this is an even weirder artifact than int64_t != long long, since it makes the "maximum" type "smaller" (same in size, but logically shorter) than long long: char < short < int < long = int64_t = intmax_t ~< long long Bruce From owner-freebsd-arch@FreeBSD.ORG Tue Dec 13 08:40:34 2005 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 18EC416A41F for ; Tue, 13 Dec 2005 08:40:34 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.FreeBSD.org (Postfix) with ESMTP id 529EA43D4C for ; Tue, 13 Dec 2005 08:40:33 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id 2831FBC66; Tue, 13 Dec 2005 08:40:29 +0000 (UTC) To: Bruce Evans From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 13 Dec 2005 18:53:15 +1100." <20051213175413.H80942@delplex.bde.org> Date: Tue, 13 Dec 2005 09:40:29 +0100 Message-ID: <9880.1134463229@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: Max Laier , freebsd-arch@freebsd.org Subject: Re: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2005 08:40:34 -0000 In message <20051213175413.H80942@delplex.bde.org>, Bruce Evans writes: >There is also fmtcheck(3). I didn't even know about that one, but given that there is only two uses in all of /src I do not feel ashamed. >Extensions should rarely be needed for printf(), Actually I disagree with you on that. It was my list of "things I keep doing over and over" that convinced me otherwise. Here are some of the formats I miss, and which I will probably write extensions for so people can trivially enable them: %T print a time_t %lT print a struct timeval %llT print a struct timespec %I print an IP# %lI print an IPv6# %H Hexdump %V stringvis a string %M Metric (like the "engineering" format on HP calculators) %H "Human" (Tera,Giga,Mega,Kilo{bits,bytes}) >>> I'm leaning towards doing what phkmalloc has migrated to over time: >>> Make a variable which can select between "normal/paranoia" and force >>> it to paranoia for (uid==0 || gid==0 || setuid || setgid). >>> >>> If the variable is set, a bogus format string will result in abort(2). > >This sometimes breaks defined behaviour. It does ? I didn't think there were defined behaviour for bogus format strings ? >>> If it is not set, the format string will be output unformatted in >>> the message "WARNING: Illegal printf() format string: \"...\". > >malloc()'s messages are better (": error ..."). Obviously. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Tue Dec 13 09:17:03 2005 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4820D16A41F for ; Tue, 13 Dec 2005 09:17:03 +0000 (GMT) (envelope-from PeterJeremy@optushome.com.au) Received: from mail20.syd.optusnet.com.au (mail20.syd.optusnet.com.au [211.29.132.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id 70B8443D5A for ; Tue, 13 Dec 2005 09:17:02 +0000 (GMT) (envelope-from PeterJeremy@optushome.com.au) Received: from cirb503493.alcatel.com.au (c220-239-19-236.belrs4.nsw.optusnet.com.au [220.239.19.236]) by mail20.syd.optusnet.com.au (8.12.11/8.12.11) with ESMTP id jBD9GwZU005165 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Tue, 13 Dec 2005 20:16:58 +1100 Received: from cirb503493.alcatel.com.au (localhost.alcatel.com.au [127.0.0.1]) by cirb503493.alcatel.com.au (8.12.10/8.12.10) with ESMTP id jBD9GvHh077969; Tue, 13 Dec 2005 20:16:57 +1100 (EST) (envelope-from pjeremy@cirb503493.alcatel.com.au) Received: (from pjeremy@localhost) by cirb503493.alcatel.com.au (8.12.10/8.12.9/Submit) id jBD9GvKW077968; Tue, 13 Dec 2005 20:16:57 +1100 (EST) (envelope-from pjeremy) Date: Tue, 13 Dec 2005 20:16:56 +1100 From: Peter Jeremy To: Bruce Evans Message-ID: <20051213091656.GD77268@cirb503493.alcatel.com.au> References: <1023.1134389663@critter.freebsd.dk> <200512121643.39236.max@love2party.net> <20051213175413.H80942@delplex.bde.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20051213175413.H80942@delplex.bde.org> User-Agent: Mutt/1.4.2.1i X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc Cc: freebsd-arch@freebsd.org Subject: Re: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2005 09:17:03 -0000 On Tue, 2005-Dec-13 18:53:15 +1100, Bruce Evans wrote: >>>Our first line of defence against this kind of error is compile-time >>>checking by GCC, but we cannot rely on the string being sane in libc, >>>we still need to do error checking. > >There is also fmtcheck(3). I'm probably not the only person who hadn't heard of this function before reading your mail. I'm not sure it is of much use other than validating user-supplied format strings (and maybe internationalisation translations via catgets(3)). >I think most checking belongs in the compiler and in fmtcheck(3). fmtcheck(3) can't check that the arguments passed to printf are valid. All it can do is verify that two arbitrary format strings expect the same arguments and the documentation states it doesn't even manage that: The fmtcheck() function does not understand all of the conversions that printf(3) does. Note that fmtcheck(3) has no way of reporting that neither format string makes sense - the documentation states that it will always return one of its arguments. If a programmer (mistakenly) believes that "%hf" is sensible then fmtcheck(3) isn't going to help. >printf() itself cannot detect most types errors without compiler >support that would basically involve passing it the types of all >args so that it could call fmtcheck(3). This is an interesting idea but fmtcheck(3) isn't adequate. In particular, consider: int i, j; const char *fmt, *string; ... fmt = "%*.*s %p"; ... printf(fmt, i, j, string, string); Based on the printf() arguments, the compiler can't really do better than: printf(fmtcheck(fmt, "%d %d %s %s"), i, j, string, string); but fmtcheck(3) doesn't consider that "%*.*s %p" and "%d %d %s %s" are equivalent. What could be useful is a function that takes a format string and a set of argument types and verifies that the argument types match the format string. The compiler could then expand the printf() to: if (fmtvalidate(fmt, "int;int;char*;char*")) printf(fmt, i, j, string, string); else abort(); (though it's not clear how the compiler should handle struct/union pointers which might make sense to a user extension). >>>If the variable is set, a bogus format string will result in abort(2). > >This sometimes breaks defined behaviour. I thought printf(3) documented the behaviour for invalid conversion specifiers and mofidiers but I can't find it in the man page right now. -- Peter Jeremy From owner-freebsd-arch@FreeBSD.ORG Tue Dec 13 12:41:40 2005 Return-Path: X-Original-To: freebsd-arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AB41E16A420 for ; Tue, 13 Dec 2005 12:41:40 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailout2.pacific.net.au (mailout2.pacific.net.au [61.8.0.115]) by mx1.FreeBSD.org (Postfix) with ESMTP id F067343D6A for ; Tue, 13 Dec 2005 12:41:38 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.0.87]) by mailout2.pacific.net.au (8.13.4/8.13.4/Debian-3) with ESMTP id jBDCfbfq004209; Tue, 13 Dec 2005 23:41:37 +1100 Received: from epsplex.bde.org (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (8.13.4/8.13.4/Debian-3) with ESMTP id jBDCfYDv005650; Tue, 13 Dec 2005 23:41:35 +1100 Date: Tue, 13 Dec 2005 23:41:34 +1100 (EST) From: Bruce Evans X-X-Sender: bde@epsplex.bde.org To: Poul-Henning Kamp In-Reply-To: <9880.1134463229@critter.freebsd.dk> Message-ID: <20051213230723.T3248@epsplex.bde.org> References: <9880.1134463229@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Max Laier , freebsd-arch@FreeBSD.org Subject: Re: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2005 12:41:40 -0000 On Tue, 13 Dec 2005, Poul-Henning Kamp wrote: > In message <20051213175413.H80942@delplex.bde.org>, Bruce Evans writes: > >> There is also fmtcheck(3). > > I didn't even know about that one, but given that there is only two > uses in all of /src I do not feel ashamed. I learned about it commit mail (or arch?) when Kris was sweeping for security holes related to printf formats. >> Extensions should rarely be needed for printf(), > > Actually I disagree with you on that. > > It was my list of "things I keep doing over and over" that convinced > me otherwise. Now I think they should be very rarely needed and more rarely used. Using them mainly gives unportable code that breaks especially badly on systems which don't support extensions. > Here are some of the formats I miss, and which I will probably write > extensions for so people can trivially enable them: > > %T print a time_t > %lT print a struct timeval > %llT print a struct timespec > %I print an IP# > %lI print an IPv6# > %H Hexdump > %V stringvis a string > %M Metric (like the "engineering" format on HP calculators) > %H "Human" (Tera,Giga,Mega,Kilo{bits,bytes}) I think these belong in specialized applications or libraries. %T is already handled better by strftime/gmtime/localtime. It has lots of subformats and delicate conversion issues. A generic %T couldn't reasonably support much more than "%[#0- +,]*.*T". If a generic version were implemented as a function in libc, then printf("%T", asprintf_time_t(tt)) wouldn't be much harder to write than printf("%T", tt), but storage management for it would be harder. Maybe you really want to write cout << tt :-). >>>> I'm leaning towards doing what phkmalloc has migrated to over time: >>>> Make a variable which can select between "normal/paranoia" and force >>>> it to paranoia for (uid==0 || gid==0 || setuid || setgid). >>>> >>>> If the variable is set, a bogus format string will result in abort(2). >> >> This sometimes breaks defined behaviour. > > It does ? I didn't think there were defined behaviour for bogus > format strings ? I mean aborting instead of returning NULL for failing malloc()s breaks defined behaviour. Bruce From owner-freebsd-arch@FreeBSD.ORG Tue Dec 13 13:01:02 2005 Return-Path: X-Original-To: freebsd-arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1533A16A41F for ; Tue, 13 Dec 2005 13:01:02 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.FreeBSD.org (Postfix) with ESMTP id E31D943D76 for ; Tue, 13 Dec 2005 13:00:56 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id 6C4F7BC66; Tue, 13 Dec 2005 13:00:51 +0000 (UTC) To: Bruce Evans From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 13 Dec 2005 23:41:34 +1100." <20051213230723.T3248@epsplex.bde.org> Date: Tue, 13 Dec 2005 14:00:51 +0100 Message-ID: <12349.1134478851@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: Max Laier , freebsd-arch@FreeBSD.org Subject: Re: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2005 13:01:02 -0000 In message <20051213230723.T3248@epsplex.bde.org>, Bruce Evans writes: >Now I think they should be very rarely needed and more rarely used. >Using them mainly gives unportable code that breaks especially badly >on systems which don't support extensions. Portability is good, but it shouldn't get in the way of improving our programs. >I think these belong in specialized applications or libraries. %T is >already handled better by strftime/gmtime/localtime. There's no handling for fractional seconds there. >I mean aborting instead of returning NULL for failing malloc()s breaks >defined behaviour. Right, that's deliberate. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Tue Dec 13 13:54:11 2005 Return-Path: X-Original-To: freebsd-arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7D46B16A422 for ; Tue, 13 Dec 2005 13:54:11 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailout1.pacific.net.au (mailout1.pacific.net.au [61.8.0.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id 96B3C43D6D for ; Tue, 13 Dec 2005 13:54:09 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.0.87]) by mailout1.pacific.net.au (8.13.4/8.13.4/Debian-3) with ESMTP id jBDDs74V009274; Wed, 14 Dec 2005 00:54:07 +1100 Received: from epsplex.bde.org (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (8.13.4/8.13.4/Debian-3) with ESMTP id jBDDs43R022399; Wed, 14 Dec 2005 00:54:05 +1100 Date: Wed, 14 Dec 2005 00:54:05 +1100 (EST) From: Bruce Evans X-X-Sender: bde@epsplex.bde.org To: Peter Jeremy In-Reply-To: <20051213091656.GD77268@cirb503493.alcatel.com.au> Message-ID: <20051213234201.E3248@epsplex.bde.org> References: <1023.1134389663@critter.freebsd.dk> <200512121643.39236.max@love2party.net> <20051213175413.H80942@delplex.bde.org> <20051213091656.GD77268@cirb503493.alcatel.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-arch@FreeBSD.org Subject: Re: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2005 13:54:11 -0000 On Tue, 13 Dec 2005, Peter Jeremy wrote: > On Tue, 2005-Dec-13 18:53:15 +1100, Bruce Evans wrote: >>>> Our first line of defence against this kind of error is compile-time >>>> checking by GCC, but we cannot rely on the string being sane in libc, >>>> we still need to do error checking. >> >> There is also fmtcheck(3). > > I'm probably not the only person who hadn't heard of this function > before reading your mail. I'm not sure it is of much use other than > validating user-supplied format strings (and maybe internationalisation > translations via catgets(3)). I think internationalisation is its main use now. >> I think most checking belongs in the compiler and in fmtcheck(3). > > fmtcheck(3) can't check that the arguments passed to printf are valid. > All it can do is verify that two arbitrary format strings expect the > same arguments and the documentation states it doesn't even manage that: > The fmtcheck() function does not understand all of the conversions > that printf(3) does. Its example also starts by saying that %p is compatible with %lu. Punning "void *" as "unsigned long" might be safe enough if these types have the same size, but fmtcheck()'s code doesn't seem to have any size checks. > If a programmer (mistakenly) believes that "%hf" is sensible then > fmtcheck(3) isn't going to help. This can be checked statically -- check the default at compile time; then any format that actually consumes the same arg types as the default will consume the ones passed. fmtcheck() could also rewrite the suspect format -- you could put all type info in the default and all other info in the suspect format and merge at runtime. E.g., afmtcheck("%-*I %#8I", "%*s%jx") = strdup("%-*s %#8jx"). >> printf() itself cannot detect most types errors without compiler >> support that would basically involve passing it the types of all >> args so that it could call fmtcheck(3). > > This is an interesting idea but fmtcheck(3) isn't adequate. In > particular, consider: > int i, j; > const char *fmt, *string; > ... > fmt = "%*.*s %p"; > ... > printf(fmt, i, j, string, string); > > Based on the printf() arguments, the compiler can't really do > better than: > printf(fmtcheck(fmt, "%d %d %s %s"), i, j, string, string); > but fmtcheck(3) doesn't consider that "%*.*s %p" and "%d %d %s %s" are > equivalent. So more that ordinary C type info is needed. The compile would have to parse the format string (which is possible in the above since fmt is const and visible but not in all cases) and pass "%*.*s %p" not "%d %d %s %s". > What could be useful is a function that takes a format string and > a set of argument types and verifies that the argument types match > the format string. The compiler could then expand the printf() to: > if (fmtvalidate(fmt, "int;int;char*;char*")) > printf(fmt, i, j, string, string); > else > abort(); > (though it's not clear how the compiler should handle struct/union > pointers which might make sense to a user extension). Both this and my afmtcheck() example could be merged into printf() itself. printf("%*.*s %p", width, prec, s, s1) could be transformed into printf("%C%*.*s %p", "%*.*s%p", width, prec, s, (void *)s1). %C indiciates a check arg and that arg is "%*.*s%p". The check arg is amost exactly the original string here but could be very different if the compiler doesn't understand the format or the check arg is from a message catalog. Here the compiler understands the format and does the following transformations: - %*.*s -> itself. No problem since the args match. - " " -> "". Not related to args. - "%s" -> "%p". The args don't match, but this is harmless for char * vs void * and the compiler has silently fixed the arg type to match the format. Perhaps it should leave this to printf(). I don't see any special problem with struct/union pointers. The compiler just won't usually be able to understand the pointed-to objects or parts of the format string related to them. An escape sequence might be needed to tell it not to warn about extensions, but pass the escaped format and args directly to the format checker. >>>> If the variable is set, a bogus format string will result in abort(2). >> >> This sometimes breaks defined behaviour. I meant in malloc(). > I thought printf(3) documented the behaviour for invalid conversion > specifiers and mofidiers but I can't find it in the man page right now. The C standard mostly says "undefined" for nonstandard things in fprintf(). >From n869.txt: [#9] If a conversion specification is invalid, the behavior is undefined.225) ... Footnote 225 points to future library extensions. Conversions specs have to be undefined so that everyone can define inconsistent extensions :). Bruce From owner-freebsd-arch@FreeBSD.ORG Tue Dec 13 17:36:55 2005 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E4EB016A41F for ; Tue, 13 Dec 2005 17:36:55 +0000 (GMT) (envelope-from babkin@verizon.net) Received: from vms046pub.verizon.net (vms046pub.verizon.net [206.46.252.46]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7B82043D60 for ; Tue, 13 Dec 2005 17:36:55 +0000 (GMT) (envelope-from babkin@verizon.net) Received: from vms076.mailsrvcs.net ([192.168.1.1]) by vms046.mailsrvcs.net (Sun Java System Messaging Server 6.2-4.02 (built Sep 9 2005)) with ESMTPA id <0IRG007R1665U3A3@vms046.mailsrvcs.net> for arch@freebsd.org; Tue, 13 Dec 2005 11:34:53 -0600 (CST) Date: Tue, 13 Dec 2005 11:34:53 -0600 (CST) From: Sergey Babkin To: Poul-Henning Kamp , George Paplas Message-id: <12481749.1134495293852.JavaMail.root@vms076.mailsrvcs.net> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7bit Cc: Peter Jeremy , arch@freebsd.org Subject: Re: Re: printf behaviour with illegal or malformed format string X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: babkin@users.sf.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2005 17:36:56 -0000 >From: Poul-Henning Kamp >In message <20051212223115.46960.qmail@web36306.mail.mud.yahoo.com>, George Paplas writes: >> >> >>--- Poul-Henning Kamp wrote: >> >>> >>If it is not set, the format string will be output unformatted in >>> >>the message "WARNING: Illegal printf() format string: \"...\". >>> > >>> >Since this check presumably applies to the entire *printf() family, >>> >where do you report the error for {s,f}printf()? >>> >>> Whereever the strings was meant to go, what else can I do ? >> >>And what if you are doing an sprintf to a buffer smaller than your >>warning message? > >"Too f**king bad" > >It's the programmer who's made a mess in the first place, so he >can't really make many demands in this situation... Probably the least intrusive thing is to print the original message to the best ability of deciphering it, and then print the error message on stderr (kind of like the message printed on the first usage of gets() and such). -SB From owner-freebsd-arch@FreeBSD.ORG Fri Dec 16 15:43:12 2005 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7502F16A41F; Fri, 16 Dec 2005 15:43:12 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1D02B43D46; Fri, 16 Dec 2005 15:43:11 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.13.5/8.13.5/NETPLEX) with ESMTP id jBGFhAVj015476; Fri, 16 Dec 2005 10:43:11 -0500 (EST) Date: Fri, 16 Dec 2005 10:43:10 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: arch@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) Cc: standards@freebsd.org Subject: (Fwd) Extended API Set Part 1 Technical Standard Company Review (fwd) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2005 15:43:12 -0000 FYI, if you want to particpate, here's your chance. -- DE ---------- Forwarded message ---------- Date: Fri, 16 Dec 2005 11:27:31 GMT From: Andrew Josey To: austin-group-l@opengroup.org Subject: (Fwd) Extended API Set Part 1 Technical Standard Company Review Resent-Date: 16 Dec 2005 11:28:02 -0000 Resent-From: austin-group-l@opengroup.org Resent-To: austin-group-l@opengroup.org For your information: --- Forwarded mail Subject: Extended API Set Part 1 Technical Standard Company Review +--------------------------------+ To : All Open Group Members The Open Group Extended API Set Part 1 From : Andrew Josey Technical Standard Company Review +--------------------------------+ Date : 16 December 2005 Dear Open Group Member, I hereby announce the Company Review of the Extended API Set Part 1 Technical Standard under the Direct Review procedure, with the Platform Forum as sponsor of the review. The purpose of this Technical Standard is to define a set of New API Extensions to further increase application capture and hence portability for systems built upon version 3 of the Single UNIX Specification . The scope of this set of extensions has been to consider interfaces drawn from existing open source implementations such as the GNU C library. The Open Group's Base Working Group intends to submit this document once approved to the Austin Group for consideration as input into the next revision of the joint standard (IEEE Std POSIX 1003.1 and The Open Group Base Specifications). An additional set of API extensions will also be submitted for Company review as the Extended API Set Part 2 in January 2006. For the benefit of those unfamiliar with The Open Group Company Review process, it is a formal process by which a document is approved for publication by The Open Group. You are invited to review and submit proposed changes ("Change Requests") which would make the document acceptable to you. A formal ballot then decides which changes are accepted. Timetable --------- The Company Review starts on January 20th 2006, and finishes on February 17 2006, and is open to all members of The Open Group. The timetable is as follows: Review materials available 13 Jan Company review start 20 Jan Company review end 17 Feb Change Request review meeting w/c Feb 20 Ottawa, Canada Recommendations posted for ballot 28 Feb Ballot of recommendations 1-7 March Address unresolved issues w/c 13 March 'Sanity check' draft review 21 March Board approval to be scheduled Apr 2006 Materials ------------ The review materials will consist of the Extended API Set Part 1 Participants ------------ Andrew Josey will conduct the review process on behalf of The Open Group. The review group is the members of the Platform Forum plus all other interested members from any other forum or group within The Open Group membership, plus associated groups such as the Austin Group. The balloting group that will decide on Change Requests is the members of the Platform Forum. How to Participate ------------------ The Company Review will be executed using The Open Group's web based review facility. Details of how to access the draft and submit comments will be published nearer the time, when the Company Review draft is made available on The Open Group web server. Overview of the Extended API Set Part 1 The Extended API Set Part 1 is expected to consist of twenty-five new system interfaces, and one extension to the ls utility. It also introduces the concept of a stream associated with a memory buffer to eliminate may of the errors encountered in the construction of strings, notably overflowing of strings. The system interfaces are as follows (listed by header): alphasort() dirfd() scandir() psignal() psiginfo() dprintf() fmemopen() getdelim() getline() open_memstream() open_wmemstream() mkdtemp() stpcpy() stpncpy() strndup() strnlen() strsignal() mbsnrtowcs() wcpcpy() wcpncpy() wcscasecmp() wcsdup() wcsncasecmp() wcsnlen() wcsnrtombs() Procedure --------- In line with The Open Group's Technical Procedures: 1. All requests for modifications must be the subject of formal change requests (CRs - see below). They should be submitted via the web interface that will be available). 2. Shortly after the close of the Company Review period, The Open Group will hold a change request review meeting where proposed resolutions to the CRs will be decided by the ballot group. The Open Group will then issue a ballot table showing all the CRs received, and will call for the balloting group to return their votes by the due date. 3. ALL members entitled to vote shall return their votes by electronic mail to the ballot group alias on EVERY CR in the ballot table. 4. The Open Group will issue a ballot results table after close of ballot. A `clear majority' is declared on each proposal if and only if at least 75% of votes cast (excluding abstentions) are in the same direction; i.e. a proposal can be `clearly accepted' or `clearly rejected'. A CR that fails to achieve a clear majority is declared `unresolved', in which case it will be decided in an Issues Resolution teleconference of the balloteers. 5. The Open Group will announce the final results immediately following resolution of all outstanding issues arising from the Company Review. 6. Sanity proof documents will then be prepared and the final results submitted for approval by The Open Group Board of Directors. CHANGE REQUESTS --------------- All requests for change must be submitted via the web interface. This should include the following: Change number: with sequence numbers of form where are company initials and is change request number of the form [digit][digit][digit] Allocate numbers sequentially within your company. Title: one-liner describing the request Qualifier: nature and severity of the change requested. Possible values are: - Severity: Minor, Major, Critical - Nature: Editorial, Technical Rationale: reason that the change is needed or is preferable to what currently exists. Change: include unambiguous identification of precise text or illustration to be changed (added, deleted, or modified). Then specify detailed edits which you propose should be made. If you identify a problem but feel unable to define an acceptable change, then you should explain this and include sufficient information by example to enable other reviewers to fully understand your concern. CHANGE REQUESTS WITHOUT EXPLICIT EDITING INSTRUCTIONS ARE NON-DETERMINISTIC SO CANNOT BE BALLOTED. regards Andrew ----- Andrew Josey The Open Group Director, Server Platforms Thames Tower, 37-45 Station Road Email: a.josey:opengroup.org Reading,Berks.RG1 1LX,England Tel: +44 118 9508311 ext 2250 Fax: +44 118 9500110 Mobile: +44 774 015 5794 UNIX is a registered trademark of The Open Group in the US and other countries. ---End of forwarded mail