From owner-svn-src-user@FreeBSD.ORG Sun Nov 15 20:32:07 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2563B106566B; Sun, 15 Nov 2009 20:32:07 +0000 (UTC) (envelope-from edwin@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id EDBBB8FC15; Sun, 15 Nov 2009 20:32:06 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAFKW6j7047300; Sun, 15 Nov 2009 20:32:06 GMT (envelope-from edwin@svn.freebsd.org) Received: (from edwin@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAFKW6Lv047299; Sun, 15 Nov 2009 20:32:06 GMT (envelope-from edwin@svn.freebsd.org) Message-Id: <200911152032.nAFKW6Lv047299@svn.freebsd.org> From: Edwin Groothuis Date: Sun, 15 Nov 2009 20:32:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199294 - in user/edwin/locale/usr.bin: unicode2utf8 unicodename2utf8 X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Nov 2009 20:32:07 -0000 Author: edwin Date: Sun Nov 15 20:32:06 2009 New Revision: 199294 URL: http://svn.freebsd.org/changeset/base/199294 Log: Rename from unicode2utf8 to unicodename2utf8, as suggested by gabor@ Added: user/edwin/locale/usr.bin/unicodename2utf8/ - copied from r199293, user/edwin/locale/usr.bin/unicode2utf8/ Deleted: user/edwin/locale/usr.bin/unicode2utf8/ From owner-svn-src-user@FreeBSD.ORG Sun Nov 15 20:35:25 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2136E106566C; Sun, 15 Nov 2009 20:35:25 +0000 (UTC) (envelope-from edwin@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 0FD928FC14; Sun, 15 Nov 2009 20:35:25 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAFKZOGr047452; Sun, 15 Nov 2009 20:35:24 GMT (envelope-from edwin@svn.freebsd.org) Received: (from edwin@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAFKZORZ047451; Sun, 15 Nov 2009 20:35:24 GMT (envelope-from edwin@svn.freebsd.org) Message-Id: <200911152035.nAFKZORZ047451@svn.freebsd.org> From: Edwin Groothuis Date: Sun, 15 Nov 2009 20:35:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199295 - user/edwin/locale/usr.bin/unicodename2utf8 X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Nov 2009 20:35:25 -0000 Author: edwin Date: Sun Nov 15 20:35:24 2009 New Revision: 199295 URL: http://svn.freebsd.org/changeset/base/199295 Log: Rename unicode2utf8.1 to unicodename2utf8.1. Added: user/edwin/locale/usr.bin/unicodename2utf8/unicodename2utf8.1 - copied unchanged from r199294, user/edwin/locale/usr.bin/unicodename2utf8/unicode2utf8.1 Deleted: user/edwin/locale/usr.bin/unicodename2utf8/unicode2utf8.1 Copied: user/edwin/locale/usr.bin/unicodename2utf8/unicodename2utf8.1 (from r199294, user/edwin/locale/usr.bin/unicodename2utf8/unicode2utf8.1) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/edwin/locale/usr.bin/unicodename2utf8/unicodename2utf8.1 Sun Nov 15 20:35:24 2009 (r199295, copy of r199294, user/edwin/locale/usr.bin/unicodename2utf8/unicode2utf8.1) @@ -0,0 +1,91 @@ +.\" Copyright (c) 2009 Edwin Groothuis +.\" All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.Dd October 4, 2009 +.Dt unicode2utf8 1 +.Os +.Sh NAME +.Nm unicode2utf8 +.Nd converts a file with Unicode name definitions into UTF-8 character +definitions. +.Sh SYNOPSIS +.Nm +.Fl -cldr Ar directory +.Fl -input Ar filename +.Fl -output Ar filename +.Sh DESCRIPTION +The +.Nm +utility is made to convert the Unicode encoded strings in the +contents of the specified input file with the corresponding UTF-8 +character definitions. +.Pp +Lines starting with a # are copied as-is. +.Pp +The Unicode encoded strings are specified between a '<' and a '>' +sign. +They are looked up against the keys in the conversion table specified +in the file +.Pa posix/UTF-8.cm +in the with +.Fl -cldr +defined directory and the matching value is written out. +.Pp +Other characters are copied as-is. +.Sh OPTIONS +.Bl -tag -width indent +.It Fl -cldr Ar directory +The directory where the file +.Pa posix/UTF-8.cm +resides. +By default this should point to +.Pa /usr/share/misc , +but for maintainers of the FreeBSD locale database this could point +to their own extracted copy of the CLDR database. +.It Fl -input Ar filename +The source file with the Unicode encoded strings. +.It Fl -output Ar filename +The destination file with the Unicode encoded strings replaced with +their UTF-8 equivalents. +.El +.Sh EXIT STATUS +The +.Nm +utility exits 0 on success, and >0 if an error occurs. +.Sh SEE ALSO +.Xr iconv 1 , +.Xr bsdiconv 1 +.Bl -tag -width indent +.It http://cldr.unicode.org/ +Website of the Common Locale Database Repository, +the maintainers of the file +.Pa /usr/share/misc/UTF-8.cm +.El +.Sh AUTHORS +The +.Nm +utility and this manual page were written by +.An Edwin Groothuis Aq edwin@FreeBSD.org . From owner-svn-src-user@FreeBSD.ORG Sun Nov 15 20:36:12 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 13D5C1065694; Sun, 15 Nov 2009 20:36:12 +0000 (UTC) (envelope-from edwin@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 025D08FC22; Sun, 15 Nov 2009 20:36:12 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAFKaB56047504; Sun, 15 Nov 2009 20:36:11 GMT (envelope-from edwin@svn.freebsd.org) Received: (from edwin@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAFKaB4Q047502; Sun, 15 Nov 2009 20:36:11 GMT (envelope-from edwin@svn.freebsd.org) Message-Id: <200911152036.nAFKaB4Q047502@svn.freebsd.org> From: Edwin Groothuis Date: Sun, 15 Nov 2009 20:36:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199296 - user/edwin/locale/usr.bin/unicodename2utf8 X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Nov 2009 20:36:12 -0000 Author: edwin Date: Sun Nov 15 20:36:11 2009 New Revision: 199296 URL: http://svn.freebsd.org/changeset/base/199296 Log: Rename unicode2utf8.c to unicodename2utf8.c Added: user/edwin/locale/usr.bin/unicodename2utf8/unicodename2utf8.c - copied unchanged from r199295, user/edwin/locale/usr.bin/unicodename2utf8/unicode2utf8.c Deleted: user/edwin/locale/usr.bin/unicodename2utf8/unicode2utf8.c Copied: user/edwin/locale/usr.bin/unicodename2utf8/unicodename2utf8.c (from r199295, user/edwin/locale/usr.bin/unicodename2utf8/unicode2utf8.c) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/edwin/locale/usr.bin/unicodename2utf8/unicodename2utf8.c Sun Nov 15 20:36:11 2009 (r199296, copy of r199295, user/edwin/locale/usr.bin/unicodename2utf8/unicode2utf8.c) @@ -0,0 +1,265 @@ +/*- + * Copyright (c) 2009 Edwin Groothuis + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +#include + +#include +#include +#include +#include +#include +#include +#include + +#define MAXBUF 512 + +struct utf8map { + char *uniname; + char *utf8char; + int utf8len; + struct utf8map *next; +}; + +struct utf8map *utf8map_head[256]; + +void usage(void); +struct utf8map *get_utf8map(char *dir); +struct utf8map *find_utf8map(char *unidata); +void translate(char *file_in, char *file_out); + +int debug = 0; + +int +main(int argc, char *argv[]) +{ + char *cldr = NULL, *file_in = NULL, *file_out = NULL; + char ch; + + static struct option longopts[] = { + { "cldr", required_argument, NULL, 1 }, + { "input", required_argument, NULL, 2 }, + { "output", required_argument, NULL, 3 }, + { "debug", no_argument, NULL, 4 }, + { NULL, 0, NULL, 0 } + }; + + while ((ch = getopt_long_only(argc, argv, "", longopts, NULL)) != -1) { + switch (ch) { + case 1: + cldr = optarg; + break; + case 2: + file_in = optarg; + break; + case 3: + file_out = optarg; + break; + case 4: + debug++; + break; + default: + usage(); + } + } + argc -= optind; + argv += optind; + + if (cldr == NULL || file_in == NULL || file_out == NULL) + usage(); + + get_utf8map(cldr); + translate(file_in, file_out); +} + +void +translate(char *file_in, char *file_out) +{ + struct utf8map *map; + FILE *fin, *fout; + char *p, *q1, *q2, line[MAXBUF]; + + if ((fin = fopen(file_in, "r")) == NULL) + errx(EX_DATAERR, "Cannot open %s for reading.", file_in); + if ((fout = fopen(file_out, "w")) == NULL) + errx(EX_DATAERR, "Cannot open %s for writing.", file_out); + + fprintf(fout, +"#\n" +"# Do not edit this file, it is created automatically by the unicode2utf8\n" +"# utility. All changes to this file will be lost.\n" +"# The source of this file was %s\n" +"#\n", + file_in); + + while (!feof(fin)) { + if (fgets(line, sizeof(line), fin) != NULL) { + if (line[0] == '#') { + fprintf(fout, "%s", line); + continue; + } + + p = line; + while (*p != '\0') { + if (*p != '<') { + fputc(*p, fout); + p++; + continue; + } + q1 = strchr(p + 1, '>'); + q2 = strchr(p + 1, '<'); + if (q2 != NULL && q2 < q1) + errx(EX_DATAERR, + "Unexpected < in line %s after %s", + line, p); + *q1 = '\0'; + if ((map = find_utf8map(p + 1)) ==NULL) + errx(EX_DATAERR, + "Cannot find translation for '%s'", + p + 1); + + *q1 = '>'; + p = q1 + 1; + fwrite(map->utf8char, map->utf8len, 1, fout); + } + + } + } + + fclose(fin); + fclose(fout); +} + +struct utf8map * +find_utf8map(char *uniname) +{ + struct utf8map *p; + int hashindex = uniname[strlen(uniname) - 1]; + + p = utf8map_head[hashindex]; + while (p != NULL) { + if (strcmp(p->uniname, uniname) == 0) + return p; + if (debug) + printf("'%s' - '%s'\n", p->uniname, uniname); + p = p->next; + } + + return NULL; +} + +struct utf8map * +get_utf8map(char *dir) +{ + struct utf8map *new; + FILE *fin; + int len, i, hashindex; + char filename[MAXPATHLEN], uniname[MAXBUF], utf8char[MAXBUF], *p; + + sprintf(filename, "%s/posix/UTF-8.cm", dir); + + if ((fin = fopen(filename, "r")) == NULL) + errx(EX_DATAERR, "Cannot open UTF-8 in %s/posix", filename); + + while (!feof(fin)) { + if (fgets(uniname, sizeof(uniname), fin) != NULL) + if (strncmp(uniname, "CHARMAP", 7) == 0) + break; + } + if (feof(fin)) + errx(EX_DATAERR, "Didn't find initial CHARMAP magic cookie.\n"); + + while (!feof(fin)) { + if (fscanf(fin, "%s %s", uniname, utf8char) == 2) { + /* ^END CHARMAP$ */ + if (strcmp(uniname, "END") == 0 + && strcmp(utf8char, "CHARMAP") == 0) + break; + + /* Get rid of the _'s in the name. */ + while ((p = strchr(uniname, '_')) != NULL) + *p = ' '; + if ((p = strchr(uniname, '>')) == NULL) + errx(EX_DATAERR, "No trailing '>' for %s", + uniname); + hashindex = p[-1]; + *p = '\0'; + if (uniname[0] != '<') + errx(EX_DATAERR, "No leading '<' for %s", + uniname); + + /* Translate hex strings into ascii-strings. */ + len = strlen(utf8char); + if (len % 4 != 0) + errx(EX_DATAERR, "Wrong length: '%s'", + utf8char); + len /= 4; + for (i = 0; i < len; i++) { + /* + * Not setting will produce wrong results for + * the unicode string NULL. + */ + errno = 0; + + /* "\xAA" -> "AA" -> chr(hextodec("AA")) */ + utf8char[i] = strtol(utf8char + 4 * i + 2, NULL, + 16); + if (utf8char[i] == 0 && errno != 0) + errx(errno, + "'%s' isn't a hex digit (%d)", + utf8char + 4 * i + 2, errno); + utf8char[len] = 0; + } + + if (debug) + printf("-%s-%s-\n", uniname, utf8char); + + new = (struct utf8map *)malloc(sizeof(struct utf8map)); + new->next = utf8map_head[hashindex]; + new->uniname = strdup(uniname + 1); + new->utf8char = strdup(utf8char); + new->utf8len = len; + utf8map_head[hashindex] = new; + } + } + + if (feof(fin)) + errx(EX_DATAERR, "Didn't find final CHARMAP magic cookie.\n"); + fclose(fin); + + return NULL; +} + +void +usage(void) +{ + + printf("Usage: unicode2utf8 --cldr=dir --input=file --output=file\n"); + exit(EX_USAGE); +} + From owner-svn-src-user@FreeBSD.ORG Sun Nov 15 20:42:18 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D5C21065694; Sun, 15 Nov 2009 20:42:18 +0000 (UTC) (envelope-from edwin@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 4CAD58FC12; Sun, 15 Nov 2009 20:42:18 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAFKgIDa047655; Sun, 15 Nov 2009 20:42:18 GMT (envelope-from edwin@svn.freebsd.org) Received: (from edwin@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAFKgIWr047653; Sun, 15 Nov 2009 20:42:18 GMT (envelope-from edwin@svn.freebsd.org) Message-Id: <200911152042.nAFKgIWr047653@svn.freebsd.org> From: Edwin Groothuis Date: Sun, 15 Nov 2009 20:42:18 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199297 - user/edwin/locale/usr.bin/unicodename2utf8 X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Nov 2009 20:42:18 -0000 Author: edwin Date: Sun Nov 15 20:42:18 2009 New Revision: 199297 URL: http://svn.freebsd.org/changeset/base/199297 Log: Clean up Makefile after renames of unicode2utf8 to unicodename2utf8. Modified: user/edwin/locale/usr.bin/unicodename2utf8/Makefile Modified: user/edwin/locale/usr.bin/unicodename2utf8/Makefile ============================================================================== --- user/edwin/locale/usr.bin/unicodename2utf8/Makefile Sun Nov 15 20:36:11 2009 (r199296) +++ user/edwin/locale/usr.bin/unicodename2utf8/Makefile Sun Nov 15 20:42:18 2009 (r199297) @@ -1,13 +1,13 @@ # $NetBSD: Makefile,v 1.6 2009/04/20 16:05:30 drochner Exp $ -PROG= unicode2utf8 -SRCS= unicode2utf8.c +PROG= unicodename2utf8 +SRCS= unicodename2utf8.c NO_MAN= yes WARNS?= 6 test: - ./unicode2utf8 \ + ./unicodename2utf8 \ --cldr=/home/edwin/unicode/cldr/1.7.1/ \ --input=nl_NL.unicode \ --output=nl_NL.UTF-8.src From owner-svn-src-user@FreeBSD.ORG Sun Nov 15 22:09:49 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 799D6106566B; Sun, 15 Nov 2009 22:09:49 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 6863D8FC0A; Sun, 15 Nov 2009 22:09:49 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAFM9nFo049324; Sun, 15 Nov 2009 22:09:49 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAFM9nJf049322; Sun, 15 Nov 2009 22:09:49 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911152209.nAFM9nJf049322@svn.freebsd.org> From: Kip Macy Date: Sun, 15 Nov 2009 22:09:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199298 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Nov 2009 22:09:49 -0000 Author: kmacy Date: Sun Nov 15 22:09:48 2009 New Revision: 199298 URL: http://svn.freebsd.org/changeset/base/199298 Log: add variable to disable page caching of buffers Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sun Nov 15 20:42:18 2009 (r199297) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sun Nov 15 22:09:48 2009 (r199298) @@ -187,6 +187,8 @@ SYSCTL_QUAD(_vfs_zfs, OID_AUTO, arc_min, SYSCTL_INT(_vfs_zfs, OID_AUTO, mdcomp_disable, CTLFLAG_RDTUN, &zfs_mdcomp_disable, 0, "Disable metadata compression"); +static int page_cache_disable = 1; + #ifdef ZIO_USE_UMA extern kmem_cache_t *zio_buf_cache[]; extern kmem_cache_t *zio_data_buf_cache[]; @@ -1325,7 +1327,8 @@ arc_getblk(arc_buf_t *buf) if ((size < PAGE_SIZE)) { data = zio_buf_alloc(size); } else if ((buf->b_hdr->b_flags & ARC_BUF_CLONING) || - BUF_EMPTY(buf->b_hdr)) { + BUF_EMPTY(buf->b_hdr) || + page_cache_disable) { newbp = geteblk(size, flags); data = newbp->b_data; } else { @@ -1363,7 +1366,8 @@ arc_getblk(arc_buf_t *buf) if ((buf->b_hdr->b_flags & ARC_BUF_CLONING) && (size >= PAGE_SIZE) && - (!BUF_EMPTY(buf->b_hdr))) + (!BUF_EMPTY(buf->b_hdr)) && + !page_cache_disable) arc_binval(buf, blkno, vp, size, newbp); buf->b_hdr->b_flags &= ~ARC_BUF_CLONING; @@ -3450,7 +3454,8 @@ arc_write_done(zio_t *zio) } else if ((hdr->b_buf == buf) && (bp != NULL) && (bp->b_bufobj == NULL) && - (bp->b_bcount >= PAGE_SIZE)) { + (bp->b_bcount >= PAGE_SIZE) && + !page_cache_disable) { arc_binval(buf, blkno, vp, bp->b_bcount, bp); } From owner-svn-src-user@FreeBSD.ORG Sun Nov 15 23:49:28 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B6B7C1065693; Sun, 15 Nov 2009 23:49:28 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id A55828FC15; Sun, 15 Nov 2009 23:49:28 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAFNnSuh051474; Sun, 15 Nov 2009 23:49:28 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAFNnS6n051472; Sun, 15 Nov 2009 23:49:28 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911152349.nAFNnS6n051472@svn.freebsd.org> From: Kip Macy Date: Sun, 15 Nov 2009 23:49:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199301 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Nov 2009 23:49:28 -0000 Author: kmacy Date: Sun Nov 15 23:49:28 2009 New Revision: 199301 URL: http://svn.freebsd.org/changeset/base/199301 Log: consolidate in-core buffer invalidation in to one function Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sun Nov 15 23:48:29 2009 (r199300) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sun Nov 15 23:49:28 2009 (r199301) @@ -494,7 +494,7 @@ static void arc_access(arc_buf_hdr_t *bu static int arc_evict_needed(arc_buf_contents_t type); static void arc_evict_ghost(arc_state_t *state, spa_t *spa, int64_t bytes); static void arc_binval(arc_buf_t *buf, off_t blkno, struct vnode *vp, - size_t size, struct buf *bp); + size_t size, int flags, struct buf *bp); #define GHOST_STATE(state) \ ((state) == arc_mru_ghost || (state) == arc_mfu_ghost || \ @@ -1298,6 +1298,67 @@ arc_buf_add_ref(arc_buf_t *buf, void* ta data, metadata, hits); } +static struct buf * +arc_gbincore_replace(struct vnode *vp, off_t blkno, size_t size, int flags, + struct buf *newbp) +{ + struct buf *bp; + struct bufobj *bo; + int associated = 0; + + /* + * We need to be careful to handle the case where the buffer + * is already held in the ARC for a previous birth transaction + */ + bo = &vp->v_bufobj; + BO_LOCK(bo); + bp = gbincore(bo, blkno); + if (bp != NULL) { + if (bp == newbp) { + BO_UNLOCK(bo); + return (bp); + } + if (BUF_ISLOCKED(bp)) { + BO_UNLOCK(bo); + brelvp(bp); + } else { + BUF_LOCK(bp, LK_EXCLUSIVE | LK_INTERLOCK, BO_MTX(bo)); + if (newbp != NULL) { + bp->b_flags |= B_INVAL; + bp->b_flags &= ~B_CACHE; + bremfree(bp); + brelse(bp); + } else { + /* + * We don't have a new buffer for which we're + * replacing the mapping, use this buffer + */ + if (bp->b_flags & B_INVAL) + bp->b_flags &= ~B_CACHE; + else if ((bp->b_flags & (B_VMIO | B_INVAL)) == 0) + bp->b_flags |= B_CACHE; + bremfree(bp); + if (bp->b_bcount != size) + allocbuf_flags(bp, size, flags); + associated = 1; + } + } + } + + if (!associated) { + if (newbp != NULL) { + if (bp != NULL) + BO_LOCK(bo); + bgetvp(vp, newbp); + BO_UNLOCK(bo); + bp = newbp; + } else + bp = getblk(vp, blkno, size, 0, 0, flags); + } + + return (bp); +} + static void arc_getblk(arc_buf_t *buf) { @@ -1332,33 +1393,7 @@ arc_getblk(arc_buf_t *buf) newbp = geteblk(size, flags); data = newbp->b_data; } else { - /* - * We need to be careful to handle the case where the buffer - * is already held in the ARC for a previous birth transaction - */ - BO_LOCK(bo); - bp = gbincore(bo, blkno); - if (bp != NULL) { - if (BUF_ISLOCKED(bp)) { - BO_UNLOCK(bo); - brelvp(bp); - } else { - BUF_LOCK(bp, LK_EXCLUSIVE | LK_INTERLOCK, BO_MTX(bo)); - if (bp->b_flags & B_INVAL) - bp->b_flags &= ~B_CACHE; - else if ((bp->b_flags & (B_VMIO | B_INVAL)) == 0) - bp->b_flags |= B_CACHE; - bremfree(bp); - if (bp->b_bcount != size) - allocbuf_flags(bp, size, flags); - newbp = bp; - } - - } else - BO_UNLOCK(bo); - - if (newbp == NULL) - newbp = getblk(vp, blkno, size, 0, 0, flags); + newbp = arc_gbincore_replace(vp, blkno, size, flags, NULL); newbp->b_offset = buf->b_hdr->b_birth; data = newbp->b_data; @@ -1368,7 +1403,7 @@ arc_getblk(arc_buf_t *buf) (size >= PAGE_SIZE) && (!BUF_EMPTY(buf->b_hdr)) && !page_cache_disable) - arc_binval(buf, blkno, vp, size, newbp); + arc_binval(buf, blkno, vp, size, flags, newbp); buf->b_hdr->b_flags &= ~ARC_BUF_CLONING; #ifdef LOGALL @@ -1409,7 +1444,8 @@ arc_brelse(arc_buf_t *buf, void *data, s } static void -arc_binval(arc_buf_t *buf, off_t blkno, struct vnode *vp, size_t size, struct buf *newbp) +arc_binval(arc_buf_t *buf, off_t blkno, struct vnode *vp, size_t size, + int flags, struct buf *newbp) { arc_buf_t *tbuf; int released = 0; @@ -1439,25 +1475,8 @@ arc_binval(arc_buf_t *buf, off_t blkno, newbp->b_flags &= ~B_INVAL; newbp->b_flags |= B_CACHE; - BO_LOCK(bo); - if (!released) { - bp = gbincore(bo, blkno); - if (bp != NULL) { - if (BUF_ISLOCKED(bp)) { - BO_UNLOCK(bo); - brelvp(bp); - } else { - BUF_LOCK(bp, LK_EXCLUSIVE | LK_INTERLOCK, BO_MTX(bo)); - bp->b_flags |= B_INVAL; - bp->b_flags &= ~B_CACHE; - bremfree(bp); - brelse(bp); - } - BO_LOCK(bo); - } - } - bgetvp(vp, newbp); - BO_UNLOCK(bo); + if (!released) + arc_gbincore_replace(vp, blkno, size, flags, newbp); } /* @@ -3456,7 +3475,7 @@ arc_write_done(zio_t *zio) (bp->b_bufobj == NULL) && (bp->b_bcount >= PAGE_SIZE) && !page_cache_disable) { - arc_binval(buf, blkno, vp, bp->b_bcount, bp); + arc_binval(buf, blkno, vp, bp->b_bcount, 0, bp); } hdr->b_flags &= ~ARC_IO_IN_PROGRESS; From owner-svn-src-user@FreeBSD.ORG Sun Nov 15 23:53:10 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4208A106566B; Sun, 15 Nov 2009 23:53:10 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 30FCC8FC0C; Sun, 15 Nov 2009 23:53:10 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAFNrA4x051629; Sun, 15 Nov 2009 23:53:10 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAFNrA8U051627; Sun, 15 Nov 2009 23:53:10 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911152353.nAFNrA8U051627@svn.freebsd.org> From: Kip Macy Date: Sun, 15 Nov 2009 23:53:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199303 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Nov 2009 23:53:10 -0000 Author: kmacy Date: Sun Nov 15 23:53:09 2009 New Revision: 199303 URL: http://svn.freebsd.org/changeset/base/199303 Log: make tunable for disabling freeing of buffers to the page cache Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sun Nov 15 23:51:59 2009 (r199302) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sun Nov 15 23:53:09 2009 (r199303) @@ -187,7 +187,9 @@ SYSCTL_QUAD(_vfs_zfs, OID_AUTO, arc_min, SYSCTL_INT(_vfs_zfs, OID_AUTO, mdcomp_disable, CTLFLAG_RDTUN, &zfs_mdcomp_disable, 0, "Disable metadata compression"); -static int page_cache_disable = 1; +static int zfs_page_cache_disable = 0; +SYSCTL_INT(_vfs_zfs, OID_AUTO, page_cache_disable, CTLFLAG_RDTUN, + &zfs_page_cache_disable, 0, "Disable backing ARC with page cache "); #ifdef ZIO_USE_UMA extern kmem_cache_t *zio_buf_cache[]; @@ -1389,7 +1391,7 @@ arc_getblk(arc_buf_t *buf) data = zio_buf_alloc(size); } else if ((buf->b_hdr->b_flags & ARC_BUF_CLONING) || BUF_EMPTY(buf->b_hdr) || - page_cache_disable) { + zfs_page_cache_disable) { newbp = geteblk(size, flags); data = newbp->b_data; } else { @@ -1402,7 +1404,7 @@ arc_getblk(arc_buf_t *buf) if ((buf->b_hdr->b_flags & ARC_BUF_CLONING) && (size >= PAGE_SIZE) && (!BUF_EMPTY(buf->b_hdr)) && - !page_cache_disable) + !zfs_page_cache_disable) arc_binval(buf, blkno, vp, size, flags, newbp); buf->b_hdr->b_flags &= ~ARC_BUF_CLONING; @@ -3474,7 +3476,7 @@ arc_write_done(zio_t *zio) (bp != NULL) && (bp->b_bufobj == NULL) && (bp->b_bcount >= PAGE_SIZE) && - !page_cache_disable) { + !zfs_page_cache_disable) { arc_binval(buf, blkno, vp, bp->b_bcount, 0, bp); } From owner-svn-src-user@FreeBSD.ORG Mon Nov 16 00:17:12 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 38D0B1065725; Mon, 16 Nov 2009 00:17:12 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 167FD8FC1A; Mon, 16 Nov 2009 00:17:12 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAG0HBHv052207; Mon, 16 Nov 2009 00:17:11 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAG0HBmt052205; Mon, 16 Nov 2009 00:17:11 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911160017.nAG0HBmt052205@svn.freebsd.org> From: Kip Macy Date: Mon, 16 Nov 2009 00:17:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199304 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2009 00:17:12 -0000 Author: kmacy Date: Mon Nov 16 00:17:11 2009 New Revision: 199304 URL: http://svn.freebsd.org/changeset/base/199304 Log: drop bufobj lock before calling getblk Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sun Nov 15 23:53:09 2009 (r199303) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Mon Nov 16 00:17:11 2009 (r199304) @@ -1354,8 +1354,10 @@ arc_gbincore_replace(struct vnode *vp, o bgetvp(vp, newbp); BO_UNLOCK(bo); bp = newbp; - } else + } else { + BO_UNLOCK(bo); bp = getblk(vp, blkno, size, 0, 0, flags); + } } return (bp); From owner-svn-src-user@FreeBSD.ORG Mon Nov 16 02:46:21 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D727D106566B; Mon, 16 Nov 2009 02:46:21 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id C5E838FC0C; Mon, 16 Nov 2009 02:46:21 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAG2kLk8055159; Mon, 16 Nov 2009 02:46:21 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAG2kLTb055157; Mon, 16 Nov 2009 02:46:21 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911160246.nAG2kLTb055157@svn.freebsd.org> From: Kip Macy Date: Mon, 16 Nov 2009 02:46:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199305 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2009 02:46:21 -0000 Author: kmacy Date: Mon Nov 16 02:46:21 2009 New Revision: 199305 URL: http://svn.freebsd.org/changeset/base/199305 Log: fix logic for end of arc_gbincore_replace Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Mon Nov 16 00:17:11 2009 (r199304) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Mon Nov 16 02:46:21 2009 (r199305) @@ -1306,7 +1306,6 @@ arc_gbincore_replace(struct vnode *vp, o { struct buf *bp; struct bufobj *bo; - int associated = 0; /* * We need to be careful to handle the case where the buffer @@ -1342,24 +1341,30 @@ arc_gbincore_replace(struct vnode *vp, o bremfree(bp); if (bp->b_bcount != size) allocbuf_flags(bp, size, flags); - associated = 1; + goto done; } } } - if (!associated) { - if (newbp != NULL) { - if (bp != NULL) - BO_LOCK(bo); - bgetvp(vp, newbp); - BO_UNLOCK(bo); - bp = newbp; - } else { + /* + * !bp brelvp(bp) brelse(bp) associated + * !newbp unlock()/getblk getblk XXX no-op + * + * newbp bgetvp()/ul l/bgetvp()/ul l/bgetvp()/ul XXX + */ + if (newbp != NULL) { + if (bp != NULL) + BO_LOCK(bo); + bgetvp(vp, newbp); + BO_UNLOCK(bo); + bp = newbp; + } else { + if (bp == NULL) BO_UNLOCK(bo); - bp = getblk(vp, blkno, size, 0, 0, flags); - } + bp = getblk(vp, blkno, size, 0, 0, flags); } +done: return (bp); } From owner-svn-src-user@FreeBSD.ORG Mon Nov 16 03:36:55 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A1FC106566C; Mon, 16 Nov 2009 03:36:55 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 089EB8FC19; Mon, 16 Nov 2009 03:36:55 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAG3asA3056383; Mon, 16 Nov 2009 03:36:54 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAG3asbh056381; Mon, 16 Nov 2009 03:36:54 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911160336.nAG3asbh056381@svn.freebsd.org> From: Kip Macy Date: Mon, 16 Nov 2009 03:36:54 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199306 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2009 03:36:55 -0000 Author: kmacy Date: Mon Nov 16 03:36:54 2009 New Revision: 199306 URL: http://svn.freebsd.org/changeset/base/199306 Log: handle case where don't call allocbuf if b_kvasize is less than size Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Mon Nov 16 02:46:21 2009 (r199305) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Mon Nov 16 03:36:54 2009 (r199306) @@ -1339,9 +1339,13 @@ arc_gbincore_replace(struct vnode *vp, o else if ((bp->b_flags & (B_VMIO | B_INVAL)) == 0) bp->b_flags |= B_CACHE; bremfree(bp); - if (bp->b_bcount != size) + if ((bp->b_bcount != size) && (buf->b_kvasize >= size)) { allocbuf_flags(bp, size, flags); - goto done; + goto done; + } else { + bp->b_flags |= B_INVAL; + brelse(bp); + } } } } From owner-svn-src-user@FreeBSD.ORG Mon Nov 16 05:12:24 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07149106568F; Mon, 16 Nov 2009 05:12:24 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id EA9578FC26; Mon, 16 Nov 2009 05:12:23 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAG5CNh9058604; Mon, 16 Nov 2009 05:12:23 GMT (envelope-from dougb@svn.freebsd.org) Received: (from dougb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAG5CNN0058602; Mon, 16 Nov 2009 05:12:23 GMT (envelope-from dougb@svn.freebsd.org) Message-Id: <200911160512.nAG5CNN0058602@svn.freebsd.org> From: Doug Barton Date: Mon, 16 Nov 2009 05:12:23 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199311 - user/dougb/portmaster X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2009 05:12:24 -0000 Author: dougb Date: Mon Nov 16 05:12:23 2009 New Revision: 199311 URL: http://svn.freebsd.org/changeset/base/199311 Log: Add a closing bracket in the usage() example for the new --packages options Modified: user/dougb/portmaster/portmaster Modified: user/dougb/portmaster/portmaster ============================================================================== --- user/dougb/portmaster/portmaster Mon Nov 16 04:55:16 2009 (r199310) +++ user/dougb/portmaster/portmaster Mon Nov 16 05:12:23 2009 (r199311) @@ -224,7 +224,7 @@ usage () { echo '' echo 'Usage:' echo "Common flags: [--force-config] [-CGHKgntvw B|b f|i D|d]" - echo " [[--packages|--packages-only] [-P|-PP] | [--packages-build]" + echo " [[--packages|--packages-only] [-P|-PP] | [--packages-build]]" echo " [--packages-if-newer] [-m ]" echo " [-x ]" echo "${0##*/} [Common flags] " From owner-svn-src-user@FreeBSD.ORG Mon Nov 16 06:10:24 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 57B9D1065676; Mon, 16 Nov 2009 06:10:24 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 474978FC08; Mon, 16 Nov 2009 06:10:24 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAG6AOgN059808; Mon, 16 Nov 2009 06:10:24 GMT (envelope-from dougb@svn.freebsd.org) Received: (from dougb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAG6AOv1059806; Mon, 16 Nov 2009 06:10:24 GMT (envelope-from dougb@svn.freebsd.org) Message-Id: <200911160610.nAG6AOv1059806@svn.freebsd.org> From: Doug Barton Date: Mon, 16 Nov 2009 06:10:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199312 - user/dougb/portmaster X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2009 06:10:24 -0000 Author: dougb Date: Mon Nov 16 06:10:24 2009 New Revision: 199312 URL: http://svn.freebsd.org/changeset/base/199312 Log: Fix 2 places where we generate a list of ports by using cut to process directory names in /var/db/pkg. The previous code assumed that the value of PKG_DBDIR would always be 3 directories deep, which is not necessarily a valid assumption. In one spot this allows us to avoid an external shell call altogether, in the other an internal function is a necessary evil to avoid other even more complex alternatives. Modified: user/dougb/portmaster/portmaster Modified: user/dougb/portmaster/portmaster ============================================================================== --- user/dougb/portmaster/portmaster Mon Nov 16 05:12:23 2009 (r199311) +++ user/dougb/portmaster/portmaster Mon Nov 16 06:10:24 2009 (r199312) @@ -449,6 +449,16 @@ check_regular_file () { [ ! -L "$1" -a -f "$1" ] || fail "ERROR: $1 is not a regular file!" } +strip_to_iport () { + local in + + while read in; do + in="${in%/+CONTENTS}" + in="${in##*/}" + echo $in + done +} + check_dependency_files () { # Global: grep_deps local origin iport ro_opd @@ -465,7 +475,7 @@ check_dependency_files () { # Always rely on the grep'ed dependencies instead of +REQUIRED_BY grep_deps=`pm_mktemp grep-deps-${iport}` egrep -l "DEPORIGIN:($origin|$ro_opd)$" $pdb/*/+CONTENTS | - cut -f 5 -d '/' | sort -u > $grep_deps + strip_to_iport | sort -u > $grep_deps if [ ! -s "$grep_deps" ]; then if [ -s "$pdb/$iport/+REQUIRED_BY" ]; then @@ -1767,8 +1777,9 @@ create_master_rb_list () { # Global: MASTER_RB_LIST local req_by - for req_by in `grep -l DEPORIGIN:$portdir$ $pdb/*/+CONTENTS | \ - cut -f 5 -d '/'`; do + for req_by in `grep -l DEPORIGIN:$portdir$ $pdb/*/+CONTENTS`; do + req_by="${req_by%/+CONTENTS}" + req_by="${req_by##*/}" MASTER_RB_LIST="${MASTER_RB_LIST}${req_by} " done From owner-svn-src-user@FreeBSD.ORG Mon Nov 16 09:37:31 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88FE11065693; Mon, 16 Nov 2009 09:37:31 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 0EDE08FC14; Mon, 16 Nov 2009 09:37:31 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAG9bUHp064149; Mon, 16 Nov 2009 09:37:30 GMT (envelope-from dougb@svn.freebsd.org) Received: (from dougb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAG9bUFH064147; Mon, 16 Nov 2009 09:37:30 GMT (envelope-from dougb@svn.freebsd.org) Message-Id: <200911160937.nAG9bUFH064147@svn.freebsd.org> From: Doug Barton Date: Mon, 16 Nov 2009 09:37:30 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199318 - user/dougb/portmaster X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2009 09:37:31 -0000 Author: dougb Date: Mon Nov 16 09:37:30 2009 New Revision: 199318 URL: http://svn.freebsd.org/changeset/base/199318 Log: Rework the --packages-build option: 1. Keep a global list of build-only deps and those that have been listed as run-deps for any port. Strip a dep from the build_only list if it shows up as a run-dep. 2. This allows us to remove the restriction of not using -t or multiport mode with --packages-build, but forces us to require running the config pass (i.e., no -G) 3. Use a much more clever way to insure that we use packages for anything on the global build_only list, but build everything else. Special funding committment for this feature and package support generally by: Modirum MDpay Modified: user/dougb/portmaster/portmaster Modified: user/dougb/portmaster/portmaster ============================================================================== --- user/dougb/portmaster/portmaster Mon Nov 16 09:28:22 2009 (r199317) +++ user/dougb/portmaster/portmaster Mon Nov 16 09:37:30 2009 (r199318) @@ -214,6 +214,10 @@ safe_exit () { fi [ -n "$URB_YES" ] && echo "URB_DONE_LIST='$URB_DONE_LIST'" >> $IPC_SAVE + if [ "$PM_PACKAGES_BUILD" = pmp_doing_build_deps ]; then + echo "build_only_dl_g='$build_only_dl_g'" >> $IPC_SAVE + echo "run_dl_g='$run_dl_g'" >> $IPC_SAVE + fi fi exit ${1:-0} @@ -894,9 +898,7 @@ while getopts 'BCDFGHKLRabde:fghilm:nop: p) portdir="${OPTARG#$pd/}" ; portdir=${portdir%/} ;; r) UPDATE_REQ_BYS=ropt ; portdir=`origin_from_pdb $OPTARG` ;; s) CLEAN_STALE=sopt ;; - t) [ "$PM_PACKAGES" = build ] && - fail 'The --packages-build and -t options are mutually exclusive' - RECURSE_THOROUGH=topt; ARGS="-t $ARGS" ;; + t) RECURSE_THOROUGH=topt; ARGS="-t $ARGS" ;; u) echo "===>>> The -u option has been deprecated" ; echo '' ;; v) PM_VERBOSE=vopt; ARGS="-v $ARGS" ;; w) SAVE_SHARED=wopt; ARGS="-w $ARGS" ;; @@ -920,6 +922,8 @@ shift $(( $OPTIND - 1 )) fail "The -d and -D options are mutually exclusive" [ -n "$FETCH_ONLY" -a -n "$NO_RECURSIVE_CONFIG" ] && fail "The -F and -G options are mutually exclusive" +[ -n "$NO_RECURSIVE_CONFIG" -a -n "$PM_PACKAGES_BUILD" ] && + fail 'The --packages-build and -G options are mutually exclusive' #=============== Begin functions for getopts features and main =============== @@ -1620,6 +1624,7 @@ update_port () { dependency_check () { # Global: doing_dep_check + # Global: run_dl_g build_only_dl_g local d_port_list @@ -1642,24 +1647,59 @@ dependency_check () { fi fi -# XXX +# XXXX + if [ "$PM_PACKAGES_BUILD" = pmp_doing_build_deps ]; then - local rundeps dep build_only build_and_run build_only_dep - rundeps=`pm_make run-depends-list | sort -u` - for dep in $d_port_list; do - case "$rundeps" in - *${dep}*) build_and_run="$build_and_run $dep" ;; - *) build_only="$build_only $dep" ;; +#echo '' ; echo "Debug> d_port_list: $d_port_list" + + local rundeps dep run_dl build_only_dl temp_bodlg + + if [ -z "$RECURSE_THOROUGH" ]; then + rundeps=`pm_make run-depends-list | sort -u` + rundeps=" $rundeps " + + for dep in $d_port_list; do + case "$rundeps" in + *${dep}*) run_dl="$run_dl $dep" ;; + *) build_only_dl="$build_only_dl $dep" ;; + esac + done + +#echo '' ; echo "Debug> build_only_dl: $build_only_dl" +#echo '' ; echo "Debug> run_dl: $run_dl" ; echo '' + + d_port_list="$build_only_dl $run_dl" + else + for dep in `pm_make run-depends-list | sort -u`; do + run_dl="$run_dl $dep" + done + build_only_dl=`pm_make build-depends-list | sort -u` + +#echo '' ; echo "Debug> build_only_dl: $build_only_dl" +#echo '' ; echo "Debug> run_dl: $run_dl" ; echo '' + fi + + run_dl_g="$run_dl_g $run_dl " + + for dep in $build_only_dl; do + case "$build_only_dl_g" in + *" ${dep} "*) ;; + *) build_only_dl_g="$build_only_dl_g $dep " ;; esac done -#echo '' ; echo "Debug> d_port_list: $d_port_list" -#echo "Debug> build_only: $build_only" -#echo "Debug> build_and_run: $build_and_run" -#echo '' + for dep in $build_only_dl_g; do + case "$run_dl_g" in + *" ${dep} "*) ;; + *) temp_bodlg="$temp_bodlg $dep" ;; + esac + done + + build_only_dl_g=" $temp_bodlg " - d_port_list="$build_only $build_and_run" +#echo '' ; echo "Debug> build_only_dl_g: $build_only_dl_g" +#echo '' ; echo "Debug> run_dl_g: $run_dl_g" ; echo '' fi local d_port origin iport conflicts glob confl_p udf @@ -1741,14 +1781,6 @@ dependency_check () { esac fi -# XXX - case "$build_only" in - *${d_port}*) -#echo '' ; echo "Debug> $origin is build-only" ; echo '' - build_only_dep=build_only_dep - PM_PACKAGES=only ; export PM_PACKAGES ;; - esac - if [ -n "$iport" ]; then # No check_interactive here because we want to tell # the user whether or not there is a new version first @@ -1757,8 +1789,6 @@ dependency_check () { check_interactive $origin || continue update_port $origin fi - - [ -n "$build_only_dep" ] && unset PM_PACKAGES done [ -n "$CONFIG_ONLY" ] && unset doing_dep_check @@ -1833,6 +1863,12 @@ multiport () { echo -e $portlist if [ -n "$CONFIG_ONLY" ]; then + if [ -n "$PM_PACKAGES_BUILD" ]; then + PM_PACKAGES_BUILD=pmp_doing_build_deps + run_dl_g=" " ; build_only_dl_g=" " + export PM_PACKAGES_BUILD run_dl_g build_only_dl_g + fi + for port in $worklist; do ($0 $ARGS $port) || fail "Update for $port failed" . $IPC_SAVE @@ -1842,8 +1878,12 @@ multiport () { echo '' echo "===>>> Starting build for multiple ports <<<===" echo '' + + [ -n "$PM_PACKAGES_BUILD" ] && unset run_dl_g fi +#echo '' ; echo "Debug> build_only_dl_g multiport: $build_only_dl_g" ; echo '' + export PM_BUILDING=pmbuildingmultiport for port in $worklist; do case "$PM_MULTI_BUILT" in *:${port}:*) continue ;; esac @@ -2232,6 +2272,17 @@ dofetch () { fi if [ -n "$CONFIG_ONLY" ]; then + +# XXXX + if [ "$$" -eq "$PM_PARENT_PID" ]; then + # Keep in sync in multiport() + if [ -n "$PM_PACKAGES_BUILD" ]; then + PM_PACKAGES_BUILD=pmp_doing_build_deps + run_dl_g=" " ; build_only_dl_g=" " + export PM_PACKAGES_BUILD run_dl_g build_only_dl_g + fi + fi + config_type=config-conditional [ -n "$FORCE_CONFIG" ] && config_type=config [ -n "$PM_SU_VERBOSE" ] && echo "===>>> Running 'make $config_type'" @@ -2286,6 +2337,10 @@ if [ -n "$CONFIG_ONLY" ]; then unset URB_YES MASTER_RB_LIST ; URB_DONE_LIST=':' fi +# XXXX + [ -n "$PM_PACKAGES_BUILD" ] && unset run_dl_g +#echo '' ; echo "Debug> final build_only_dl_g: $build_only_dl_g" ; echo '' + check_fetch_only unset CONFIG_SEEN_LIST CONFIG_ONLY echo "===>>> Starting build for $portdir <<<===" @@ -2296,14 +2351,20 @@ fi cd $pd/$portdir +if [ -n "$PM_PACKAGES_BUILD" ]; then + case "$build_only_dl_g" in + *" $pd/$portdir "*) + +#echo '' ; echo "Debug> DOING BUILD ONLY DEP" ; echo '' + + PM_PACKAGES_BUILD=doing_build_only_dep ;; + *) PM_PACKAGES_BUILD=pmp_build ;; + esac +fi + if [ -z "$NO_DEP_UPDATES" ]; then if [ -z "$RECURSE_THOROUGH" ]; then if [ ! "$PM_PACKAGES" = only ]; then -# XXX - if [ "$$" -eq "$PM_PARENT_PID" ]; then - [ -n "$PM_PACKAGES_BUILD" ] && - PM_PACKAGES_BUILD=pmp_doing_build_deps - fi echo "===>>> Starting check for build dependencies" dependency_check build-depends-list fi @@ -2311,10 +2372,6 @@ if [ -z "$NO_DEP_UPDATES" ]; then echo "===>>> Starting check for all dependencies" dependency_check all-depends-list fi - if [ "$$" -eq "$PM_PARENT_PID" ]; then - [ -n "$PM_PACKAGES_BUILD" ] && - unset PM_PACKAGES_BUILD PM_PACKAGES - fi cd $pd/$portdir elif [ -z "$NO_RECURSIVE_CONFIG" -a "$$" -eq "$PM_PARENT_PID" ]; then echo "===>>> The 'make config' check found no dependencies to update" @@ -2330,7 +2387,7 @@ new_port=`pm_make -V PKGNAME` # XXX Build or package? -if [ -n "$PM_PACKAGES" ]; then +if [ -n "$PM_PACKAGES" -o "$PM_PACKAGES_BUILD" = doing_build_only_dep ]; then if [ -z "$PACKAGESITE" ]; then release=`uname -r` #release=7.0-RELEASE-p12 @@ -2359,9 +2416,7 @@ if [ -n "$PM_PACKAGES" ]; then sitepath="${sitepath%/}/${portdir%/*}/" -#echo '' -#echo "Debug> sitepath: $sitepath" -#echo '' +#echo '' ; echo "Debug> sitepath: $sitepath" ; echo '' [ -n "$PM_VERBOSE" ] && echo "===>>> Checking package repository" @@ -2386,9 +2441,7 @@ if [ -n "$PM_PACKAGES" ]; then *%2[cC]*) latest_pv=`echo $latest_pv | sed s#%2[cC]#,#` ;; esac -#echo '' -#echo "Debug> new_port: $new_port latest_pv: $latest_pv" -#echo '' +#echo '' ; echo "Debug> new_port: $new_port latest_pv: $latest_pv" ; echo '' notnewer () { echo '' From owner-svn-src-user@FreeBSD.ORG Tue Nov 17 00:49:42 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A9854106566C; Tue, 17 Nov 2009 00:49:42 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 990E48FC20; Tue, 17 Nov 2009 00:49:42 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAH0ngIk086493; Tue, 17 Nov 2009 00:49:42 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAH0ngVZ086492; Tue, 17 Nov 2009 00:49:42 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911170049.nAH0ngVZ086492@svn.freebsd.org> From: Kip Macy Date: Tue, 17 Nov 2009 00:49:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199340 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2009 00:49:42 -0000 Author: kmacy Date: Tue Nov 17 00:49:41 2009 New Revision: 199340 URL: http://svn.freebsd.org/changeset/base/199340 Log: - Only mark a buffer as newly valid on read and write completion - add comments to arc_gbincore_replace - don't treat cloned buffers as the newly valid mapping - this eliminates the need for arc_binval Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Mon Nov 16 22:52:52 2009 (r199339) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Tue Nov 17 00:49:41 2009 (r199340) @@ -1319,20 +1319,30 @@ arc_gbincore_replace(struct vnode *vp, o BO_UNLOCK(bo); return (bp); } + if (BUF_ISLOCKED(bp)) { + /* + * buffer is currently referenced in the ARC with older birth txg + * + */ + bp->b_flags |= B_INVAL; + bp->b_flags &= ~B_CACHE; BO_UNLOCK(bo); brelvp(bp); } else { BUF_LOCK(bp, LK_EXCLUSIVE | LK_INTERLOCK, BO_MTX(bo)); if (newbp != NULL) { + /* + * mapping exists but we wish to replace it with + * newbp + */ bp->b_flags |= B_INVAL; bp->b_flags &= ~B_CACHE; bremfree(bp); brelse(bp); } else { /* - * We don't have a new buffer for which we're - * replacing the mapping, use this buffer + * buffer is usable for this mapping */ if (bp->b_flags & B_INVAL) bp->b_flags &= ~B_CACHE; @@ -1404,21 +1414,15 @@ arc_getblk(arc_buf_t *buf) BUF_EMPTY(buf->b_hdr) || zfs_page_cache_disable) { newbp = geteblk(size, flags); - data = newbp->b_data; + data = newbp->b_data; + buf->b_hdr->b_flags &= ~ARC_BUF_CLONING; } else { newbp = arc_gbincore_replace(vp, blkno, size, flags, NULL); newbp->b_offset = buf->b_hdr->b_birth; - data = newbp->b_data; + data = newbp->b_data; } - if ((buf->b_hdr->b_flags & ARC_BUF_CLONING) && - (size >= PAGE_SIZE) && - (!BUF_EMPTY(buf->b_hdr)) && - !zfs_page_cache_disable) - arc_binval(buf, blkno, vp, size, flags, newbp); - buf->b_hdr->b_flags &= ~ARC_BUF_CLONING; - #ifdef LOGALL /* * not useful for tracking down collisions @@ -1456,42 +1460,6 @@ arc_brelse(arc_buf_t *buf, void *data, s brelse(bp); } -static void -arc_binval(arc_buf_t *buf, off_t blkno, struct vnode *vp, size_t size, - int flags, struct buf *newbp) -{ - arc_buf_t *tbuf; - int released = 0; - struct buf *bp = NULL; - uint64_t birth; - struct bufobj *bo; - - /* - * disassociate backing buffers from the vnode - * - */ - for (tbuf = buf; tbuf != NULL; tbuf = tbuf->b_next) { - if ((tbuf->b_bp != NULL) && (tbuf->b_bp->b_vp != NULL)) { - bp = tbuf->b_bp; - KASSERT((bp->b_xflags & (BX_VNCLEAN|BX_VNDIRTY)) == BX_VNCLEAN, ("brelvp() on buffer that is not in splay")); - - bp->b_flags |= B_INVAL; - bp->b_flags &= ~B_CACHE; - brelvp(bp); - released = 1; - } - } - bo = newbp->b_bufobj = &vp->v_bufobj; - newbp->b_lblkno = blkno; - newbp->b_blkno = blkno; - newbp->b_offset = buf->b_hdr->b_birth; - newbp->b_flags &= ~B_INVAL; - newbp->b_flags |= B_CACHE; - - if (!released) - arc_gbincore_replace(vp, blkno, size, flags, newbp); -} - /* * Free the arc data buffer. If it is an l2arc write in progress, * the buffer is placed on l2arc_free_on_write to be freed later. @@ -2739,11 +2707,6 @@ arc_read_done(zio_t *zio) arc_cksum_compute(buf, B_FALSE); - if (buf->b_bp != NULL) { - buf->b_bp->b_flags &= ~B_INVAL; - buf->b_bp->b_flags |= B_CACHE; - } - /* create copies of the data buffer for the callers */ abuf = buf; for (acb = callback_list; acb; acb = acb->acb_next) { @@ -2771,6 +2734,16 @@ arc_read_done(zio_t *zio) freeable = refcount_is_zero(&hdr->b_refcnt); } + if ((freeable == FALSE) && !zfs_page_cache_disable) { + off_t blkno = hdr->b_dva.dva_word[1] & ~(1UL<<63); + struct buf *bp = buf->b_bp; + + bp->b_flags |= B_CACHE; + bp->b_flags &= ~B_INVAL; + arc_gbincore_replace(spa_get_vnode(hdr->b_spa), + blkno, bp->b_bcount, 0, bp); + } + /* * Broadcast before we drop the hash_lock to avoid the possibility * that the hdr (and hence the cv) might be freed before we get to @@ -3483,12 +3456,13 @@ arc_write_done(zio_t *zio) arc_hdr_destroy(exists); exists = buf_hash_insert(hdr, &hash_lock); ASSERT3P(exists, ==, NULL); - } else if ((hdr->b_buf == buf) && - (bp != NULL) && + } else if ((bp != NULL) && (bp->b_bufobj == NULL) && (bp->b_bcount >= PAGE_SIZE) && !zfs_page_cache_disable) { - arc_binval(buf, blkno, vp, bp->b_bcount, 0, bp); + bp->b_flags |= B_CACHE; + bp->b_flags &= ~B_INVAL; + arc_gbincore_replace(vp, blkno, bp->b_bcount, 0, bp); } hdr->b_flags &= ~ARC_IO_IN_PROGRESS; From owner-svn-src-user@FreeBSD.ORG Tue Nov 17 04:59:44 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B6141065672; Tue, 17 Nov 2009 04:59:44 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 387D18FC12; Tue, 17 Nov 2009 04:59:44 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAH4xixo091873; Tue, 17 Nov 2009 04:59:44 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAH4xiTO091871; Tue, 17 Nov 2009 04:59:44 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911170459.nAH4xiTO091871@svn.freebsd.org> From: Kip Macy Date: Tue, 17 Nov 2009 04:59:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199343 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2009 04:59:44 -0000 Author: kmacy Date: Tue Nov 17 04:59:43 2009 New Revision: 199343 URL: http://svn.freebsd.org/changeset/base/199343 Log: only update the page cache when freeing the last buffer belonging to a buffer header Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Tue Nov 17 03:37:24 2009 (r199342) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Tue Nov 17 04:59:43 2009 (r199343) @@ -1300,86 +1300,45 @@ arc_buf_add_ref(arc_buf_t *buf, void* ta data, metadata, hits); } -static struct buf * -arc_gbincore_replace(struct vnode *vp, off_t blkno, size_t size, int flags, - struct buf *newbp) -{ - struct buf *bp; - struct bufobj *bo; +static void +arc_bgetvp(arc_buf_t *buf) +{ + off_t blkno = buf->b_hdr->b_dva.dva_word[1] & ~(1UL<<63); + struct buf *bp = buf->b_bp; + struct vnode *vp = spa_get_vnode(buf->b_hdr->b_spa); + struct bufobj *bo = &vp->v_bufobj; + arc_buf_hdr_t *hdr = buf->b_hdr; - /* - * We need to be careful to handle the case where the buffer - * is already held in the ARC for a previous birth transaction - */ - bo = &vp->v_bufobj; BO_LOCK(bo); bp = gbincore(bo, blkno); if (bp != NULL) { - if (bp == newbp) { - BO_UNLOCK(bo); - return (bp); - } - if (BUF_ISLOCKED(bp)) { - /* - * buffer is currently referenced in the ARC with older birth txg - * - */ + /* + * XXX we have a race with getblk here + */ + BUF_LOCK(bp, LK_EXCLUSIVE | LK_INTERLOCK, BO_MTX(bo)); + bremfree(bp); + /* + * buffer is usable for this mapping + */ + if (!(hdr->b_flags & ARC_IO_ERROR) && + (bp->b_offset <= hdr->b_birth) && + (bp->b_flags & (B_CACHE|B_INVAL)) == B_CACHE) { bp->b_flags |= B_INVAL; bp->b_flags &= ~B_CACHE; - BO_UNLOCK(bo); - brelvp(bp); - } else { - BUF_LOCK(bp, LK_EXCLUSIVE | LK_INTERLOCK, BO_MTX(bo)); - if (newbp != NULL) { - /* - * mapping exists but we wish to replace it with - * newbp - */ - bp->b_flags |= B_INVAL; - bp->b_flags &= ~B_CACHE; - bremfree(bp); - brelse(bp); - } else { - /* - * buffer is usable for this mapping - */ - if (bp->b_flags & B_INVAL) - bp->b_flags &= ~B_CACHE; - else if ((bp->b_flags & (B_VMIO | B_INVAL)) == 0) - bp->b_flags |= B_CACHE; - bremfree(bp); - if ((bp->b_bcount != size) && (buf->b_kvasize >= size)) { - allocbuf_flags(bp, size, flags); - goto done; - } else { - bp->b_flags |= B_INVAL; - brelse(bp); - } - } - } - } - /* - * !bp brelvp(bp) brelse(bp) associated - * !newbp unlock()/getblk getblk XXX no-op - * - * newbp bgetvp()/ul l/bgetvp()/ul l/bgetvp()/ul XXX - */ - if (newbp != NULL) { - if (bp != NULL) + brelse(bp); + buf->b_bp->b_offset = hdr->b_birth; + buf->b_bp->b_flags |= B_CACHE; + buf->b_bp->b_flags &= ~B_INVAL; + BO_LOCK(bo); - bgetvp(vp, newbp); - BO_UNLOCK(bo); - bp = newbp; - } else { - if (bp == NULL) + bgetvp(vp, buf->b_bp); BO_UNLOCK(bo); - bp = getblk(vp, blkno, size, 0, 0, flags); - } - -done: - return (bp); + } else + brelse(bp); + } else + BO_UNLOCK(bo); } static void @@ -1411,15 +1370,14 @@ arc_getblk(arc_buf_t *buf) if ((size < PAGE_SIZE)) { data = zio_buf_alloc(size); } else if ((buf->b_hdr->b_flags & ARC_BUF_CLONING) || - BUF_EMPTY(buf->b_hdr) || - zfs_page_cache_disable) { + BUF_EMPTY(buf->b_hdr)) { newbp = geteblk(size, flags); data = newbp->b_data; buf->b_hdr->b_flags &= ~ARC_BUF_CLONING; } else { - newbp = arc_gbincore_replace(vp, blkno, size, flags, NULL); - - newbp->b_offset = buf->b_hdr->b_birth; + newbp = getblk(vp, blkno, size, 0, 0, flags); + if (newbp->b_vp != NULL) + brelvp(newbp); data = newbp->b_data; } @@ -1441,20 +1399,16 @@ arc_getblk(arc_buf_t *buf) static void arc_brelse(arc_buf_t *buf, void *data, size_t size) { - struct buf *bp; + struct buf *bp = buf->b_bp; + arc_buf_hdr_t *hdr = buf->b_hdr; - bp = buf->b_bp; if (bp == NULL) { zio_buf_free(buf->b_data, size); return; } -#ifdef INVARIANTS - if (bp->b_vp) { - KASSERT((buf->b_bp->b_xflags & (BX_VNCLEAN|BX_VNDIRTY)) == BX_VNCLEAN, ("brelse() on buffer that is not clean")); - } -#endif - + if (hdr->b_datacnt == 1) + arc_bgetvp(buf); CTR4(KTR_SPARE2, "arc_brelse() bp=%p flags %X size %ld blkno=%ld", bp, bp->b_flags, size, bp->b_blkno); brelse(bp); @@ -2734,16 +2688,6 @@ arc_read_done(zio_t *zio) freeable = refcount_is_zero(&hdr->b_refcnt); } - if ((freeable == FALSE) && !zfs_page_cache_disable) { - off_t blkno = hdr->b_dva.dva_word[1] & ~(1UL<<63); - struct buf *bp = buf->b_bp; - - bp->b_flags |= B_CACHE; - bp->b_flags &= ~B_INVAL; - arc_gbincore_replace(spa_get_vnode(hdr->b_spa), - blkno, bp->b_bcount, 0, bp); - } - /* * Broadcast before we drop the hash_lock to avoid the possibility * that the hdr (and hence the cv) might be freed before we get to @@ -2985,7 +2929,6 @@ top: mutex_exit(hash_lock); goto top; } - buf->b_bp->b_offset = bp->blk_birth; } acb = kmem_zalloc(sizeof (arc_callback_t), KM_SLEEP); @@ -3456,14 +3399,7 @@ arc_write_done(zio_t *zio) arc_hdr_destroy(exists); exists = buf_hash_insert(hdr, &hash_lock); ASSERT3P(exists, ==, NULL); - } else if ((bp != NULL) && - (bp->b_bufobj == NULL) && - (bp->b_bcount >= PAGE_SIZE) && - !zfs_page_cache_disable) { - bp->b_flags |= B_CACHE; - bp->b_flags &= ~B_INVAL; - arc_gbincore_replace(vp, blkno, bp->b_bcount, 0, bp); - } + } hdr->b_flags &= ~ARC_IO_IN_PROGRESS; /* if it's not anon, we are doing a scrub */ From owner-svn-src-user@FreeBSD.ORG Tue Nov 17 05:25:55 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A5D891065676; Tue, 17 Nov 2009 05:25:55 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 939F58FC0A; Tue, 17 Nov 2009 05:25:55 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAH5Pt4u092398; Tue, 17 Nov 2009 05:25:55 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAH5Ptl0092396; Tue, 17 Nov 2009 05:25:55 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911170525.nAH5Ptl0092396@svn.freebsd.org> From: Kip Macy Date: Tue, 17 Nov 2009 05:25:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199344 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2009 05:25:55 -0000 Author: kmacy Date: Tue Nov 17 05:25:55 2009 New Revision: 199344 URL: http://svn.freebsd.org/changeset/base/199344 Log: track flag changes Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Tue Nov 17 04:59:43 2009 (r199343) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Tue Nov 17 05:25:55 2009 (r199344) @@ -1354,6 +1354,7 @@ arc_getblk(arc_buf_t *buf) struct vnode *vp; struct bufobj *bo; int flags = 0; + int preflags, postflags; if (type == ARC_BUFC_METADATA) { arc_space_consume(size); @@ -1376,8 +1377,14 @@ arc_getblk(arc_buf_t *buf) buf->b_hdr->b_flags &= ~ARC_BUF_CLONING; } else { newbp = getblk(vp, blkno, size, 0, 0, flags); + preflags = newbp->b_flags; + if (newbp->b_vp != NULL) brelvp(newbp); + postflags = newbp->b_flags; + + if (preflags != postflags) + CTR2(KTR_SPARE2, "arc_getblk() flags change pre %X post %X", preflags, postflags); data = newbp->b_data; } From owner-svn-src-user@FreeBSD.ORG Tue Nov 17 05:59:27 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CEDC0106566B; Tue, 17 Nov 2009 05:59:27 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id BCA148FC15; Tue, 17 Nov 2009 05:59:27 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAH5xQkR093040; Tue, 17 Nov 2009 05:59:26 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAH5xQGT093038; Tue, 17 Nov 2009 05:59:26 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911170559.nAH5xQGT093038@svn.freebsd.org> From: Kip Macy Date: Tue, 17 Nov 2009 05:59:26 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199345 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2009 05:59:27 -0000 Author: kmacy Date: Tue Nov 17 05:59:26 2009 New Revision: 199345 URL: http://svn.freebsd.org/changeset/base/199345 Log: associate buf with vnode if none exists currently Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Tue Nov 17 05:25:55 2009 (r199344) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Tue Nov 17 05:59:26 2009 (r199345) @@ -1337,8 +1337,13 @@ arc_bgetvp(arc_buf_t *buf) BO_UNLOCK(bo); } else brelse(bp); - } else + } else { + buf->b_bp->b_flags |= B_CACHE; + buf->b_bp->b_flags &= ~B_INVAL; + bgetvp(vp, buf->b_bp); BO_UNLOCK(bo); + } + } static void @@ -1384,7 +1389,7 @@ arc_getblk(arc_buf_t *buf) postflags = newbp->b_flags; if (preflags != postflags) - CTR2(KTR_SPARE2, "arc_getblk() flags change pre %X post %X", preflags, postflags); + CTR2(KTR_SPARE2, "arc_getblk() flags change pre %X post %X", preflags, postflags); data = newbp->b_data; } From owner-svn-src-user@FreeBSD.ORG Tue Nov 17 06:32:57 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 65098106566C; Tue, 17 Nov 2009 06:32:57 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 396FD8FC15; Tue, 17 Nov 2009 06:32:57 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAH6Wv3Y093756; Tue, 17 Nov 2009 06:32:57 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAH6WvX9093754; Tue, 17 Nov 2009 06:32:57 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911170632.nAH6WvX9093754@svn.freebsd.org> From: Kip Macy Date: Tue, 17 Nov 2009 06:32:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199346 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2009 06:32:57 -0000 Author: kmacy Date: Tue Nov 17 06:32:56 2009 New Revision: 199346 URL: http://svn.freebsd.org/changeset/base/199346 Log: don't bgetvp on a new buffer if the blkno is not set Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Tue Nov 17 05:59:26 2009 (r199345) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Tue Nov 17 06:32:56 2009 (r199346) @@ -495,8 +495,6 @@ static void arc_get_data_buf(arc_buf_t * static void arc_access(arc_buf_hdr_t *buf, kmutex_t *hash_lock); static int arc_evict_needed(arc_buf_contents_t type); static void arc_evict_ghost(arc_state_t *state, spa_t *spa, int64_t bytes); -static void arc_binval(arc_buf_t *buf, off_t blkno, struct vnode *vp, - size_t size, int flags, struct buf *bp); #define GHOST_STATE(state) \ ((state) == arc_mru_ghost || (state) == arc_mfu_ghost || \ @@ -1303,7 +1301,7 @@ arc_buf_add_ref(arc_buf_t *buf, void* ta static void arc_bgetvp(arc_buf_t *buf) { - off_t blkno = buf->b_hdr->b_dva.dva_word[1] & ~(1UL<<63); + uint64_t blkno = buf->b_hdr->b_dva.dva_word[1] & ~(1UL<<63); struct buf *bp = buf->b_bp; struct vnode *vp = spa_get_vnode(buf->b_hdr->b_spa); struct bufobj *bo = &vp->v_bufobj; @@ -1338,9 +1336,11 @@ arc_bgetvp(arc_buf_t *buf) } else brelse(bp); } else { - buf->b_bp->b_flags |= B_CACHE; - buf->b_bp->b_flags &= ~B_INVAL; - bgetvp(vp, buf->b_bp); + if (blkno != 0) { + buf->b_bp->b_flags |= B_CACHE; + buf->b_bp->b_flags &= ~B_INVAL; + bgetvp(vp, buf->b_bp); + } BO_UNLOCK(bo); } @@ -1352,7 +1352,7 @@ arc_getblk(arc_buf_t *buf) uint64_t size = buf->b_hdr->b_size; arc_buf_contents_t type = buf->b_hdr->b_type; spa_t *spa = buf->b_hdr->b_spa; - off_t blkno = buf->b_hdr->b_dva.dva_word[1] & ~(1UL<<63); + uint64_t blkno = buf->b_hdr->b_dva.dva_word[1] & ~(1UL<<63); void *data; struct buf *newbp, *bp; arc_buf_t *tbuf; @@ -3385,7 +3385,7 @@ arc_write_done(zio_t *zio) */ struct buf *bp = buf->b_bp; struct vnode *vp = spa_get_vnode(hdr->b_spa); - off_t blkno = hdr->b_dva.dva_word[1] & ~(1UL<<63); + uint64_t blkno = hdr->b_dva.dva_word[1] & ~(1UL<<63); CTR3(KTR_SPARE2, "arc_write_done() bp=%p flags %X blkno %ld", bp, bp ? bp->b_flags : 0, blkno); From owner-svn-src-user@FreeBSD.ORG Tue Nov 17 06:47:29 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2001D106566B; Tue, 17 Nov 2009 06:47:29 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id E8E458FC16; Tue, 17 Nov 2009 06:47:28 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAH6lSTu094128; Tue, 17 Nov 2009 06:47:28 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAH6lSGJ094126; Tue, 17 Nov 2009 06:47:28 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911170647.nAH6lSGJ094126@svn.freebsd.org> From: Kip Macy Date: Tue, 17 Nov 2009 06:47:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199347 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2009 06:47:29 -0000 Author: kmacy Date: Tue Nov 17 06:47:28 2009 New Revision: 199347 URL: http://svn.freebsd.org/changeset/base/199347 Log: - skip arc_bgetvp if blkno is 0 - set blkno and birth for newbp before brelse Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Tue Nov 17 06:32:56 2009 (r199346) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Tue Nov 17 06:47:28 2009 (r199347) @@ -1302,15 +1302,21 @@ static void arc_bgetvp(arc_buf_t *buf) { uint64_t blkno = buf->b_hdr->b_dva.dva_word[1] & ~(1UL<<63); - struct buf *bp = buf->b_bp; + struct buf *newbp, *bp = buf->b_bp; struct vnode *vp = spa_get_vnode(buf->b_hdr->b_spa); struct bufobj *bo = &vp->v_bufobj; arc_buf_hdr_t *hdr = buf->b_hdr; + if (blkno == 0) + return; + + newbp = buf->b_bp; + newbp->b_offset = hdr->b_birth; + newbp->b_blkno = newbp->b_lblkno = blkno; + BO_LOCK(bo); bp = gbincore(bo, blkno); if (bp != NULL) { - /* * XXX we have a race with getblk here */ @@ -1324,26 +1330,22 @@ arc_bgetvp(arc_buf_t *buf) (bp->b_flags & (B_CACHE|B_INVAL)) == B_CACHE) { bp->b_flags |= B_INVAL; bp->b_flags &= ~B_CACHE; - brelse(bp); - buf->b_bp->b_offset = hdr->b_birth; - buf->b_bp->b_flags |= B_CACHE; - buf->b_bp->b_flags &= ~B_INVAL; - + + newbp->b_flags |= B_CACHE; + newbp->b_flags &= ~B_INVAL; + BO_LOCK(bo); - bgetvp(vp, buf->b_bp); + bgetvp(vp, newbp); BO_UNLOCK(bo); } else brelse(bp); } else { - if (blkno != 0) { - buf->b_bp->b_flags |= B_CACHE; - buf->b_bp->b_flags &= ~B_INVAL; - bgetvp(vp, buf->b_bp); - } + newbp->b_flags |= B_CACHE; + newbp->b_flags &= ~B_INVAL; + bgetvp(vp, newbp); BO_UNLOCK(bo); } - } static void From owner-svn-src-user@FreeBSD.ORG Tue Nov 17 07:01:50 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11FAD106568D; Tue, 17 Nov 2009 07:01:50 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 00FFD8FC12; Tue, 17 Nov 2009 07:01:50 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAH71nhW094439; Tue, 17 Nov 2009 07:01:49 GMT (envelope-from dougb@svn.freebsd.org) Received: (from dougb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAH71ndS094437; Tue, 17 Nov 2009 07:01:49 GMT (envelope-from dougb@svn.freebsd.org) Message-Id: <200911170701.nAH71ndS094437@svn.freebsd.org> From: Doug Barton Date: Tue, 17 Nov 2009 07:01:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199348 - user/dougb/portmaster X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2009 07:01:50 -0000 Author: dougb Date: Tue Nov 17 07:01:49 2009 New Revision: 199348 URL: http://svn.freebsd.org/changeset/base/199348 Log: Add the --delete-build-only option to delete ports that are only build dependencies which are installed on this run. To make that work split the "create a global list of build-only deps" feature out from --packages-build and enable PM_BUILD_ONLY_LIST if either option is chosen. This allows the --delete-build-only option to also work without any of the --packages* options. Modified: user/dougb/portmaster/portmaster Modified: user/dougb/portmaster/portmaster ============================================================================== --- user/dougb/portmaster/portmaster Tue Nov 17 06:47:28 2009 (r199347) +++ user/dougb/portmaster/portmaster Tue Nov 17 07:01:49 2009 (r199348) @@ -193,6 +193,15 @@ safe_exit () { echo -e $INSTALLED_LIST ;; one) echo "===>>> $ilist complete" ; echo '' ;; esac + + if [ -n "$build_deps_il" ]; then + echo "===>>> Deleting installed build-only dependencies" + for f in $build_deps_il; do + [ -n "$PM_VERBOSE" ] && echo " $f" + pkg_delete -f $f + done + echo '' + fi else [ -n "$grep_deps" ] && pm_unlink $grep_deps @@ -214,10 +223,12 @@ safe_exit () { fi [ -n "$URB_YES" ] && echo "URB_DONE_LIST='$URB_DONE_LIST'" >> $IPC_SAVE - if [ "$PM_PACKAGES_BUILD" = pmp_doing_build_deps ]; then + if [ "$PM_BUILD_ONLY_LIST" = pmp_doing_build_deps ]; then echo "build_only_dl_g='$build_only_dl_g'" >> $IPC_SAVE echo "run_dl_g='$run_dl_g'" >> $IPC_SAVE fi + [ -n "$PM_DEL_BUILD_ONLY" ] && + echo "build_deps_il='$build_deps_il'" >> $IPC_SAVE fi exit ${1:-0} @@ -230,6 +241,7 @@ usage () { echo "Common flags: [--force-config] [-CGHKgntvw B|b f|i D|d]" echo " [[--packages|--packages-only] [-P|-PP] | [--packages-build]]" echo " [--packages-if-newer] [-m ]" + echo " [--delete-build-only]" echo " [-x ]" echo "${0##*/} [Common flags] " echo "${0##*/} [Common flags] " @@ -293,6 +305,9 @@ usage () { echo '-R used with -[rf] to skip ports updated on a previous run' echo '-a check all ports, update as necessary' echo '' + echo '--delete-build-only delete ports that are build-only dependencies' + echo ' after a successful run, only if installed this run' + echo '' echo 'NOTE: The -P[P] options imply -G' echo '-P|--packages use packages, but build port if not available' echo '-PP|--packages-only fail if no package is available' @@ -405,11 +420,15 @@ for var in "$@" ; do --packages-build) packages_init build unset PM_PACKAGES PM_PACKAGES_BUILD=pmp_build - export PM_PACKAGES_BUILD ;; + PM_BUILD_ONLY_LIST=pm_bol + export PM_PACKAGES_BUILD PM_BUILD_ONLY_LIST ;; --packages-if-newer) packages_init newer PM_PACKAGES_NEWER=pmp_newer export PM_PACKAGES_NEWER ;; -[A-Za-z0-9]*) newopts="$newopts $var" ;; + --delete-build-only) PM_DEL_BUILD_ONLY=pm_dbo + PM_BUILD_ONLY_LIST=pm_bol + export PM_DEL_BUILD_ONLY PM_BUILD_ONLY_LIST ;; --help) usage 0 ;; --version) version ; exit 0 ;; --clean-distfiles) CLEAN_DISTFILES=clean_distfiles ;; @@ -924,6 +943,8 @@ shift $(( $OPTIND - 1 )) fail "The -F and -G options are mutually exclusive" [ -n "$NO_RECURSIVE_CONFIG" -a -n "$PM_PACKAGES_BUILD" ] && fail 'The --packages-build and -G options are mutually exclusive' +[ -n "$NO_RECURSIVE_CONFIG" -a -n "$PM_DEL_BUILD_ONLY" ] && + fail 'The --delete-build-only and -G options are mutually exclusive' #=============== Begin functions for getopts features and main =============== @@ -1649,7 +1670,7 @@ dependency_check () { # XXXX - if [ "$PM_PACKAGES_BUILD" = pmp_doing_build_deps ]; then + if [ "$PM_BUILD_ONLY_LIST" = pmp_doing_build_deps ]; then #echo '' ; echo "Debug> d_port_list: $d_port_list" @@ -1698,8 +1719,8 @@ dependency_check () { build_only_dl_g=" $temp_bodlg " -#echo '' ; echo "Debug> build_only_dl_g: $build_only_dl_g" -#echo '' ; echo "Debug> run_dl_g: $run_dl_g" ; echo '' +#echo '' ; echo "Debug> build_only_dl_g: X${build_only_dl_g}X" +#echo '' ; echo "Debug> run_dl_g: X${run_dl_g}X" ; echo '' fi local d_port origin iport conflicts glob confl_p udf @@ -1863,10 +1884,14 @@ multiport () { echo -e $portlist if [ -n "$CONFIG_ONLY" ]; then - if [ -n "$PM_PACKAGES_BUILD" ]; then - PM_PACKAGES_BUILD=pmp_doing_build_deps - run_dl_g=" " ; build_only_dl_g=" " - export PM_PACKAGES_BUILD run_dl_g build_only_dl_g + if [ -n "$PM_BUILD_ONLY_LIST" ]; then + PM_BUILD_ONLY_LIST=pmp_doing_build_deps + run_dl_g='' ; build_only_dl_g='' + export run_dl_g build_only_dl_g + fi + if [ -n "$PM_DEL_BUILD_ONLY" ]; then + build_deps_il='' + export build_deps_il fi for port in $worklist; do @@ -1879,10 +1904,12 @@ multiport () { echo "===>>> Starting build for multiple ports <<<===" echo '' - [ -n "$PM_PACKAGES_BUILD" ] && unset run_dl_g - fi - + if [ -n "$PM_BUILD_ONLY_LIST" ]; then + unset run_dl_g + PM_BUILD_ONLY_LIST=pm_bol #echo '' ; echo "Debug> build_only_dl_g multiport: $build_only_dl_g" ; echo '' + fi + fi export PM_BUILDING=pmbuildingmultiport for port in $worklist; do @@ -2276,10 +2303,14 @@ if [ -n "$CONFIG_ONLY" ]; then # XXXX if [ "$$" -eq "$PM_PARENT_PID" ]; then # Keep in sync in multiport() - if [ -n "$PM_PACKAGES_BUILD" ]; then - PM_PACKAGES_BUILD=pmp_doing_build_deps - run_dl_g=" " ; build_only_dl_g=" " - export PM_PACKAGES_BUILD run_dl_g build_only_dl_g + if [ -n "$PM_BUILD_ONLY_LIST" ]; then + PM_BUILD_ONLY_LIST=pmp_doing_build_deps + run_dl_g='' ; build_only_dl_g='' + export run_dl_g build_only_dl_g + fi + if [ -n "$PM_DEL_BUILD_ONLY" ]; then + build_deps_il='' + export build_deps_il fi fi @@ -2338,8 +2369,11 @@ if [ -n "$CONFIG_ONLY" ]; then fi # XXXX - [ -n "$PM_PACKAGES_BUILD" ] && unset run_dl_g + if [ -n "$PM_BUILD_ONLY_LIST" ]; then + unset run_dl_g + PM_BUILD_ONLY_LIST=pm_bol #echo '' ; echo "Debug> final build_only_dl_g: $build_only_dl_g" ; echo '' + fi check_fetch_only unset CONFIG_SEEN_LIST CONFIG_ONLY @@ -2351,14 +2385,17 @@ fi cd $pd/$portdir -if [ -n "$PM_PACKAGES_BUILD" ]; then +if [ -n "$PM_BUILD_ONLY_LIST" ]; then case "$build_only_dl_g" in *" $pd/$portdir "*) + [ -n "$PM_PACKAGES_BUILD" ] && PM_PACKAGES_BUILD=doing_build_only_dep + [ -n "$PM_DEL_BUILD_ONLY" ] && PM_DEL_BUILD_ONLY=doing_build_only_dep #echo '' ; echo "Debug> DOING BUILD ONLY DEP" ; echo '' + ;; - PM_PACKAGES_BUILD=doing_build_only_dep ;; - *) PM_PACKAGES_BUILD=pmp_build ;; + *) [ -n "$PM_PACKAGES_BUILD" ] && PM_PACKAGES_BUILD=pmp_build + [ -n "$PM_DEL_BUILD_ONLY" ] && PM_DEL_BUILD_ONLY=pm_dbo ;; esac fi @@ -2651,6 +2688,9 @@ else fi echo '' +[ "$PM_DEL_BUILD_ONLY" = doing_build_only_dep ] && + build_deps_il="$build_deps_il $new_port" + # Remove saved libs that match newly installed files temp=`find $LOCALBASE_COMPAT/pkg -type d -empty 2>/dev/null` From owner-svn-src-user@FreeBSD.ORG Tue Nov 17 07:21:28 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 43E2E106566B; Tue, 17 Nov 2009 07:21:28 +0000 (UTC) (envelope-from edwin@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 316DB8FC1D; Tue, 17 Nov 2009 07:21:28 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAH7LScx094882; Tue, 17 Nov 2009 07:21:28 GMT (envelope-from edwin@svn.freebsd.org) Received: (from edwin@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAH7LS5t094878; Tue, 17 Nov 2009 07:21:28 GMT (envelope-from edwin@svn.freebsd.org) Message-Id: <200911170721.nAH7LS5t094878@svn.freebsd.org> From: Edwin Groothuis Date: Tue, 17 Nov 2009 07:21:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199350 - user/edwin/locale/usr.bin/unicodename2utf8 X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Nov 2009 07:21:28 -0000 Author: edwin Date: Tue Nov 17 07:21:27 2009 New Revision: 199350 URL: http://svn.freebsd.org/changeset/base/199350 Log: Add the utf82unicode feature as requested by gabor@ Modified: user/edwin/locale/usr.bin/unicodename2utf8/Makefile user/edwin/locale/usr.bin/unicodename2utf8/unicodename2utf8.1 user/edwin/locale/usr.bin/unicodename2utf8/unicodename2utf8.c Modified: user/edwin/locale/usr.bin/unicodename2utf8/Makefile ============================================================================== --- user/edwin/locale/usr.bin/unicodename2utf8/Makefile Tue Nov 17 07:06:41 2009 (r199349) +++ user/edwin/locale/usr.bin/unicodename2utf8/Makefile Tue Nov 17 07:21:27 2009 (r199350) @@ -3,8 +3,11 @@ PROG= unicodename2utf8 SRCS= unicodename2utf8.c -NO_MAN= yes WARNS?= 6 +MAN= unicodename2utf8.1 + +LINKS= ${BINDIR}/unicodename2utf8 ${BINDIR}/utf82unicodename +MLINKS= unicodename2utf8.1 utf82unicodename.1 test: ./unicodename2utf8 \ Modified: user/edwin/locale/usr.bin/unicodename2utf8/unicodename2utf8.1 ============================================================================== --- user/edwin/locale/usr.bin/unicodename2utf8/unicodename2utf8.1 Tue Nov 17 07:06:41 2009 (r199349) +++ user/edwin/locale/usr.bin/unicodename2utf8/unicodename2utf8.1 Tue Nov 17 07:21:27 2009 (r199350) @@ -28,22 +28,31 @@ .Dt unicode2utf8 1 .Os .Sh NAME -.Nm unicode2utf8 -.Nd converts a file with Unicode name definitions into UTF-8 character -definitions. +.Nm unicodename2utf8 , +.Nm utf82unicodename +.Nd convert a file with Unicode name definitions into UTF-8 character and +vice-versa. .Sh SYNOPSIS .Nm +.Fl cldr Ar directory +.Op Fl -input Ar filename +.Op Fl -output Ar filename +.Op Fl -reverse +.Nm utf82unicode .Fl -cldr Ar directory -.Fl -input Ar filename -.Fl -output Ar filename +.Op Fl -input Ar filename +.Op Fl -output Ar filename +.Op Fl -reverse .Sh DESCRIPTION The .Nm -utility is made to convert the Unicode encoded strings in the -contents of the specified input file with the corresponding UTF-8 -character definitions. -.Pp -Lines starting with a # are copied as-is. +utility converts the Unicode encoded strings in the contents of the +specified input file with the corresponding UTF-8 character +definitions. +The +.Nm utf82unicodename +utility converts the UTF-8 encoded strings in the contents of the +specified input file with the corresponding Unicode names. .Pp The Unicode encoded strings are specified between a '<' and a '>' sign. @@ -66,15 +75,16 @@ By default this should point to but for maintainers of the FreeBSD locale database this could point to their own extracted copy of the CLDR database. .It Fl -input Ar filename -The source file with the Unicode encoded strings. +The source file. +If not specific, stdin will be used. .It Fl -output Ar filename -The destination file with the Unicode encoded strings replaced with -their UTF-8 equivalents. +The output file. +If not specific, stdout will be used. +.It Fl -reverse +If specified, do the reverse conversions. .El .Sh EXIT STATUS -The -.Nm -utility exits 0 on success, and >0 if an error occurs. +The utilties exit with 0 on success, and >0 if an error occurs. .Sh SEE ALSO .Xr iconv 1 , .Xr bsdiconv 1 @@ -85,7 +95,5 @@ the maintainers of the file .Pa /usr/share/misc/UTF-8.cm .El .Sh AUTHORS -The -.Nm -utility and this manual page were written by +The utilities and this manual page were written by .An Edwin Groothuis Aq edwin@FreeBSD.org . Modified: user/edwin/locale/usr.bin/unicodename2utf8/unicodename2utf8.c ============================================================================== --- user/edwin/locale/usr.bin/unicodename2utf8/unicodename2utf8.c Tue Nov 17 07:06:41 2009 (r199349) +++ user/edwin/locale/usr.bin/unicodename2utf8/unicodename2utf8.c Tue Nov 17 07:21:27 2009 (r199350) @@ -37,21 +37,25 @@ __FBSDID("$FreeBSD$"); #include #include -#define MAXBUF 512 +#define MAXBUF 4098 struct utf8map { - char *uniname; + char *unicodename; char *utf8char; int utf8len; - struct utf8map *next; + struct utf8map *next_utf8; + struct utf8map *next_unicodename; }; -struct utf8map *utf8map_head[256]; +struct utf8map *head_utf8[256]; +struct utf8map *head_unicodename[256]; void usage(void); -struct utf8map *get_utf8map(char *dir); -struct utf8map *find_utf8map(char *unidata); -void translate(char *file_in, char *file_out); +struct utf8map *get_mappings(char *dir); +struct utf8map *find_utf8map(char *unicodename); +struct utf8map *find_unicodemap(char *utf8data, int len); +void translate_into_utf8(char *file_in, char *file_out); +void translate_into_unicodename(char *file_in, char *file_out); int debug = 0; @@ -60,12 +64,17 @@ main(int argc, char *argv[]) { char *cldr = NULL, *file_in = NULL, *file_out = NULL; char ch; + int reverse = 0; + + if (strcmp(argv[0], "utf82unicode") == 0) + reverse = 1; static struct option longopts[] = { { "cldr", required_argument, NULL, 1 }, + { "debug", no_argument, NULL, 4 }, { "input", required_argument, NULL, 2 }, { "output", required_argument, NULL, 3 }, - { "debug", no_argument, NULL, 4 }, + { "reverse", no_argument, NULL, 5 }, { NULL, 0, NULL, 0 } }; @@ -83,6 +92,9 @@ main(int argc, char *argv[]) case 4: debug++; break; + case 5: + reverse = !reverse; + break; default: usage(); } @@ -90,32 +102,110 @@ main(int argc, char *argv[]) argc -= optind; argv += optind; - if (cldr == NULL || file_in == NULL || file_out == NULL) + if (cldr == NULL) usage(); - get_utf8map(cldr); - translate(file_in, file_out); + get_mappings(cldr); + if (!reverse) + translate_into_utf8(file_in, file_out); + else + translate_into_unicodename(file_in, file_out); +} + +void +translate_into_unicodename(char *file_in, char *file_out) +{ + struct utf8map *map; + FILE *fin, *fout; + unsigned char *p, line[MAXBUF]; + int len; + + if (file_in == NULL) + fin = stdin; + else + if ((fin = fopen(file_in, "r")) == NULL) + errx(EX_DATAERR, "Cannot open %s for reading.", + file_in); + if (file_out == NULL) + fout = stdout; + else + if ((fout = fopen(file_out, "w")) == NULL) + errx(EX_DATAERR, "Cannot open %s for writing.", + file_out); + + fprintf(fout, +"#\n" +"# Do not edit this file, it is created automatically by the utf82unicodename\n" +"# utility. All changes to this file will be lost.\n" +"# The source of this file was %s\n" +"#\n", + file_in == NULL ? "read from stdin" : file_in); + + while (!feof(fin)) { + if (fgets(line, sizeof(line), fin) != NULL) { + if (line[0] == '#') { + fprintf(fout, "%s", line); + continue; + } + + p = line; + while (*p != '\0') { + if (*p == 0x0a) { + fwrite("\n", 1, 1, fout); + p++; + continue; + } + if ((*p > 0x7F && *p < 0xC2) + || (*p > 0xDF && *p < 0xE0) + || (*p > 0xEF)) + errx(EX_DATAERR, + "Invalid UTF-8 character '%c'", + *p); + + len = *p <= 0x7F ? 1 : *p <= 0xDF ? 2 : 3; + if ((map = find_unicodemap(p, len)) == NULL) { + errx(EX_DATAERR, + "Cannot find translation for '%s'", + p + 1); + } + fprintf(fout, "<%s>", map->unicodename); + p += len; + } + + } + } + + fclose(fin); + fclose(fout); } void -translate(char *file_in, char *file_out) +translate_into_utf8(char *file_in, char *file_out) { struct utf8map *map; FILE *fin, *fout; char *p, *q1, *q2, line[MAXBUF]; - if ((fin = fopen(file_in, "r")) == NULL) - errx(EX_DATAERR, "Cannot open %s for reading.", file_in); - if ((fout = fopen(file_out, "w")) == NULL) - errx(EX_DATAERR, "Cannot open %s for writing.", file_out); + if (file_in == NULL) + fin = stdin; + else + if ((fin = fopen(file_in, "r")) == NULL) + errx(EX_DATAERR, "Cannot open %s for reading.", + file_in); + if (file_out == NULL) + fout = stdout; + else + if ((fout = fopen(file_out, "w")) == NULL) + errx(EX_DATAERR, "Cannot open %s for writing.", + file_out); fprintf(fout, "#\n" -"# Do not edit this file, it is created automatically by the unicode2utf8\n" +"# Do not edit this file, it is created automatically by the unicodename2utf8\n" "# utility. All changes to this file will be lost.\n" "# The source of this file was %s\n" "#\n", - file_in); + file_in == NULL ? "read from stdin" : file_in); while (!feof(fin)) { if (fgets(line, sizeof(line), fin) != NULL) { @@ -156,29 +246,47 @@ translate(char *file_in, char *file_out) } struct utf8map * +find_unicodemap(char *utf8, int len) +{ + struct utf8map *p; + int hashindex = utf8[len - 1]; + + p = head_utf8[hashindex]; + while (p != NULL) { + if (debug) + printf("'%s' - '%s'\n", p->utf8char, utf8); + if (strncmp(p->utf8char, utf8, len) == 0) + return p; + p = p->next_utf8; + } + + return NULL; +} + +struct utf8map * find_utf8map(char *uniname) { struct utf8map *p; int hashindex = uniname[strlen(uniname) - 1]; - p = utf8map_head[hashindex]; + p = head_unicodename[hashindex]; while (p != NULL) { - if (strcmp(p->uniname, uniname) == 0) - return p; if (debug) - printf("'%s' - '%s'\n", p->uniname, uniname); - p = p->next; + printf("'%s' - '%s'\n", p->unicodename, uniname); + if (strcmp(p->unicodename, uniname) == 0) + return p; + p = p->next_unicodename; } return NULL; } struct utf8map * -get_utf8map(char *dir) +get_mappings(char *dir) { - struct utf8map *new; + struct utf8map *new, *prev = NULL; FILE *fin; - int len, i, hashindex; + int len, i, hashindex_utf8, hashindex_unicodename; char filename[MAXPATHLEN], uniname[MAXBUF], utf8char[MAXBUF], *p; sprintf(filename, "%s/posix/UTF-8.cm", dir); @@ -207,7 +315,10 @@ get_utf8map(char *dir) if ((p = strchr(uniname, '>')) == NULL) errx(EX_DATAERR, "No trailing '>' for %s", uniname); - hashindex = p[-1]; + + /* Use the last character in the for hashing */ + hashindex_unicodename = p[-1]; + *p = '\0'; if (uniname[0] != '<') errx(EX_DATAERR, "No leading '<' for %s", @@ -236,15 +347,30 @@ get_utf8map(char *dir) utf8char[len] = 0; } + /* use the last character in the utf8data for hashing */ + hashindex_utf8 = utf8char[len - 1]; + if (debug) printf("-%s-%s-\n", uniname, utf8char); new = (struct utf8map *)malloc(sizeof(struct utf8map)); - new->next = utf8map_head[hashindex]; - new->uniname = strdup(uniname + 1); + new->next_utf8 = head_utf8[hashindex_utf8]; + new->next_unicodename = + head_unicodename[hashindex_unicodename]; + new->unicodename = strdup(uniname + 1); new->utf8char = strdup(utf8char); new->utf8len = len; - utf8map_head[hashindex] = new; + head_unicodename[hashindex_unicodename] = new; + + /* + * If the previous UTF-8 character has the same name as + * this one, then don't put it in the hash_utf8 array. + * For example: and + */ + if (prev == NULL + || strncmp(prev->utf8char, utf8char, len) != 0) + head_utf8[hashindex_utf8] = new; + prev = new; } } @@ -259,7 +385,10 @@ void usage(void) { - printf("Usage: unicode2utf8 --cldr=dir --input=file --output=file\n"); + printf( +"Usage: unicodename2utf8 --cldr=dir [--input=file] [--output=file] [--reverse]\n" +"Usage: utf82unicodename --cldr=dir [--input=file] [--output=file] [--reverse]\n" +); exit(EX_USAGE); } From owner-svn-src-user@FreeBSD.ORG Wed Nov 18 00:04:04 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 43FA7106568D; Wed, 18 Nov 2009 00:04:04 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 186398FC1E; Wed, 18 Nov 2009 00:04:04 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAI0431w047416; Wed, 18 Nov 2009 00:04:03 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAI043NM047414; Wed, 18 Nov 2009 00:04:03 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911180004.nAI043NM047414@svn.freebsd.org> From: Kip Macy Date: Wed, 18 Nov 2009 00:04:03 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199460 - in user/kmacy/releng_8_fcs_buf/sys: cddl/contrib/opensolaris/uts/common/fs/zfs sys X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2009 00:04:04 -0000 Author: kmacy Date: Wed Nov 18 00:04:03 2009 New Revision: 199460 URL: http://svn.freebsd.org/changeset/base/199460 Log: don't rely on b_offset to be unmodified Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c user/kmacy/releng_8_fcs_buf/sys/sys/buf.h Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Tue Nov 17 23:03:38 2009 (r199459) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 00:04:03 2009 (r199460) @@ -452,6 +452,7 @@ struct arc_write_callback { arc_buf_t *awcb_buf; }; + struct arc_buf_hdr { /* protected by hash lock */ dva_t b_dva; @@ -1307,11 +1308,14 @@ arc_bgetvp(arc_buf_t *buf) struct bufobj *bo = &vp->v_bufobj; arc_buf_hdr_t *hdr = buf->b_hdr; - if (blkno == 0) + if (zfs_page_cache_disable) + return; + + if (blkno == 0 || hdr->b_birth == 0) return; newbp = buf->b_bp; - newbp->b_offset = hdr->b_birth; + newbp->b_birth = hdr->b_birth; newbp->b_blkno = newbp->b_lblkno = blkno; BO_LOCK(bo); @@ -1325,11 +1329,16 @@ arc_bgetvp(arc_buf_t *buf) /* * buffer is usable for this mapping */ - if (!(hdr->b_flags & ARC_IO_ERROR) && - (bp->b_offset <= hdr->b_birth) && - (bp->b_flags & (B_CACHE|B_INVAL)) == B_CACHE) { + if (((bp->b_flags & (B_CACHE|B_INVAL)) == B_CACHE) && + (bp->b_birth > hdr->b_birth)) { + brelse(bp); + } else if (hdr->b_flags & ARC_IO_ERROR) { bp->b_flags |= B_INVAL; - bp->b_flags &= ~B_CACHE; + bp->b_birth = 0; + brelse(bp); + } else { + bp->b_flags |= B_INVAL; + bp->b_birth = 0; brelse(bp); newbp->b_flags |= B_CACHE; @@ -1338,8 +1347,7 @@ arc_bgetvp(arc_buf_t *buf) BO_LOCK(bo); bgetvp(vp, newbp); BO_UNLOCK(bo); - } else - brelse(bp); + } } else { newbp->b_flags |= B_CACHE; newbp->b_flags &= ~B_INVAL; Modified: user/kmacy/releng_8_fcs_buf/sys/sys/buf.h ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/sys/buf.h Tue Nov 17 23:03:38 2009 (r199459) +++ user/kmacy/releng_8_fcs_buf/sys/sys/buf.h Wed Nov 18 00:04:03 2009 (r199460) @@ -104,7 +104,8 @@ struct buf { long b_resid; void (*b_iodone)(struct buf *); daddr_t b_blkno; /* Underlying physical block number. */ - off_t b_offset; /* Offset into file. */ + off_t b_offset; /* Offset into file. */ + uint64_t b_birth; /* birth txg */ TAILQ_ENTRY(buf) b_bobufs; /* (V) Buffer's associated vnode. */ struct buf *b_left; /* (V) splay tree link */ struct buf *b_right; /* (V) splay tree link */ From owner-svn-src-user@FreeBSD.ORG Wed Nov 18 00:15:24 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 215E51065697; Wed, 18 Nov 2009 00:15:24 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 10C048FC26; Wed, 18 Nov 2009 00:15:24 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAI0FNu2048120; Wed, 18 Nov 2009 00:15:23 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAI0FN8c048119; Wed, 18 Nov 2009 00:15:23 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911180015.nAI0FN8c048119@svn.freebsd.org> From: Kip Macy Date: Wed, 18 Nov 2009 00:15:23 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199461 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2009 00:15:24 -0000 Author: kmacy Date: Wed Nov 18 00:15:23 2009 New Revision: 199461 URL: http://svn.freebsd.org/changeset/base/199461 Log: mark saved buffer has not having an offset Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 00:04:03 2009 (r199460) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 00:15:23 2009 (r199461) @@ -1317,6 +1317,7 @@ arc_bgetvp(arc_buf_t *buf) newbp = buf->b_bp; newbp->b_birth = hdr->b_birth; newbp->b_blkno = newbp->b_lblkno = blkno; + newbp->b_offset = NOOFFSET; BO_LOCK(bo); bp = gbincore(bo, blkno); From owner-svn-src-user@FreeBSD.ORG Wed Nov 18 00:51:32 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5C954106566B; Wed, 18 Nov 2009 00:51:32 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 4BC7B8FC1F; Wed, 18 Nov 2009 00:51:32 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAI0pWMs049596; Wed, 18 Nov 2009 00:51:32 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAI0pWU5049594; Wed, 18 Nov 2009 00:51:32 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911180051.nAI0pWU5049594@svn.freebsd.org> From: Kip Macy Date: Wed, 18 Nov 2009 00:51:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199462 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2009 00:51:32 -0000 Author: kmacy Date: Wed Nov 18 00:51:31 2009 New Revision: 199462 URL: http://svn.freebsd.org/changeset/base/199462 Log: ensure that we have a "valid" offset for a freed buffer mSubmitted by: Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 00:15:23 2009 (r199461) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 00:51:31 2009 (r199462) @@ -1315,9 +1315,8 @@ arc_bgetvp(arc_buf_t *buf) return; newbp = buf->b_bp; - newbp->b_birth = hdr->b_birth; + newbp->b_offset = newbp->b_birth = hdr->b_birth; newbp->b_blkno = newbp->b_lblkno = blkno; - newbp->b_offset = NOOFFSET; BO_LOCK(bo); bp = gbincore(bo, blkno); From owner-svn-src-user@FreeBSD.ORG Wed Nov 18 01:43:12 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2DD651065679; Wed, 18 Nov 2009 01:43:12 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 1C9AA8FC12; Wed, 18 Nov 2009 01:43:12 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAI1hCVa051834; Wed, 18 Nov 2009 01:43:12 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAI1hBHo051832; Wed, 18 Nov 2009 01:43:12 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911180143.nAI1hBHo051832@svn.freebsd.org> From: Kip Macy Date: Wed, 18 Nov 2009 01:43:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199466 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2009 01:43:12 -0000 Author: kmacy Date: Wed Nov 18 01:43:11 2009 New Revision: 199466 URL: http://svn.freebsd.org/changeset/base/199466 Log: mark buffer invalid if it doesn't match the expect birth txg Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 01:35:36 2009 (r199465) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 01:43:11 2009 (r199466) @@ -1369,7 +1369,6 @@ arc_getblk(arc_buf_t *buf) struct vnode *vp; struct bufobj *bo; int flags = 0; - int preflags, postflags; if (type == ARC_BUFC_METADATA) { arc_space_consume(size); @@ -1392,14 +1391,9 @@ arc_getblk(arc_buf_t *buf) buf->b_hdr->b_flags &= ~ARC_BUF_CLONING; } else { newbp = getblk(vp, blkno, size, 0, 0, flags); - preflags = newbp->b_flags; - - if (newbp->b_vp != NULL) - brelvp(newbp); - postflags = newbp->b_flags; - - if (preflags != postflags) - CTR2(KTR_SPARE2, "arc_getblk() flags change pre %X post %X", preflags, postflags); + if (newbp->b_birth != buf->b_hdr->b_birth) + newbp->b_flags |= B_INVAL; + brelvp(newbp); data = newbp->b_data; } From owner-svn-src-user@FreeBSD.ORG Wed Nov 18 04:07:16 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87BAF106566C; Wed, 18 Nov 2009 04:07:16 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 5C42B8FC16; Wed, 18 Nov 2009 04:07:16 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAI47GQf057188; Wed, 18 Nov 2009 04:07:16 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAI47Gh4057186; Wed, 18 Nov 2009 04:07:16 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911180407.nAI47Gh4057186@svn.freebsd.org> From: Kip Macy Date: Wed, 18 Nov 2009 04:07:16 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199467 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2009 04:07:16 -0000 Author: kmacy Date: Wed Nov 18 04:07:16 2009 New Revision: 199467 URL: http://svn.freebsd.org/changeset/base/199467 Log: only cache buffers that have been read from or written to disk Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 01:43:11 2009 (r199466) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 04:07:16 2009 (r199467) @@ -1315,6 +1315,10 @@ arc_bgetvp(arc_buf_t *buf) return; newbp = buf->b_bp; + + if ((newbp->b_flags & (B_INVAL|B_CACHE)) != B_CACHE) + return; + newbp->b_offset = newbp->b_birth = hdr->b_birth; newbp->b_blkno = newbp->b_lblkno = blkno; @@ -1341,9 +1345,6 @@ arc_bgetvp(arc_buf_t *buf) bp->b_birth = 0; brelse(bp); - newbp->b_flags |= B_CACHE; - newbp->b_flags &= ~B_INVAL; - BO_LOCK(bo); bgetvp(vp, newbp); BO_UNLOCK(bo); @@ -1423,8 +1424,7 @@ arc_brelse(arc_buf_t *buf, void *data, s return; } - if (hdr->b_datacnt == 1) - arc_bgetvp(buf); + arc_bgetvp(buf); CTR4(KTR_SPARE2, "arc_brelse() bp=%p flags %X size %ld blkno=%ld", bp, bp->b_flags, size, bp->b_blkno); brelse(bp); @@ -2702,8 +2702,11 @@ arc_read_done(zio_t *zio) if (HDR_IN_HASH_TABLE(hdr)) buf_hash_remove(hdr); freeable = refcount_is_zero(&hdr->b_refcnt); + } else if (buf->b_bp != NULL) { + buf->b_bp->b_flags |= B_CACHE; + buf->b_bp->b_flags &= ~B_INVAL; } - + /* * Broadcast before we drop the hash_lock to avoid the possibility * that the hdr (and hence the cv) might be freed before we get to @@ -3415,7 +3418,10 @@ arc_write_done(zio_t *zio) arc_hdr_destroy(exists); exists = buf_hash_insert(hdr, &hash_lock); ASSERT3P(exists, ==, NULL); - } + } else if (buf->b_bp != NULL) { + buf->b_bp->b_flags |= B_CACHE; + buf->b_bp->b_flags &= ~B_INVAL; + } hdr->b_flags &= ~ARC_IO_IN_PROGRESS; /* if it's not anon, we are doing a scrub */ From owner-svn-src-user@FreeBSD.ORG Wed Nov 18 04:34:44 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 071CC106566C; Wed, 18 Nov 2009 04:34:44 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id E9C208FC1D; Wed, 18 Nov 2009 04:34:43 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAI4YhDo058183; Wed, 18 Nov 2009 04:34:43 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAI4YhX5058181; Wed, 18 Nov 2009 04:34:43 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911180434.nAI4YhX5058181@svn.freebsd.org> From: Kip Macy Date: Wed, 18 Nov 2009 04:34:43 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199468 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2009 04:34:44 -0000 Author: kmacy Date: Wed Nov 18 04:34:43 2009 New Revision: 199468 URL: http://svn.freebsd.org/changeset/base/199468 Log: only bind last buffer Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 04:07:16 2009 (r199467) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 04:34:43 2009 (r199468) @@ -1424,9 +1424,13 @@ arc_brelse(arc_buf_t *buf, void *data, s return; } - arc_bgetvp(buf); - CTR4(KTR_SPARE2, "arc_brelse() bp=%p flags %X size %ld blkno=%ld", - bp, bp->b_flags, size, bp->b_blkno); + if (hdr->b_datacnt == 1) { + arc_bgetvp(buf); + CTR4(KTR_SPARE2, "arc_brelse() bp=%p flags %X" + " size %ld blkno=%ld", + bp, bp->b_flags, size, bp->b_blkno); + } + brelse(bp); } From owner-svn-src-user@FreeBSD.ORG Wed Nov 18 05:21:35 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6EC6B106566C; Wed, 18 Nov 2009 05:21:35 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 5DACD8FC16; Wed, 18 Nov 2009 05:21:35 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAI5LZeE059379; Wed, 18 Nov 2009 05:21:35 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAI5LZeM059377; Wed, 18 Nov 2009 05:21:35 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911180521.nAI5LZeM059377@svn.freebsd.org> From: Kip Macy Date: Wed, 18 Nov 2009 05:21:35 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199472 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2009 05:21:35 -0000 Author: kmacy Date: Wed Nov 18 05:21:35 2009 New Revision: 199472 URL: http://svn.freebsd.org/changeset/base/199472 Log: always check for a buffer to invalidate Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 05:20:40 2009 (r199471) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 05:21:35 2009 (r199472) @@ -1315,10 +1315,6 @@ arc_bgetvp(arc_buf_t *buf) return; newbp = buf->b_bp; - - if ((newbp->b_flags & (B_INVAL|B_CACHE)) != B_CACHE) - return; - newbp->b_offset = newbp->b_birth = hdr->b_birth; newbp->b_blkno = newbp->b_lblkno = blkno; @@ -1340,7 +1336,7 @@ arc_bgetvp(arc_buf_t *buf) bp->b_flags |= B_INVAL; bp->b_birth = 0; brelse(bp); - } else { + } else if ((newbp->b_flags & (B_INVAL|B_CACHE)) == B_CACHE) { bp->b_flags |= B_INVAL; bp->b_birth = 0; brelse(bp); @@ -1348,7 +1344,8 @@ arc_bgetvp(arc_buf_t *buf) BO_LOCK(bo); bgetvp(vp, newbp); BO_UNLOCK(bo); - } + } else + brelse(bp); } else { newbp->b_flags |= B_CACHE; newbp->b_flags &= ~B_INVAL; From owner-svn-src-user@FreeBSD.ORG Wed Nov 18 05:44:18 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 563841065694; Wed, 18 Nov 2009 05:44:18 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 451E18FC15; Wed, 18 Nov 2009 05:44:18 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAI5iIB5059878; Wed, 18 Nov 2009 05:44:18 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAI5iIm4059876; Wed, 18 Nov 2009 05:44:18 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911180544.nAI5iIm4059876@svn.freebsd.org> From: Kip Macy Date: Wed, 18 Nov 2009 05:44:18 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199473 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2009 05:44:18 -0000 Author: kmacy Date: Wed Nov 18 05:44:17 2009 New Revision: 199473 URL: http://svn.freebsd.org/changeset/base/199473 Log: always invalidate older cached buffer Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 05:21:35 2009 (r199472) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 05:44:17 2009 (r199473) @@ -1336,16 +1336,17 @@ arc_bgetvp(arc_buf_t *buf) bp->b_flags |= B_INVAL; bp->b_birth = 0; brelse(bp); - } else if ((newbp->b_flags & (B_INVAL|B_CACHE)) == B_CACHE) { + } else { + bp->b_flags |= B_INVAL; bp->b_birth = 0; brelse(bp); - - BO_LOCK(bo); - bgetvp(vp, newbp); - BO_UNLOCK(bo); - } else - brelse(bp); + if ((newbp->b_flags & (B_INVAL|B_CACHE)) == B_CACHE) { + BO_LOCK(bo); + bgetvp(vp, newbp); + BO_UNLOCK(bo); + } + } } else { newbp->b_flags |= B_CACHE; newbp->b_flags &= ~B_INVAL; From owner-svn-src-user@FreeBSD.ORG Wed Nov 18 05:52:10 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DB0B110656C2; Wed, 18 Nov 2009 05:52:10 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id CA0728FC26; Wed, 18 Nov 2009 05:52:10 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAI5q914060079; Wed, 18 Nov 2009 05:52:09 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAI5q9rG060077; Wed, 18 Nov 2009 05:52:09 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911180552.nAI5q9rG060077@svn.freebsd.org> From: Kip Macy Date: Wed, 18 Nov 2009 05:52:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199474 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2009 05:52:10 -0000 Author: kmacy Date: Wed Nov 18 05:52:09 2009 New Revision: 199474 URL: http://svn.freebsd.org/changeset/base/199474 Log: simplify arc_bgetvp Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 05:44:17 2009 (r199473) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 05:52:09 2009 (r199474) @@ -1332,16 +1332,12 @@ arc_bgetvp(arc_buf_t *buf) if (((bp->b_flags & (B_CACHE|B_INVAL)) == B_CACHE) && (bp->b_birth > hdr->b_birth)) { brelse(bp); - } else if (hdr->b_flags & ARC_IO_ERROR) { - bp->b_flags |= B_INVAL; - bp->b_birth = 0; - brelse(bp); } else { - bp->b_flags |= B_INVAL; bp->b_birth = 0; brelse(bp); - if ((newbp->b_flags & (B_INVAL|B_CACHE)) == B_CACHE) { + if (!(hdr->b_flags & ARC_IO_ERROR) && + (newbp->b_flags & (B_INVAL|B_CACHE)) == B_CACHE) { BO_LOCK(bo); bgetvp(vp, newbp); BO_UNLOCK(bo); From owner-svn-src-user@FreeBSD.ORG Wed Nov 18 22:31:10 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE2A9106566B; Wed, 18 Nov 2009 22:31:10 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id CC8068FC1A; Wed, 18 Nov 2009 22:31:10 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAIMVAqc086233; Wed, 18 Nov 2009 22:31:10 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAIMVAEf086231; Wed, 18 Nov 2009 22:31:10 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911182231.nAIMVAEf086231@svn.freebsd.org> From: Kip Macy Date: Wed, 18 Nov 2009 22:31:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199495 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2009 22:31:11 -0000 Author: kmacy Date: Wed Nov 18 22:31:10 2009 New Revision: 199495 URL: http://svn.freebsd.org/changeset/base/199495 Log: only cache new buffer if the buffer is valid and the I/O succeded Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 22:14:36 2009 (r199494) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Nov 18 22:31:10 2009 (r199495) @@ -188,6 +188,7 @@ SYSCTL_INT(_vfs_zfs, OID_AUTO, mdcomp_di &zfs_mdcomp_disable, 0, "Disable metadata compression"); static int zfs_page_cache_disable = 0; +TUNABLE_INT("vfs.zfs.page_cache_disable", &zfs_page_cache_disable); SYSCTL_INT(_vfs_zfs, OID_AUTO, page_cache_disable, CTLFLAG_RDTUN, &zfs_page_cache_disable, 0, "Disable backing ARC with page cache "); @@ -1343,9 +1344,8 @@ arc_bgetvp(arc_buf_t *buf) BO_UNLOCK(bo); } } - } else { - newbp->b_flags |= B_CACHE; - newbp->b_flags &= ~B_INVAL; + } else if (!(hdr->b_flags & ARC_IO_ERROR) && + (newbp->b_flags & (B_INVAL|B_CACHE)) == B_CACHE) { bgetvp(vp, newbp); BO_UNLOCK(bo); } From owner-svn-src-user@FreeBSD.ORG Thu Nov 19 01:04:32 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F518106566C; Thu, 19 Nov 2009 01:04:32 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 2D8FE8FC1B; Thu, 19 Nov 2009 01:04:32 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAJ14WNP089335; Thu, 19 Nov 2009 01:04:32 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAJ14Was089333; Thu, 19 Nov 2009 01:04:32 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911190104.nAJ14Was089333@svn.freebsd.org> From: Kip Macy Date: Thu, 19 Nov 2009 01:04:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199500 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2009 01:04:32 -0000 Author: kmacy Date: Thu Nov 19 01:04:31 2009 New Revision: 199500 URL: http://svn.freebsd.org/changeset/base/199500 Log: don't return from arc_bgetvp without dropping the bufobj lock Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Nov 19 00:00:31 2009 (r199499) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Nov 19 01:04:31 2009 (r199500) @@ -1348,7 +1348,8 @@ arc_bgetvp(arc_buf_t *buf) (newbp->b_flags & (B_INVAL|B_CACHE)) == B_CACHE) { bgetvp(vp, newbp); BO_UNLOCK(bo); - } + } else + BO_UNLOCK(bo); } static void From owner-svn-src-user@FreeBSD.ORG Thu Nov 19 01:29:10 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC35F106566C; Thu, 19 Nov 2009 01:29:10 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id A0CE58FC17; Thu, 19 Nov 2009 01:29:10 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAJ1TAKM089898; Thu, 19 Nov 2009 01:29:10 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAJ1TAlb089896; Thu, 19 Nov 2009 01:29:10 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911190129.nAJ1TAlb089896@svn.freebsd.org> From: Kip Macy Date: Thu, 19 Nov 2009 01:29:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199503 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2009 01:29:10 -0000 Author: kmacy Date: Thu Nov 19 01:29:10 2009 New Revision: 199503 URL: http://svn.freebsd.org/changeset/base/199503 Log: invalidate older buffer on first free Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Nov 19 01:27:22 2009 (r199502) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Nov 19 01:29:10 2009 (r199503) @@ -1328,23 +1328,24 @@ arc_bgetvp(arc_buf_t *buf) BUF_LOCK(bp, LK_EXCLUSIVE | LK_INTERLOCK, BO_MTX(bo)); bremfree(bp); /* - * buffer is usable for this mapping + * buffer is not valid or is older */ - if (((bp->b_flags & (B_CACHE|B_INVAL)) == B_CACHE) && - (bp->b_birth > hdr->b_birth)) { - brelse(bp); - } else { + if (((bp->b_flags & (B_CACHE|B_INVAL)) != B_CACHE) || + (bp->b_birth <= hdr->b_birth)) { bp->b_flags |= B_INVAL; bp->b_birth = 0; - brelse(bp); - if (!(hdr->b_flags & ARC_IO_ERROR) && - (newbp->b_flags & (B_INVAL|B_CACHE)) == B_CACHE) { - BO_LOCK(bo); - bgetvp(vp, newbp); - BO_UNLOCK(bo); - } } - } else if (!(hdr->b_flags & ARC_IO_ERROR) && + brelse(bp); + + if ((hdr->b_datacnt == 1) && + !(hdr->b_flags & ARC_IO_ERROR) && + (newbp->b_flags & (B_INVAL|B_CACHE)) == B_CACHE) { + BO_LOCK(bo); + bgetvp(vp, newbp); + BO_UNLOCK(bo); + } + } else if ((hdr->b_datacnt == 1) && + !(hdr->b_flags & ARC_IO_ERROR) && (newbp->b_flags & (B_INVAL|B_CACHE)) == B_CACHE) { bgetvp(vp, newbp); BO_UNLOCK(bo); @@ -1419,12 +1420,10 @@ arc_brelse(arc_buf_t *buf, void *data, s return; } - if (hdr->b_datacnt == 1) { - arc_bgetvp(buf); - CTR4(KTR_SPARE2, "arc_brelse() bp=%p flags %X" - " size %ld blkno=%ld", - bp, bp->b_flags, size, bp->b_blkno); - } + arc_bgetvp(buf); + CTR4(KTR_SPARE2, "arc_brelse() bp=%p flags %X" + " size %ld blkno=%ld", + bp, bp->b_flags, size, bp->b_blkno); brelse(bp); } From owner-svn-src-user@FreeBSD.ORG Thu Nov 19 04:47:43 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EED57106566C; Thu, 19 Nov 2009 04:47:43 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id C577B8FC17; Thu, 19 Nov 2009 04:47:43 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAJ4lhN7093929; Thu, 19 Nov 2009 04:47:43 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAJ4lhw3093927; Thu, 19 Nov 2009 04:47:43 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911190447.nAJ4lhw3093927@svn.freebsd.org> From: Kip Macy Date: Thu, 19 Nov 2009 04:47:43 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199505 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2009 04:47:44 -0000 Author: kmacy Date: Thu Nov 19 04:47:43 2009 New Revision: 199505 URL: http://svn.freebsd.org/changeset/base/199505 Log: aggressively invalidate any cached buffers on write completion Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Nov 19 03:37:06 2009 (r199504) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Nov 19 04:47:43 2009 (r199505) @@ -1301,6 +1301,41 @@ arc_buf_add_ref(arc_buf_t *buf, void* ta } static void +arc_brelvp(arc_buf_hdr_t *hdr) +{ + uint64_t blkno = hdr->b_dva.dva_word[1] & ~(1UL<<63); + struct vnode *vp = spa_get_vnode(hdr->b_spa); + struct bufobj *bo = &vp->v_bufobj; + struct buf *bp; + + if (zfs_page_cache_disable) + return; + + if (blkno == 0 || hdr->b_birth == 0) + return; + + BO_LOCK(bo); + bp = gbincore(bo, blkno); + if (bp != NULL) { + /* + * XXX we have a race with getblk here + */ + BUF_LOCK(bp, LK_EXCLUSIVE | LK_INTERLOCK, BO_MTX(bo)); + bremfree(bp); + /* + * buffer is not valid or is older + */ + if (((bp->b_flags & (B_CACHE|B_INVAL)) != B_CACHE) || + (bp->b_birth <= hdr->b_birth)) { + bp->b_flags |= B_INVAL; + bp->b_birth = 0; + } + brelse(bp); + } else + BO_UNLOCK(bo); +} + +static void arc_bgetvp(arc_buf_t *buf) { uint64_t blkno = buf->b_hdr->b_dva.dva_word[1] & ~(1UL<<63); @@ -1344,7 +1379,7 @@ arc_bgetvp(arc_buf_t *buf) bgetvp(vp, newbp); BO_UNLOCK(bo); } - } else if ((hdr->b_datacnt == 1) && + } else if ((hdr->b_datacnt == 1) && !(hdr->b_flags & ARC_IO_ERROR) && (newbp->b_flags & (B_INVAL|B_CACHE)) == B_CACHE) { bgetvp(vp, newbp); @@ -3417,6 +3452,8 @@ arc_write_done(zio_t *zio) exists = buf_hash_insert(hdr, &hash_lock); ASSERT3P(exists, ==, NULL); } else if (buf->b_bp != NULL) { + arc_brelvp(hdr); + buf->b_bp->b_flags |= B_CACHE; buf->b_bp->b_flags &= ~B_INVAL; } @@ -3432,6 +3469,7 @@ arc_write_done(zio_t *zio) * This is an anonymous buffer with no user callback, * destroy it if there are no active references. */ + arc_brelvp(hdr); mutex_enter(&arc_eviction_mtx); destroy_hdr = refcount_is_zero(&hdr->b_refcnt); hdr->b_flags &= ~ARC_IO_IN_PROGRESS; @@ -3440,6 +3478,7 @@ arc_write_done(zio_t *zio) arc_hdr_destroy(hdr); } else { hdr->b_flags &= ~ARC_IO_IN_PROGRESS; + arc_brelvp(hdr); } hdr->b_flags &= ~ARC_STORED; From owner-svn-src-user@FreeBSD.ORG Thu Nov 19 05:03:38 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1671B1065672; Thu, 19 Nov 2009 05:03:38 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id E0D358FC15; Thu, 19 Nov 2009 05:03:37 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAJ53bod094277; Thu, 19 Nov 2009 05:03:37 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAJ53bEi094275; Thu, 19 Nov 2009 05:03:37 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911190503.nAJ53bEi094275@svn.freebsd.org> From: Kip Macy Date: Thu, 19 Nov 2009 05:03:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199506 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2009 05:03:38 -0000 Author: kmacy Date: Thu Nov 19 05:03:37 2009 New Revision: 199506 URL: http://svn.freebsd.org/changeset/base/199506 Log: - simplify arc_brelvp to always invalidate - call at the start of arc_write_done Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Nov 19 04:47:43 2009 (r199505) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Nov 19 05:03:37 2009 (r199506) @@ -1322,14 +1322,8 @@ arc_brelvp(arc_buf_hdr_t *hdr) */ BUF_LOCK(bp, LK_EXCLUSIVE | LK_INTERLOCK, BO_MTX(bo)); bremfree(bp); - /* - * buffer is not valid or is older - */ - if (((bp->b_flags & (B_CACHE|B_INVAL)) != B_CACHE) || - (bp->b_birth <= hdr->b_birth)) { - bp->b_flags |= B_INVAL; - bp->b_birth = 0; - } + bp->b_flags |= B_INVAL; + bp->b_birth = 0; brelse(bp); } else BO_UNLOCK(bo); @@ -3411,6 +3405,8 @@ arc_write_done(zio_t *zio) hdr->b_dva = *BP_IDENTITY(zio->io_bp); hdr->b_birth = zio->io_bp->blk_birth; hdr->b_cksum0 = zio->io_bp->blk_cksum.zc_word[0]; + arc_brelvp(hdr); + /* * If the block to be written was all-zero, we may have * compressed it away. In this case no write was performed @@ -3452,8 +3448,6 @@ arc_write_done(zio_t *zio) exists = buf_hash_insert(hdr, &hash_lock); ASSERT3P(exists, ==, NULL); } else if (buf->b_bp != NULL) { - arc_brelvp(hdr); - buf->b_bp->b_flags |= B_CACHE; buf->b_bp->b_flags &= ~B_INVAL; } @@ -3469,7 +3463,6 @@ arc_write_done(zio_t *zio) * This is an anonymous buffer with no user callback, * destroy it if there are no active references. */ - arc_brelvp(hdr); mutex_enter(&arc_eviction_mtx); destroy_hdr = refcount_is_zero(&hdr->b_refcnt); hdr->b_flags &= ~ARC_IO_IN_PROGRESS; @@ -3478,7 +3471,6 @@ arc_write_done(zio_t *zio) arc_hdr_destroy(hdr); } else { hdr->b_flags &= ~ARC_IO_IN_PROGRESS; - arc_brelvp(hdr); } hdr->b_flags &= ~ARC_STORED; From owner-svn-src-user@FreeBSD.ORG Thu Nov 19 07:01:11 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 05F59106568F; Thu, 19 Nov 2009 07:01:10 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 015658FC0A; Thu, 19 Nov 2009 07:01:10 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAJ7194u096735; Thu, 19 Nov 2009 07:01:09 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAJ719fg096733; Thu, 19 Nov 2009 07:01:09 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911190701.nAJ719fg096733@svn.freebsd.org> From: Kip Macy Date: Thu, 19 Nov 2009 07:01:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199509 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2009 07:01:11 -0000 Author: kmacy Date: Thu Nov 19 07:01:09 2009 New Revision: 199509 URL: http://svn.freebsd.org/changeset/base/199509 Log: invalidate a cached read Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Nov 19 05:59:15 2009 (r199508) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Nov 19 07:01:09 2009 (r199509) @@ -2673,6 +2673,7 @@ arc_read_done(zio_t *zio) buf = zio->io_private; hdr = buf->b_hdr; + arc_brelvp(hdr); /* * The hdr was inserted into hash-table and removed from lists * prior to starting I/O. We should find this header, since From owner-svn-src-user@FreeBSD.ORG Thu Nov 19 21:48:01 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78C05106566C; Thu, 19 Nov 2009 21:48:01 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 61E138FC08; Thu, 19 Nov 2009 21:48:01 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAJLm1kA020637; Thu, 19 Nov 2009 21:48:01 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAJLm103020633; Thu, 19 Nov 2009 21:48:01 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911192148.nAJLm103020633@svn.freebsd.org> From: Kip Macy Date: Thu, 19 Nov 2009 21:48:01 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199555 - in user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2009 21:48:01 -0000 Author: kmacy Date: Thu Nov 19 21:48:01 2009 New Revision: 199555 URL: http://svn.freebsd.org/changeset/base/199555 Log: move buffer invalidation in to zio write functions Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/arc.h user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Nov 19 21:47:54 2009 (r199554) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu Nov 19 21:48:01 2009 (r199555) @@ -1300,18 +1300,18 @@ arc_buf_add_ref(arc_buf_t *buf, void* ta data, metadata, hits); } -static void -arc_brelvp(arc_buf_hdr_t *hdr) +void +arc_binval(spa_t *spa, dva_t *dva) { - uint64_t blkno = hdr->b_dva.dva_word[1] & ~(1UL<<63); - struct vnode *vp = spa_get_vnode(hdr->b_spa); + uint64_t blkno = dva->dva_word[1] & ~(1UL<<63); + struct vnode *vp = spa_get_vnode(spa); struct bufobj *bo = &vp->v_bufobj; struct buf *bp; if (zfs_page_cache_disable) return; - if (blkno == 0 || hdr->b_birth == 0) + if (blkno == 0) return; BO_LOCK(bo); @@ -2673,7 +2673,6 @@ arc_read_done(zio_t *zio) buf = zio->io_private; hdr = buf->b_hdr; - arc_brelvp(hdr); /* * The hdr was inserted into hash-table and removed from lists * prior to starting I/O. We should find this header, since @@ -3406,7 +3405,6 @@ arc_write_done(zio_t *zio) hdr->b_dva = *BP_IDENTITY(zio->io_bp); hdr->b_birth = zio->io_bp->blk_birth; hdr->b_cksum0 = zio->io_bp->blk_cksum.zc_word[0]; - arc_brelvp(hdr); /* * If the block to be written was all-zero, we may have Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/arc.h ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/arc.h Thu Nov 19 21:47:54 2009 (r199554) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/arc.h Thu Nov 19 21:48:01 2009 (r199555) @@ -112,6 +112,7 @@ int arc_tryread(spa_t *spa, blkptr_t *bp void arc_set_callback(arc_buf_t *buf, arc_evict_func_t *func, void *private); int arc_buf_evict(arc_buf_t *buf); +void arc_binval(spa_t *spa, dva_t *dva); void arc_flush(spa_t *spa); void arc_tempreserve_clear(uint64_t reserve); int arc_tempreserve_space(uint64_t reserve, uint64_t txg); Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Thu Nov 19 21:47:54 2009 (r199554) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Thu Nov 19 21:48:01 2009 (r199555) @@ -25,6 +25,7 @@ #include #include +#include #include #include #include @@ -548,6 +549,7 @@ zio_write(zio_t *pio, spa_t *spa, uint64 zp->zp_ndvas <= spa_max_replication(spa)); ASSERT(ready != NULL); + arc_binval(spa, BP_IDENTITY(bp)); zio = zio_create(pio, spa, txg, bp, data, size, done, private, ZIO_TYPE_WRITE, priority, flags, NULL, 0, zb, ZIO_STAGE_OPEN, ZIO_WRITE_PIPELINE); @@ -565,6 +567,7 @@ zio_rewrite(zio_t *pio, spa_t *spa, uint { zio_t *zio; + arc_binval(spa, BP_IDENTITY(bp)); zio = zio_create(pio, spa, txg, bp, data, size, done, private, ZIO_TYPE_WRITE, priority, flags, NULL, 0, zb, ZIO_STAGE_OPEN, ZIO_REWRITE_PIPELINE); From owner-svn-src-user@FreeBSD.ORG Fri Nov 20 23:14:09 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 312DD1065670; Fri, 20 Nov 2009 23:14:09 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 0562D8FC18; Fri, 20 Nov 2009 23:14:09 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAKNE8Tf059222; Fri, 20 Nov 2009 23:14:08 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAKNE8f8059220; Fri, 20 Nov 2009 23:14:08 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911202314.nAKNE8f8059220@svn.freebsd.org> From: Kip Macy Date: Fri, 20 Nov 2009 23:14:08 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199618 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2009 23:14:09 -0000 Author: kmacy Date: Fri Nov 20 23:14:08 2009 New Revision: 199618 URL: http://svn.freebsd.org/changeset/base/199618 Log: - call in to zio pipeline even in the case of a page cache hit, but bypass the I/O stages Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Fri Nov 20 22:22:53 2009 (r199617) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Fri Nov 20 23:14:08 2009 (r199618) @@ -1303,17 +1303,21 @@ arc_buf_add_ref(arc_buf_t *buf, void* ta void arc_binval(spa_t *spa, dva_t *dva) { - uint64_t blkno = dva->dva_word[1] & ~(1UL<<63); - struct vnode *vp = spa_get_vnode(spa); - struct bufobj *bo = &vp->v_bufobj; + uint64_t blkno; + struct vnode *vp; + struct bufobj *bo; struct buf *bp; if (zfs_page_cache_disable) return; - if (blkno == 0) + if (dva == NULL || spa == NULL || blkno == 0) return; + blkno = dva->dva_word[1] & ~(1UL<<63); + vp = spa_get_vnode(spa); + bo = &vp->v_bufobj; + BO_LOCK(bo); bp = gbincore(bo, blkno); if (bp != NULL) { @@ -1408,7 +1412,7 @@ arc_getblk(arc_buf_t *buf) bo = &vp->v_bufobj; newbp = NULL; - if ((size < PAGE_SIZE)) { + if (size < PAGE_SIZE) { data = zio_buf_alloc(size); } else if ((buf->b_hdr->b_flags & ARC_BUF_CLONING) || BUF_EMPTY(buf->b_hdr)) { @@ -2961,21 +2965,6 @@ top: ASSERT(hdr->b_datacnt == 0); hdr->b_datacnt = 1; } - /* - * We hit in the page cache - * - */ - if (buf->b_bp != NULL) { - if ((buf->b_bp->b_flags & (B_CACHE|B_INVAL)) == B_CACHE) { - /* - * track the number of times - * the buffer was found in the cache - */ - ARCSTAT_BUMP(arcstat_page_cache_hits); - mutex_exit(hash_lock); - goto top; - } - } acb = kmem_zalloc(sizeof (arc_callback_t), KM_SLEEP); acb->acb_done = done; @@ -3077,6 +3066,20 @@ top: rzio = zio_read(pio, spa, bp, buf->b_data, size, arc_read_done, buf, priority, zio_flags, zb); + /* + * We hit in the page cache + * + */ + if ((buf->b_bp != NULL) && + ((buf->b_bp->b_flags & (B_CACHE|B_INVAL)) == B_CACHE)) { + /* + * track the number of times + * the buffer was found in the cache + */ + ARCSTAT_BUMP(arcstat_page_cache_hits); + rzio->io_pipeline = ZIO_INTERLOCK_STAGES; + } + if (*arc_flags & ARC_WAIT) return (zio_wait(rzio)); From owner-svn-src-user@FreeBSD.ORG Sat Nov 21 00:34:51 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 991381065670; Sat, 21 Nov 2009 00:34:51 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 86D048FC14; Sat, 21 Nov 2009 00:34:51 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAL0YpUe060877; Sat, 21 Nov 2009 00:34:51 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAL0Yp9u060875; Sat, 21 Nov 2009 00:34:51 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911210034.nAL0Yp9u060875@svn.freebsd.org> From: Kip Macy Date: Sat, 21 Nov 2009 00:34:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199620 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Nov 2009 00:34:51 -0000 Author: kmacy Date: Sat Nov 21 00:34:51 2009 New Revision: 199620 URL: http://svn.freebsd.org/changeset/base/199620 Log: - eliminate gratuitous white space changes with pre-buf support - allow arc to recycle sub-PAGE_SIZE buffers - validate stolen against data in recycle Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sat Nov 21 00:19:09 2009 (r199619) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sat Nov 21 00:34:51 2009 (r199620) @@ -453,7 +453,6 @@ struct arc_write_callback { arc_buf_t *awcb_buf; }; - struct arc_buf_hdr { /* protected by hash lock */ dva_t b_dva; @@ -498,7 +497,7 @@ static void arc_access(arc_buf_hdr_t *bu static int arc_evict_needed(arc_buf_contents_t type); static void arc_evict_ghost(arc_state_t *state, spa_t *spa, int64_t bytes); -#define GHOST_STATE(state) \ +#define GHOST_STATE(state) \ ((state) == arc_mru_ghost || (state) == arc_mfu_ghost || \ (state) == arc_l2c_only) @@ -1704,7 +1703,7 @@ arc_buf_size(arc_buf_t *buf) * it can't get a hash_lock on, and so may not catch all candidates. * It may also return without evicting as much space as requested. */ -static struct buf * +static void * arc_evict(arc_state_t *state, spa_t *spa, int64_t bytes, boolean_t recycle, arc_buf_contents_t type) { @@ -1716,18 +1715,16 @@ arc_evict(arc_state_t *state, spa_t *spa kmutex_t *lock, *evicted_lock; kmutex_t *hash_lock; boolean_t have_lock; - struct buf *stolen = NULL; + void *stolen = NULL; static int evict_metadata_offset, evict_data_offset; int i, idx, offset, list_count, count; ASSERT(state == arc_mru || state == arc_mfu); - ASSERT(recycle == FALSE); evicted_state = (state == arc_mru) ? arc_mru_ghost : arc_mfu_ghost; if (type == ARC_BUFC_METADATA) { offset = 0; - list_count = ARC_BUFC_NUMMETADATALISTS; list_start = &state->arcs_lists[0]; evicted_list_start = &evicted_state->arcs_lists[0]; @@ -1792,7 +1789,7 @@ evict_start: if (buf->b_efunc) { mutex_enter(&arc_eviction_mtx); arc_buf_destroy(buf, - buf->b_bp == stolen, FALSE); + buf->b_data == stolen, FALSE); ab->b_buf = buf->b_next; buf->b_hdr = &arc_eviction_hdr; buf->b_next = arc_eviction_list; @@ -1802,7 +1799,7 @@ evict_start: } else { rw_exit(&buf->b_lock); arc_buf_destroy(buf, - buf->b_bp == stolen, TRUE); + buf->b_data == stolen, TRUE); } } if (ab->b_datacnt == 0) { @@ -2486,9 +2483,11 @@ arc_get_data_buf(arc_buf_t *buf) state = (arc_mru->arcs_lsize[type] > 0 && mfu_space > arc_mfu->arcs_size) ? arc_mru : arc_mfu; } - (void) arc_evict(state, NULL, size, FALSE, type); - arc_getblk(buf); - ASSERT(buf->b_data != NULL); + if ((buf->b_data = arc_evict(state, NULL, size, (size < PAGE_SIZE), type)) == NULL) { + arc_getblk(buf); + ASSERT(buf->b_data != NULL); + } + out: /* * Update the state size. Note that ghost states have a @@ -2737,7 +2736,7 @@ arc_read_done(zio_t *zio) buf->b_bp->b_flags |= B_CACHE; buf->b_bp->b_flags &= ~B_INVAL; } - + /* * Broadcast before we drop the hash_lock to avoid the possibility * that the hdr (and hence the cv) might be freed before we get to @@ -2839,9 +2838,11 @@ arc_read_nolock(zio_t *pio, spa_t *spa, top: hdr = buf_hash_find(spa, BP_IDENTITY(bp), bp->blk_birth, &hash_lock); if (hdr && hdr->b_datacnt > 0) { + *arc_flags |= ARC_CACHED; if (HDR_IO_IN_PROGRESS(hdr)) { + if (*arc_flags & ARC_WAIT) { cv_wait(&hdr->b_cv, hash_lock); mutex_exit(hash_lock); @@ -2915,9 +2916,7 @@ top: /* this block is not in the cache */ arc_buf_hdr_t *exists; arc_buf_contents_t type = BP_GET_BUFC_TYPE(bp); - - buf = _arc_buf_alloc(spa, size, private, type, - bp); + buf = _arc_buf_alloc(spa, size, private, type, bp); hdr = buf->b_hdr; hdr->b_cksum0 = bp->blk_cksum.zc_word[0]; exists = buf_hash_insert(hdr, &hash_lock); @@ -2964,6 +2963,7 @@ top: arc_get_data_buf(buf); ASSERT(hdr->b_datacnt == 0); hdr->b_datacnt = 1; + } acb = kmem_zalloc(sizeof (arc_callback_t), KM_SLEEP); @@ -3067,7 +3067,7 @@ top: arc_read_done, buf, priority, zio_flags, zb); /* - * We hit in the page cache + * We hit in the page cache - can bypass the I/O stages * */ if ((buf->b_bp != NULL) && @@ -3405,10 +3405,10 @@ arc_write_done(zio_t *zio) arc_buf_hdr_t *hdr = buf->b_hdr; hdr->b_acb = NULL; + hdr->b_dva = *BP_IDENTITY(zio->io_bp); hdr->b_birth = zio->io_bp->blk_birth; hdr->b_cksum0 = zio->io_bp->blk_cksum.zc_word[0]; - /* * If the block to be written was all-zero, we may have * compressed it away. In this case no write was performed @@ -3418,15 +3418,6 @@ arc_write_done(zio_t *zio) if (!BUF_EMPTY(hdr)) { arc_buf_hdr_t *exists; kmutex_t *hash_lock; - /* - * Associate buffer with offset in the page cache - */ - struct buf *bp = buf->b_bp; - struct vnode *vp = spa_get_vnode(hdr->b_spa); - uint64_t blkno = hdr->b_dva.dva_word[1] & ~(1UL<<63); - - CTR3(KTR_SPARE2, "arc_write_done() bp=%p flags %X blkno %ld", - bp, bp ? bp->b_flags : 0, blkno); arc_cksum_verify(buf); @@ -3453,7 +3444,6 @@ arc_write_done(zio_t *zio) buf->b_bp->b_flags |= B_CACHE; buf->b_bp->b_flags &= ~B_INVAL; } - hdr->b_flags &= ~ARC_IO_IN_PROGRESS; /* if it's not anon, we are doing a scrub */ if (hdr->b_state == arc_anon) @@ -3920,8 +3910,8 @@ arc_init(void) TS_RUN, minclsyspri); #ifdef _KERNEL - arc_event_lowmem = EVENTHANDLER_REGISTER(vm_lowmem, arc_lowmem, - NULL, EVENTHANDLER_PRI_FIRST); + arc_event_lowmem = EVENTHANDLER_REGISTER(vm_lowmem, arc_lowmem, NULL, + EVENTHANDLER_PRI_FIRST); arc_event_shutdown = EVENTHANDLER_REGISTER(shutdown_pre_sync, arc_shutdown, NULL, EVENTHANDLER_PRI_FIRST); #endif From owner-svn-src-user@FreeBSD.ORG Sat Nov 21 01:54:33 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85FBC106566C; Sat, 21 Nov 2009 01:54:33 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 5AFB68FC13; Sat, 21 Nov 2009 01:54:33 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAL1sXBx062510; Sat, 21 Nov 2009 01:54:33 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAL1sXon062508; Sat, 21 Nov 2009 01:54:33 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911210154.nAL1sXon062508@svn.freebsd.org> From: Kip Macy Date: Sat, 21 Nov 2009 01:54:33 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199622 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Nov 2009 01:54:33 -0000 Author: kmacy Date: Sat Nov 21 01:54:32 2009 New Revision: 199622 URL: http://svn.freebsd.org/changeset/base/199622 Log: - enforce size limit in arc_evict itself - mark new buffer as B_VMIO so that they are maintained in the page cache Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sat Nov 21 01:43:22 2009 (r199621) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sat Nov 21 01:54:32 2009 (r199622) @@ -1419,10 +1419,11 @@ arc_getblk(arc_buf_t *buf) data = newbp->b_data; buf->b_hdr->b_flags &= ~ARC_BUF_CLONING; } else { - newbp = getblk(vp, blkno, size, 0, 0, flags); - if (newbp->b_birth != buf->b_hdr->b_birth) - newbp->b_flags |= B_INVAL; - brelvp(newbp); + newbp = getblk(vp, blkno, size, 0, 0, flags | GB_NOCREAT); + if (newbp == NULL) + newbp = geteblk(size, flags); + else + brelvp(newbp); data = newbp->b_data; } @@ -1435,8 +1436,10 @@ arc_getblk(arc_buf_t *buf) newbp, newbp->b_flags); #endif - if (newbp != NULL) + if (newbp != NULL) { BUF_KERNPROC(newbp); + newbp->b_flags |= B_VMIO; + } buf->b_bp = newbp; buf->b_data = data; } @@ -1722,7 +1725,13 @@ arc_evict(arc_state_t *state, spa_t *spa ASSERT(state == arc_mru || state == arc_mfu); evicted_state = (state == arc_mru) ? arc_mru_ghost : arc_mfu_ghost; - + + /* + * don't recycle page cache bufs + * + */ + if (recycle && (size >= PAGE_SIZE)) + recycle = FALSE; if (type == ARC_BUFC_METADATA) { offset = 0; list_count = ARC_BUFC_NUMMETADATALISTS; @@ -2483,7 +2492,7 @@ arc_get_data_buf(arc_buf_t *buf) state = (arc_mru->arcs_lsize[type] > 0 && mfu_space > arc_mfu->arcs_size) ? arc_mru : arc_mfu; } - if ((buf->b_data = arc_evict(state, NULL, size, (size < PAGE_SIZE), type)) == NULL) { + if ((buf->b_data = arc_evict(state, NULL, size, TRUE, type)) == NULL) { arc_getblk(buf); ASSERT(buf->b_data != NULL); } From owner-svn-src-user@FreeBSD.ORG Sat Nov 21 01:57:19 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E0992106566B; Sat, 21 Nov 2009 01:57:19 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id CF85A8FC08; Sat, 21 Nov 2009 01:57:19 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAL1vJpb062608; Sat, 21 Nov 2009 01:57:19 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAL1vJZN062606; Sat, 21 Nov 2009 01:57:19 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911210157.nAL1vJZN062606@svn.freebsd.org> From: Kip Macy Date: Sat, 21 Nov 2009 01:57:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199623 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Nov 2009 01:57:20 -0000 Author: kmacy Date: Sat Nov 21 01:57:19 2009 New Revision: 199623 URL: http://svn.freebsd.org/changeset/base/199623 Log: fix typo Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sat Nov 21 01:54:32 2009 (r199622) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sat Nov 21 01:57:19 2009 (r199623) @@ -1730,7 +1730,7 @@ arc_evict(arc_state_t *state, spa_t *spa * don't recycle page cache bufs * */ - if (recycle && (size >= PAGE_SIZE)) + if (recycle && (bytes >= PAGE_SIZE)) recycle = FALSE; if (type == ARC_BUFC_METADATA) { offset = 0; From owner-svn-src-user@FreeBSD.ORG Sat Nov 21 02:09:22 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7826B1065672; Sat, 21 Nov 2009 02:09:22 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 670B78FC1F; Sat, 21 Nov 2009 02:09:22 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nAL29MG8062894; Sat, 21 Nov 2009 02:09:22 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nAL29M3j062892; Sat, 21 Nov 2009 02:09:22 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200911210209.nAL29M3j062892@svn.freebsd.org> From: Kip Macy Date: Sat, 21 Nov 2009 02:09:22 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199624 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Nov 2009 02:09:22 -0000 Author: kmacy Date: Sat Nov 21 02:09:21 2009 New Revision: 199624 URL: http://svn.freebsd.org/changeset/base/199624 Log: revert B_VMIO change Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sat Nov 21 01:57:19 2009 (r199623) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sat Nov 21 02:09:21 2009 (r199624) @@ -1436,10 +1436,8 @@ arc_getblk(arc_buf_t *buf) newbp, newbp->b_flags); #endif - if (newbp != NULL) { + if (newbp != NULL) BUF_KERNPROC(newbp); - newbp->b_flags |= B_VMIO; - } buf->b_bp = newbp; buf->b_data = data; } From owner-svn-src-user@FreeBSD.ORG Sat Nov 21 17:11:52 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42E3F106566B; Sat, 21 Nov 2009 17:11:52 +0000 (UTC) (envelope-from ed@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 19B578FC1C; Sat, 21 Nov 2009 17:11:52 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id nALHBpAi085837; Sat, 21 Nov 2009 17:11:51 GMT (envelope-from ed@svn.freebsd.org) Received: (from ed@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id nALHBpa1085833; Sat, 21 Nov 2009 17:11:51 GMT (envelope-from ed@svn.freebsd.org) Message-Id: <200911211711.nALHBpa1085833@svn.freebsd.org> From: Ed Schouten Date: Sat, 21 Nov 2009 17:11:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r199640 - user/ed/newcons/sys/dev/vt X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Nov 2009 17:11:52 -0000 Author: ed Date: Sat Nov 21 17:11:51 2009 New Revision: 199640 URL: http://svn.freebsd.org/changeset/base/199640 Log: Tidy up this code a bit. Add VBM_DIRTY that we can use throughout the code to serve as a dirty mask for rows/columns. Modified: user/ed/newcons/sys/dev/vt/vt.h user/ed/newcons/sys/dev/vt/vt_buf.c user/ed/newcons/sys/dev/vt/vt_core.c Modified: user/ed/newcons/sys/dev/vt/vt.h ============================================================================== --- user/ed/newcons/sys/dev/vt/vt.h Sat Nov 21 17:01:38 2009 (r199639) +++ user/ed/newcons/sys/dev/vt/vt.h Sat Nov 21 17:11:51 2009 (r199640) @@ -99,6 +99,7 @@ struct vt_device { struct vt_bufmask { uint64_t vbm_row, vbm_col; +#define VBM_DIRTY UINT64_MAX }; struct vt_buf { Modified: user/ed/newcons/sys/dev/vt/vt_buf.c ============================================================================== --- user/ed/newcons/sys/dev/vt/vt_buf.c Sat Nov 21 17:01:38 2009 (r199639) +++ user/ed/newcons/sys/dev/vt/vt_buf.c Sat Nov 21 17:11:51 2009 (r199640) @@ -44,8 +44,6 @@ static MALLOC_DEFINE(M_VTBUF, "vtbuf", " #define VTBUF_LOCK(vb) mtx_lock_spin(&(vb)->vb_lock) #define VTBUF_UNLOCK(vb) mtx_unlock_spin(&(vb)->vb_lock) -#define ALLBITS ((uint64_t)~0) - static inline uint64_t vtbuf_dirty_axis(unsigned int begin, unsigned int end) { @@ -61,12 +59,12 @@ vtbuf_dirty_axis(unsigned int begin, uns /* Obvious case. Mark everything dirty. */ if (end - begin >= 64) - return (ALLBITS); + return (VBM_DIRTY); /* 1....0; used bits on the left. */ - left = ALLBITS << begin % 64; + left = VBM_DIRTY << begin % 64; /* 0....1; used bits on the right. */ - right = ALLBITS >> -end % 64; + right = VBM_DIRTY >> -end % 64; /* * Only take the intersection. If the result of that is 0, it Modified: user/ed/newcons/sys/dev/vt/vt_core.c ============================================================================== --- user/ed/newcons/sys/dev/vt/vt_core.c Sat Nov 21 17:01:38 2009 (r199639) +++ user/ed/newcons/sys/dev/vt/vt_core.c Sat Nov 21 17:11:51 2009 (r199640) @@ -505,7 +505,7 @@ vt_flush(struct vt_device *vd) if (vd->vd_scrollpos != scrollpos || vd->vd_flags & VDF_INVALID) { tarea.tr_begin.tp_row = tarea.tr_begin.tp_col = 0; tarea.tr_end = size; - tmask.vbm_row = tmask.vbm_col = ~0; + tmask.vbm_row = tmask.vbm_col = VBM_DIRTY; /* * Blank to prevent borders with artifacts. This is