Date: Mon, 25 Nov 2013 01:26:07 +0000 (UTC) From: Hiroki Sato <hrs@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r258537 - in head: include lib/libc/iconv lib/libiconv_modules/iconv_std Message-ID: <201311250126.rAP1Q7Vu006022@svn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: hrs Date: Mon Nov 25 01:26:06 2013 New Revision: 258537 URL: http://svnweb.freebsd.org/changeset/base/258537 Log: Add ICONV_{GET,SET}_ILSEQ_INVALID iconvctl. GNU iconv returns EILSEQ when there is an invalid character in the output codeset while it is valid in the input. However, POSIX requires iconv() to perform an implementation-defined conversion on the character. So, Citrus iconv converts such a character to a special character which means it is invalid in the output codeset. This is not a problem in most cases but some software like libxml2 depends on GNU's behavior to determine if a character is output as-is or another form such as a character entity (&#NNN;). Modified: head/include/iconv.h head/lib/libc/iconv/citrus_iconv_local.h head/lib/libc/iconv/iconv.c head/lib/libc/iconv/iconvctl.3 head/lib/libiconv_modules/iconv_std/citrus_iconv_std.c Modified: head/include/iconv.h ============================================================================== --- head/include/iconv.h Sun Nov 24 23:30:23 2013 (r258536) +++ head/include/iconv.h Mon Nov 25 01:26:06 2013 (r258537) @@ -86,6 +86,8 @@ void iconv_set_relocation_prefix(const #define ICONV_SET_DISCARD_ILSEQ 4 #define ICONV_SET_HOOKS 5 #define ICONV_SET_FALLBACKS 6 +#define ICONV_GET_ILSEQ_INVALID 128 +#define ICONV_SET_ILSEQ_INVALID 129 typedef void (*iconv_unicode_char_hook) (unsigned int mbr, void *data); typedef void (*iconv_wide_char_hook) (wchar_t wc, void *data); Modified: head/lib/libc/iconv/citrus_iconv_local.h ============================================================================== --- head/lib/libc/iconv/citrus_iconv_local.h Sun Nov 24 23:30:23 2013 (r258536) +++ head/lib/libc/iconv/citrus_iconv_local.h Mon Nov 25 01:26:06 2013 (r258537) @@ -99,6 +99,7 @@ struct _citrus_iconv_shared { char *ci_convname; bool ci_discard_ilseq; struct iconv_hooks *ci_hooks; + bool ci_ilseq_invalid; }; struct _citrus_iconv { Modified: head/lib/libc/iconv/iconv.c ============================================================================== --- head/lib/libc/iconv/iconv.c Sun Nov 24 23:30:23 2013 (r258536) +++ head/lib/libc/iconv/iconv.c Mon Nov 25 01:26:06 2013 (r258537) @@ -298,6 +298,12 @@ __bsd_iconvctl(iconv_t cd, int request, case ICONV_SET_FALLBACKS: errno = EOPNOTSUPP; return (-1); + case ICONV_GET_ILSEQ_INVALID: + *i = cv->cv_shared->ci_ilseq_invalid ? 1 : 0; + return (0); + case ICONV_SET_ILSEQ_INVALID: + cv->cv_shared->ci_ilseq_invalid = *i; + return (0); default: errno = EINVAL; return (-1); Modified: head/lib/libc/iconv/iconvctl.3 ============================================================================== --- head/lib/libc/iconv/iconvctl.3 Sun Nov 24 23:30:23 2013 (r258536) +++ head/lib/libc/iconv/iconvctl.3 Mon Nov 25 01:26:06 2013 (r258537) @@ -34,7 +34,7 @@ .\" .\" $FreeBSD$ .\" -.Dd November 7, 2009 +.Dd November 25, 2009 .Dt ICONVCTL 3 .Os .Sh NAME @@ -110,6 +110,31 @@ variable, which is passed to via .Fa argument by its address. +.It ICONV_GET_ILSEQ_INVALID +Determines if a character in the input buffer that is valid, +but for which an identical character does not exist in the target +codeset returns +.Er EILSEQ +or not. +The answer is stored in +.Fa argument , +which is of +.Ft int * . +It will be set to 1 if this feature is enabled or set to 0 otherwise. +.It ICONV_SET_ILSEQ_INVALID +Sets whether a character in the input buffer that is valid, +but for which an identical character does not exist in the target +codeset returns +.Er EILSEQ +or not. +If +.Fa argument , +which is of +.Ft int * +is set to 1 it will be enabled, +and if +.Fa argument +is set to 0 it will be disabled. .El .\" XXX: fallbacks are unimplemented and trying to set them will always .\" return EOPNOTSUPP but definitions are provided for source-level Modified: head/lib/libiconv_modules/iconv_std/citrus_iconv_std.c ============================================================================== --- head/lib/libiconv_modules/iconv_std/citrus_iconv_std.c Sun Nov 24 23:30:23 2013 (r258536) +++ head/lib/libiconv_modules/iconv_std/citrus_iconv_std.c Mon Nov 25 01:26:06 2013 (r258537) @@ -543,6 +543,16 @@ _citrus_iconv_std_iconv_convert(struct _ ret = do_conv(is, &csid, &idx); if (ret) { if (ret == E_NO_CORRESPONDING_CHAR) { + /* + * GNU iconv returns EILSEQ when no + * corresponding character in the output. + * Some software depends on this behavior + * though this is against POSIX specification. + */ + if (cv->cv_shared->ci_ilseq_invalid != 0) { + ret = EILSEQ; + goto err; + } inval++; szrout = 0; if ((((flags & _CITRUS_ICONV_F_HIDE_INVALID) == 0) &&
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201311250126.rAP1Q7Vu006022>