From owner-freebsd-java@FreeBSD.ORG  Thu May 20 14:05:21 2004
Return-Path: <owner-freebsd-java@FreeBSD.ORG>
Delivered-To: freebsd-java@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id AA68316A4CE
	for <freebsd-java@freebsd.org>; Thu, 20 May 2004 14:05:21 -0700 (PDT)
Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 8BF6343D1F
	for <freebsd-java@freebsd.org>; Thu, 20 May 2004 14:05:21 -0700 (PDT)
	(envelope-from sk@pressevent.com)
Received: from komarovsbc91
	(pcp04745553pcs.bowie01.md.comcast.net[68.50.32.109])
	by comcast.net (rwcrmhc13) with SMTP
	id <2004052021052101500q5r31e>; Thu, 20 May 2004 21:05:21 +0000
From: "Sergei Komarov" <sk@pressevent.com>
To: <freebsd-java@freebsd.org>
Date: Thu, 20 May 2004 17:05:20 -0400
Message-ID: <009301c43eae$296f7b20$1401a8c0@na.corp.yr.com>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook, Build 10.0.2627
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409
Importance: Normal
Subject: JDK142_p6 - non-ascii encoding problem?
X-BeenThere: freebsd-java@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: Porting Java to FreeBSD <freebsd-java.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-java>,
	<mailto:freebsd-java-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-java>
List-Post: <mailto:freebsd-java@freebsd.org>
List-Help: <mailto:freebsd-java-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-java>,
	<mailto:freebsd-java-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 20 May 2004 21:05:21 -0000

Hello,

I'm running FreeBSD 4.9-STABLE, jdk1.4.2_p6 and Tomcat 4.1.30 or 5.0.24
and keep having a problem with non-Latin1 characters output by JSPs.
This problem seems to be specific to FreeBSD as it doesn't occur on
Windows (I'm testing Solaris as well).

A JSP receives a request attribute containing a string of non-Latin1
characters (Cyrillic Windows, Simplified Chinese gb2312, etc.)

When the string is output within a page via c:out, struts bean:write or
<%=name%>, JspWriter converts all non-Latin1 chars to '?'. 

I can correct it with either of the following tricks, but the
application is too big to replicate them in all JSPs:
<%@page language="java"
        contentType="text/html;charset=gb2312"
        import= "java.nio.charset.*,
                 java.nio.CharBuffer,
                 java.nio.ByteBuffer"
%>
<%
String orig = (String) request.getAttribute("original");
String encoded =  new String(orig.getBytes("ISO-8859-1"),"gb2312");
%>
...
<%= encoded %>
...
<%
Charset gb2312 = Charset.forName("gb2312");
Charset iso = Charset.forName("ISO-8859-1");
CharBuffer cb = CharBuffer.wrap(orig);
ByteBuffer bb = iso.encode(cb);
%>
...
<%= gb2312.decode(bb) %>

Am I missing something in the configuration of FreeBSD and JDK? 

I tried setting 
LANGUAGE          zh_CN
LANG              zh_CN.gb2312
LC_TYPE           zh_CN
LC_ALL            zh_CN
MM_CHARSET        gb2312

In /etc/profile etc., but to no avail. Perhaps someone knows the
solution to this?

Thanks in advance,

Sergei Komarov