From owner-freebsd-java@FreeBSD.ORG  Tue May 25 10:17:04 2004
Return-Path: <owner-freebsd-java@FreeBSD.ORG>
Delivered-To: freebsd-java@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 102B316A4CE
	for <freebsd-java@freebsd.org>; Tue, 25 May 2004 10:17:04 -0700 (PDT)
Received: from rwcrmhc12.comcast.net (rwcrmhc12.comcast.net [216.148.227.85])
	by mx1.FreeBSD.org (Postfix) with ESMTP id C98F043D3F
	for <freebsd-java@freebsd.org>; Tue, 25 May 2004 10:17:03 -0700 (PDT)
	(envelope-from sk@pressevent.com)
Received: from bursononxhwpz9
	(pcp04745553pcs.bowie01.md.comcast.net[68.50.32.109])
	by comcast.net (rwcrmhc12) with SMTP
	id <2004052517163401400834cje>; Tue, 25 May 2004 17:16:35 +0000
From: "Sergei Komarov" <sk@pressevent.com>
To: "'Panagiotis Astithas'" <past@noc.ntua.gr>
Date: Tue, 25 May 2004 13:16:33 -0400
Message-ID: <011701c4427c$07985c20$c2289298@bursononxhwpz9>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook, Build 10.0.2627
X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2800.1409
In-Reply-To: <40ADA7AC.20407@noc.ntua.gr>
Importance: Normal
cc: freebsd-java@freebsd.org
Subject: RE: JDK142_p6 - non-ascii encoding problem?
X-BeenThere: freebsd-java@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: Porting Java to FreeBSD <freebsd-java.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-java>,
	<mailto:freebsd-java-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-java>
List-Post: <mailto:freebsd-java@freebsd.org>
List-Help: <mailto:freebsd-java-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-java>,
	<mailto:freebsd-java-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 25 May 2004 17:17:04 -0000

Thank you for your suggestions. It turns out that the issue is not
FreeBSD-specific: I was able to replicate it on Solaris. In the end, I
opted for implementing the transcoding in a custom taglib and upgrading
the app.

Thank you,


Sergei Komarov 


-----Original Message-----
From: Panagiotis Astithas [mailto:past@noc.ntua.gr] 
Sent: Friday, May 21, 2004 2:55 AM
To: Sergei Komarov
Cc: freebsd-java@freebsd.org
Subject: Re: JDK142_p6 - non-ascii encoding problem?


Sergei Komarov wrote:
> Hello,
> 
> I'm running FreeBSD 4.9-STABLE, jdk1.4.2_p6 and Tomcat 4.1.30 or 
> 5.0.24 and keep having a problem with non-Latin1 characters output by 
> JSPs. This problem seems to be specific to FreeBSD as it doesn't occur

> on Windows (I'm testing Solaris as well).
> 
> A JSP receives a request attribute containing a string of non-Latin1 
> characters (Cyrillic Windows, Simplified Chinese gb2312, etc.)
> 
> When the string is output within a page via c:out, struts bean:write 
> or <%=name%>, JspWriter converts all non-Latin1 chars to '?'.
> 
> I can correct it with either of the following tricks, but the 
> application is too big to replicate them in all JSPs: <%@page 
> language="java"
>         contentType="text/html;charset=gb2312"
>         import= "java.nio.charset.*,
>                  java.nio.CharBuffer,
>                  java.nio.ByteBuffer"
> %>
> <%
> String orig = (String) request.getAttribute("original"); String 
> encoded =  new String(orig.getBytes("ISO-8859-1"),"gb2312");
> %>
> ...
> <%= encoded %>
> ...
> <%
> Charset gb2312 = Charset.forName("gb2312");
> Charset iso = Charset.forName("ISO-8859-1");
> CharBuffer cb = CharBuffer.wrap(orig);
> ByteBuffer bb = iso.encode(cb);
> %>
> ...
> <%= gb2312.decode(bb) %>
> 
> Am I missing something in the configuration of FreeBSD and JDK?
> 
> I tried setting 
> LANGUAGE          zh_CN
> LANG              zh_CN.gb2312
> LC_TYPE           zh_CN
> LC_ALL            zh_CN
> MM_CHARSET        gb2312
> 
> In /etc/profile etc., but to no avail. Perhaps someone knows the 
> solution to this?

You could try adding the following to the default class in
/etc/login.conf:
         :charset=zh_CN.GB2312:\
         :lang=zh_CN.GB2312:\
and then doing "cap_mkdb /etc/login.conf".

You could also have just mis-spelled the locale.

Cheers,
-- 
Panagiotis Astithas
Electrical & Computer Engineer, PhD
Network Management Center
National Technical University of Athens, Greece