can you attach some sample messages? we don't have anyone that can do
japanese mail with dtmail so it's not gonna be easy for us to
reproduce. it will also help narrow down where the problem is
(probably is gtkhtml but ya never know).

Created attachment 41495 [details]
japanese mail file

okay, doesn't seem to be a mailer bug so probably a GtkHTML issue?

btw, doesn't crash for me but doen't display "correctly" eithger
afaict. Looks like all underlines, but then again - I am using an
iso-8859-1 font (don't have any japanese fonts installed).

If I copy/paste from Evolution's gtkhtml message display widget into
Mozilla which can render japanese characters encoded in UTF-8, it
displays correctly.

I guess this suggests that if I had japanese fonts for X that GtkHTML
would render this correctly with Evolution 1.1.x at least (dunno about
1.0.8)

Yeah, the issue in this case is just the font/rendering.  You either
need to choose a font encoding that supports the characters in the
message.  Underscores "_" typically mean the character code is outside
of the font encoding, little boxes ~"[]" typically mean the charset of
the font contains that character but the glyph is missing from the
font.  In both cases evolutio/gtkhtml correctly understand the message
but are unable to display it.

japanese PCK is not UTF-8 encoded. It is called Shift JIS/ Miscrosoft
kanji etc. and is the popular encoding in Japanese PCs. 
With 1.0.8 when as it was not crashing as as beta1, I was able to
input japanese characters in the mail search text box, but not in
compose mail. That means font is there but gtkHTML is nor finding it
or issue with processing the encoding, or it can be a iconv issue too

1) We know that the message is not utf-8 encoded.  Evolution decodes
the message internally into utf-8 and that is what gets passed to
gtkhtml.  All the internal processing is done in utf-8.

2) Gtkhtml uses different fonts than the input fields so the fact that
they respond differently is to be expected.

In 1.1.x you can change the gtkhtml font in evolution by going to
Tools->Prefernces then clicking on the font tab.  For gtkhtml-1.0.x
you have to use the gnome-1.4 control center and change them in the
HTML Viewer section then restart evolution. 

If 1.1 is crashing for you we would greatly appreciate stack traces.

*** bug 231311 has been marked as a duplicate of this bug. ***

I have Japanese fonts for X installed, and I still can't get Evo to
display Japanese mail.  In the past, I discovered that running Evo in
a Japanese Gnome session would do the trick, but ever since updating
my Gnome 2 with the Red Carpet snapshots on 9/16, Evo won't do the
trick, even in a Japanese Gnome session and even after launching it
from a term with 

$ env LC_ALL=ja_JP.eucJP evolution

Changing fonts with gtkHTML won't work either (I've tried), as the
encodings for Japanese fonts are too complicated.  gtkHTML insists
that you also choose the encoding, which bolloxes things up as
Japanese uses at least two different encodings, afaict -- one for the
Roman alphabet, one for Chinese ideographs, and one for the native
Japanese syllabaries.  Some font settings seem to lump the Roman and
syllabaries together, but any Japanese message is bound to have some
combination of all three.  Having to futz with font encodings to read
half the message simply isn't acceptible.  

Running Evo under a Japanese Gnome session (prior to the 9/16 Red
Carpet snapshots) seemed somehow to get around this problem by using
totally different fonts than the ones I can manually select using
gtkHTML, but sadly I am not (yet) a programmer and I don't know why
this was so.  

All of this seems to suggest that the problem is not with Evo but with
something in the underlying Gnome framework.  

The change you noticed around 9/16 is that we stoped loading fonts as
fontsets by default because it was breaking other things.  To get that
behavior back you will need to specify a font set by hand in the gconf
keys that gtkhtml uses, and there still may be a couple of issues, if
you'd like to test some changes to get this working let me know.

NOTE: These problems with display are all due to limitations in
gtk-1.2 when evolution moves to gtk2 (evolution-1.4) the display
issues will for the most part disappear (thank goodness).

*** bug 231917 has been marked as a duplicate of this bug. ***

Larry,

Is there a work around to see Japanese emails with evolution 1.1.2. I
used to be able to read Japanese emails with 1.0.8 before I upgraded.
I've played with the font settings under the "Settings" configuration
in evolution but no luck. I had been starting evolution with the
following command with 1.0.8.

oaf-slay ; LANG=en_US LC_CTYPE=ja_JP LC_MESSAGES=en_US
XMODIFIERS="@im=kinput2" evolution

Currently, I can see Japanese in the From and Subject lines area of
the message summary but it appears as ___ when viewing the mail in the
preview pane or opening a window.

I'm using Redhat 7.1 and Ximian Gnome 1.4.

I'd love to get this working again, as I use Japanese everyday.

Tom.

can anyone having trouble loading fonsets in 1.1.x please try the
patch attached to
http://bugzilla.gnome.org/show_bug.cgi?id=221208

Larry,

Don't suppose there is any other easier way to test this? I use Red
Carpet to grab evolution and don't really have the time to try patch
and build it all from scratch. I can you a Japanese email if you want
so that you can try it out. I can send you a screen grab of how it
should look like in the unlikely event that you don't read Japanese :)

Let me know.

Tom.

The patch is in RC1 and will be in 1.2, I'm still not sure it fixes
everything for everyone but it should improve things.

Hi Larry,

I'm using 1.1.90 now which is RC1. I'm still not able to see Japanese
properly though :-( As per above, I used to start Evolution using the
following:

oaf-slay ; LANG=en_US LC_CTYPE=ja_JP LC_MESSAGES=en_US
XMODIFIERS="@im=kinput2" evolution

And it used to display Japanese for me. It still does on the subject
lines in the message summary but not in the actual emails. I'm not
sure what font I used to use with 1.0.8. It used to just work out of
the box with the courier/helvetica pair.

Perhaps, I need to start evolution in another way now? Any
suggestions? Thanks!

Tom.

This japanese mail display issue in Solaris with evolution seem to get
fixed with the following hacks.
Haven't created a patch yet, but the following stuff seem to work

Summary of changes.
1. 
./gal/gal/util/e-iconv.c
Adding the following line to known_iconv_charsets[]
        { "pck","SJIS"       },
Seem to fix issues with the display of japanese in al gal areas,
similarly  

        { "5601",   "EUC-KR"     },
Seem to fix the issue http://bugzilla.gnome.org/show_bug.cgi?id=231081
. So all gal widget display issues need correct iconv aliases in the
file.

The iconv aliases for each locale will be returned by this program in
Solaris.

#include <locale.h>
#include <langinfo.h>
void main() {
        setlocale(LC_ALL,"");
        printf("codeset=%s\n", nl_langinfo(CODESET));
}

2. 
Loading the correct FontSet for multibyte fonts, seems like in gtk-1.2
multibyte font display does not work correctly for some fonts. The
check that is in place to test for fontset if the font name is
seperated by commas is not right as there can be a alias XLFD fontset
name for a bunch of fonts which is used in a locale. But forcing call
to gdk_fontset_load instead of the gdk_font_load in
gtkhtml/src/htmlgdkpainter.c made Japanese mail entry into compose
mail as well as mail display come OK. I changed the
load_font_with_name in that file to
gboolean need_fontset = TRUE;
instead of  the default value FALSE.
I know this will result in performance penalties in UTF-8 locales and
is not an acceptable solution, but the issue is
gdk_draw_string/gdk_draw_text functions in gtk+-1.2.10/gdk/gdkdraw.c
won't work correctly for some multibyte locales for font->type ==
GDK_FONT_FONT, which hopefully will get fixed with latest gtk+2.0 

Inshort

providing the right alias in ./gal/gal/util/e-iconv.c
providing the right fontset, and loading the fontset whenever needed
seems to solve most of the display/entry issues in other bugs too.

The following patch checks whether font/fontset is needed for current
fot selected. This also seem to work with aliased fontset names. This
rectifies display issues in japanese locale.

--- htmlgdkpainter.c.orig       2003-02-12 16:12:43.993026000 -0800
+++ htmlgdkpainter.c    2003-02-13 12:40:05.085976000 -0800
@@ -270,17 +270,24 @@
 load_font_with_name (gchar *name)
 {
        GdkFont *gdk_font;
-       gboolean need_fontset = FALSE;
        gpointer font;
        GTimer *timer;
+       gchar *defonts, **missingfonts, **fname_list;
+       gint count, i=0;
        gdouble elapsed;
-
-       if (strchr (name, ','))
-               need_fontset = TRUE;
+       XFontSet fset;
+       XFontStruct  **fstruct_list;

        timer = g_timer_new ();
        g_timer_start (timer);
-       gdk_font = need_fontset ? gdk_fontset_load (name) :
gdk_font_load (name)
;
++       if ((fset = XCreateFontSet(gdk_display, name, &missingfonts,
&count, &de
fonts)) != NULL) {
+               i = XFontsOfFontSet (fset, &fstruct_list, &fname_list);
+               XFreeFontSet(gdk_display, fset);
+       }
+
+       gdk_font = (i>1) ? gdk_fontset_load (name) : gdk_font_load (name);
+
        elapsed = g_timer_elapsed (timer, NULL);
        g_timer_destroy (timer);
        printf ("(%1.4fs) [load] %s --> %p\n", elapsed, name, gdk_font);

Excellent.  I appreciate this patch.  I'll try to get it in as soon as
possible.

Actually I spoke too soon.  This doesn't work well with UTF-8 locales
in redhat.  It ends up getting double width characters in ASCII for
several fonts which is very bad.  I'm not sure what to do.

Do you have issues with both fixed/variable width fonts in redhat
UTF-8 locales ? Are you using a single iso10646-1 encoded font ?

We have a redhat here, pl. indicate what fonts you are using in UTF-8
locales.

on rh73 with no fonts specified (falling back to the default code)

with LC_ALL=en_US.UTF-8
and running testgtkhtml pointed at www.gimp.org some of the headline
fonts are double width.

Hi Larry,

Haven't tried this yet, but both Abiword and Gnumeric (Gnome 1.x) have
the same problem on RedHat 8.0. Some people have reported forcing
their locale to en_US  (no utf8) to get around it... Wonder if you
could try that and see if its any better. Not sure if this is a
feasible work around for you though.

Tom.

It isn't a problem for me personally, I just don't want apply a patch
to the stable tree that will break gtkhtml for a large number of users.

Hi Larry,
Could you attach a snap shot of the double width fonts.
I don't have the build in redhat machine.
BTW, default fonts that you are mentioning must be
-*-courier-medium-r-*-*-12-*-*-*-*-*-ISO8859-1

Created attachment 41967 [details] [review]
patch with additional checks

I want to test the patch, but it seems that testgtkhtml can not go 
through firewall. no place to set proxy, and sock5 do not work too.

So I just save the gimp page by mozilla and then attach it to mail. 
then view it in evolution mail. But I cannot reproduce the Larry said 
"double width" font.

Do not know if there are problem in my reproduce procedure?

> I want to test the patch, but it seems that testgtkhtml can not go 
> through firewall. no place to set proxy, and sock5 do not work too.

Have you set the firewall with gconf?  Try this:

gconftool-1 --type=bool --set  /system/gnome-vfs/use-http-proxy "TRUE"

gconftool-1 --type=string --set /system/gnome-vfs/http-proxy-host
"wcsuna.eng.sun.com"

gconftool-1 --type=int --set /system/gnome-vfs/http-proxy-port "8080"

this is fixed in 1.3 but I am setting the milestone to 1.4 so I don't
forget to do something about it for evolution-1.2.3

Created attachment 42159 [details]
example of the double wide ascii

I tried again with suresh's second patch with the addictional checks
and the problem seems to be fixed.  Is everyone else statisfied with
that patch?  (I'm planning on making a new gtkhtml-1.1.x release very
soon and would like to get that in if possible).

Created attachment 42160 [details] [review]
slightly refactored patch

ok this works for me in my test cases.  I'm still slightly concerned
that it might end up restricting people who are trying to use
characters outside their locale by specifying a spefici encoding, but
I think that is a lot less likely a case than people who would like
their fontset to be loaded correctly as a fontset.

So the only question remaining is wether this change is appropriate
for a stable release since it might change the fonts that people have
already specified.

This is fixed in 1.4.x regardless, closing

*** bug 230601 has been marked as a duplicate of this bug. ***