Korean language file

A place to submit .patch fixes for the DOL SVN

Moderator: Developer Team

Korean language file

Postby whria78 » Sun Nov 16, 2014 9:21 am

1)
Korean(KR) language file (UTF-8 text file) / About 15% are translated.
Google Docs, work in progress
[url]https://docs.google.com/spreadsheets/d/1l_SjJgcahkaPg0y5lXJH-9jiF4Fv45O1jojs336H7tE/edit?usp=sharing
[/url]
2)
If I set to use SQL language (MYSQL utf8_general_ci charset), The lastest SVN does not show non-ASCII letters , only show "??". It works correct if I set to use Text language file (KR, UTF-8 text file). I think there may be a problem to process the UTF-8 SQL query results. DE language also does not show non-ASCII letters(umlaut , Ä, Ö, Ü) when I use SQL language DB.



ps)
I added one screenshot.

Environment:
SVN 3375 Dolserver (Use Text Language File ; use_dblanguage=FALSE )
(language TXT(UTF-8) file : GamePlayer.Sit.YouSitDown: DE(äü) 앉았습니다. 일어나려면 '/stand' 치거나 움직이세요.)
Client 1.109 , chat font 굴림(korean font) , main.dat ([international], mbcs=1, codepage=949)


1) It shows "Do you want to be teleported to [CLASHED LETTER].
Actually [CRASHED LETTER] is "돌토피아", UTF-8 encoded in script folder.
It crashed but it's because client does not support Korean font. It's a client-side problem.
I will work OK, if I change the client font which can support Korean.

2) It shows "DE(au) 앉았습니다. 일어나려면 '/stand' 치거나... ".
Dolserver can read UTF-8 language text database (/release/language folder) . All Korean correct.
In this time, because I set chatting font to 굴림-font(Korean font), it show correctly.

"DE(au)" is not correct. actually it is DE(äü) . The reason it show false , is the korean font does not support German umlaut.



ps) I added one more screenshot. in some case, UTF-8 SQL DB works.

SQL (edited by phpmyadmin)
column dolcharacters.name --> "피부미인" (UTF8)

In screenshot (chracter selection), you can see correct Korean character name.
Attachments
aaa.jpg
aaa.jpg (38.86 KiB) Viewed 2717 times
bbb.jpg
bbb.jpg (74.77 KiB) Viewed 2725 times
DOL-KR.xls
Korean(KR) language file (Microsoft Excel) / About 20% are translated.
(590 KiB) Downloaded 5 times
kr.txt
Korean(KR) language file (UTF-8 text file) / About 15% are translated.
(49.88 KiB) Downloaded 6 times
Last edited by whria78 on Wed Nov 19, 2014 10:32 am, edited 8 times in total.
User avatar
whria78
Contributor
 
Posts: 128
Joined: Fri Sep 21, 2007 11:27 pm

Re: Korean language file

Postby Leodagan » Mon Nov 17, 2014 6:29 pm

Is it some translation file for DOL Server ?
User avatar
Leodagan
Developer
 
Posts: 1350
Joined: Tue May 01, 2012 9:30 am
Location: Lyon

Re: Korean language file

Postby whria78 » Mon Nov 17, 2014 8:35 pm

Yes! Excel file is the newest and 20% are translated.
User avatar
whria78
Contributor
 
Posts: 128
Joined: Fri Sep 21, 2007 11:27 pm

Re: Korean language file

Postby Graveen » Tue Nov 18, 2014 8:44 pm

You can search on theses forums iirc there was similar problems for russian, i wonder if UTF-16 was not the key.
Image
* pm me to contribute in Dawn of Light: code, database *
User avatar
Graveen
Project Leader
 
Posts: 12613
Joined: Fri Oct 19, 2007 9:22 pm
Location: France

Re: Korean language file

Postby whria78 » Tue Nov 18, 2014 9:33 pm

Server script and language file works perfect if I set to use Text(UTF8) language file (use_dblanguage=FALSE) , But, when I set to use DB (use_dblanguage=TRUE), it does not works. it only show "??? ?? ???..".

In addition, even though I see "??? ???" when I set use_dblanguage=TRUE , I can chat with Korean, with 1.109 client (Korean font, main.dat ([international], mbcs=1, codepage=949)) . My chat does not changed to "??? ??". It works.


I guess it is locale (UTF8 <-> UTF16LE) problem. Windows use UTF16LE unicode, while MySQL use UTF8 unicode.

I think there a missing of translation ( UTF-8 (SQL DB)<---> Local locale (UTF-16LE (windows unicode) or ASCII (windows non-unicode, codepage) ).

UTF-8 (TEXT DB) ---> UTF16LE (dol sever) ==> works fine
UTF-8 (SQL DB) ---> UTF16LE (dol server) ==> not works (legacy ASCII letter[A-Z, 0-9] works, but foreign letters are changed to "??" )

I guess, dolserver is reading and writing SQL DB in non-unicode.
User avatar
whria78
Contributor
 
Posts: 128
Joined: Fri Sep 21, 2007 11:27 pm

Re: Korean language file

Postby whria78 » Wed Nov 19, 2014 4:50 am

I added one screenshot, that shows Korean works,
But If I changed use_dblanguage=TRUE, All Korean is changed to "????"
User avatar
whria78
Contributor
 
Posts: 128
Joined: Fri Sep 21, 2007 11:27 pm

Re: Korean language file

Postby Leodagan » Wed Nov 19, 2014 6:36 am

You changed some parameters in the client to support multi byte ?

I can chat with Korean, with 1.109 client (Korean font, main.dat ([international], mbcs=1, codepage=949))


I was pretty sure DAOC only supported Codepage 1252

DOL shouldn't be able to handle these strings, I think it works because of a "raw copy/send" from the text you provided :)

When Using database, the C# SQL driver will convert the table character set (most of time UTF-8 or Latin-1) to the Software "Locale" (and here DOL should request CP1252 !)

So here the trouble, you should store a UTF-8 (or latin-1) string in database that is "garbage" but that should give expected CP949 conversion when using an UTF-8 > CP1252 conversion... (mostly impossible... even with tool because of multi byte...)

You can look around how to change the "Culture Info" when launching the Server, I know I can do this on Linux with Mono using the "LANG" environnement variable...

here is some "Codepage" : http://msdn.microsoft.com/en-us/library ... 85%29.aspx
User avatar
Leodagan
Developer
 
Posts: 1350
Joined: Tue May 01, 2012 9:30 am
Location: Lyon

Re: Korean language file

Postby whria78 » Wed Nov 19, 2014 10:28 am

1) DAOC Client supports 2 byte unicode
In default, daoc support only CP1252, but One Korean user analysis Korean client and English client and find that
main.dat have some clue to support Unicode.

If main.dat is changed to as,

[international]
mbcs=1
codepage=949

In memory analysis, the client saves data as 2 bytes unicode.
If mbcs is not set to "1", the client saves data as 1 byte ASCII code, as you know.

To support Korean language, True type Korean font is also required.
The default font does not have Korean letters, so there's a little modification to change the font to Korean font.

2) Unicode works. But in some case, it does not works in recent SVN(3375).

*** \language\EN\language.txt (UTF-8 text file) --> ok
SQL (all, UTF-8 charset)
*** column dolcharacters.name --> ok ( I added one more screenshot )
*** column language.EN --> !!!! only shows "??? ?? ???" !!!!

3) Unicode problem

<Windows>
local - 1 byte ASCII
unicode - 2 byte UTF-16LE
<linux> <MySQL>
local - 1 byte ASCII
unicode - multibyte UTF-8

I have no experience in C# programming, but as I know, C# all support 2 byte UTF16LE by default.
In windows, almost all function consider 1 byte string as ASCII string. In windows, UTF-8 string is also considered as 1 byte ASCII string.

When importing SQL data(UTF-8), it must be traslated local ASCII pagecode first. and then to 2 byte UTF16LE.
In windows, UTF-8 string is considered as like 1 byte ASCII string, the translation error can occur.
User avatar
whria78
Contributor
 
Posts: 128
Joined: Fri Sep 21, 2007 11:27 pm

Re: Korean language file

Postby Leodagan » Wed Nov 19, 2014 10:54 am

Very interesting, so the main.dat could contain other Multi Byte Character Set (mbcs...) than CP 949 ? (which is not an UTF16-LE compliant codepage...)

By the way UTF-8 is a 1 byte length coded charset, so it's seems normal that 1 char == 1 byte, but UTF-8 like other charset can handle "multi byte", that mean a control character will be used to reference char that need more than one byte to be writed... (from what I know of UTF-8 can go up to 4 bytes / character)
User avatar
Leodagan
Developer
 
Posts: 1350
Joined: Tue May 01, 2012 9:30 am
Location: Lyon

Re: Korean language file

Postby whria78 » Wed Nov 19, 2014 11:11 am

1)
CP949 = Korean codepage (euc-kr)

2)
Yes! We succeeded to chat in Korean, by small modification of the client. No modification of game.DLL. The client saves data as 2 byte UTF16-LE.
It can be adopted to other language, german, russian ... etcs.

3)
Unicode support was long problem in windows.

In windows API, there are two version of API. one is for ASCII, another is for UTF16LE.
In case of MessageBox API, there's MessageBoxW (UTF16LE) , and MessageBoxA (ASCII) . The compiler choose the function in C++ programming.

Because UTF-8 is multi - 1 byte string, windows fuctions consider UTF-8 string as ASCII 1 byte string.
so, function looks works (it does not result error), but it fail to show correct if UTF-8 string is not to translated to ASCII first.

In Dolserver,
TXT UTF-8 string works. [ /language/ , /script/ ]
some SQL UTF-8 string works, [ dolchracter.name ]
some SQL UTF-8 string does not work. [ lanauage.** ]

I guess there 's some missing of translation, when the SQL data is imported.
User avatar
whria78
Contributor
 
Posts: 128
Joined: Fri Sep 21, 2007 11:27 pm

Re: Korean language file / Korean Client Patch

Postby whria78 » Wed Nov 19, 2014 11:59 am

Here is Korean Client Patch file.

1) unicode config ( main.dat )
2) korean truetype font setting ( /pregame , /ui )

"Gulim" is the name of Korean Truetype font.
Attachments
gulim.zip
Korean Gulim font
(4.71 MiB) Downloaded 15 times
patch.zip
(2.4 KiB) Downloaded 16 times
User avatar
whria78
Contributor
 
Posts: 128
Joined: Fri Sep 21, 2007 11:27 pm

Re: Korean language file

Postby Graveen » Sun Nov 23, 2014 8:00 pm

Thank you whria78 to troubleshoot theses issues. Kudos ! :)
Image
* pm me to contribute in Dawn of Light: code, database *
User avatar
Graveen
Project Leader
 
Posts: 12613
Joined: Fri Oct 19, 2007 9:22 pm
Location: France

Re: Korean language file

Postby whria78 » Fri Nov 28, 2014 1:08 pm

I attached screenshots related to language issue.

If it can display character name correctly and / packet/client/168/character***.cs allows non-english letter,
user can name their character in native language.

Interestingly, I succeeded to rename the character name by GM command (/player name)

/player name 피부미인

and then, I checked the SQL have the correct value(피부미인).

and I can see the right name(피부미인) in the character selection window, but I cannot login because it does not allow. (related to packet/client/168/character***)

1)
Server Language : KR
User Language : EN (/language set EN)

The names of Craftskill are translated in the language of the server(KR).
It's better to be as the user language setting.

Untitled-1 copy.jpg
Untitled-1 copy.jpg (118.58 KiB) Viewed 2667 times



2)
Server Language : KR
/language/KR/KR-language.txt (UTF-8) -->
SetGuardName.Armsman: 암스맨
SetGuardName.Armswoman: 암스우먼

The guard name are not displayed well.

Untitled-2 copy.jpg
Untitled-2 copy.jpg (122.59 KiB) Viewed 2667 times


3)
Server Language : KR
/script/custom/liveteleport.cs

The display windows shows correct Korean letters.
But, In the game character, the name of the teleporter is not correct.

Untitled-3 copy.jpg
Untitled-3 copy.jpg (164.79 KiB) Viewed 2667 times
User avatar
whria78
Contributor
 
Posts: 128
Joined: Fri Sep 21, 2007 11:27 pm

Re: Korean language file

Postby whria78 » Fri Nov 28, 2014 10:24 pm

Incorrect Guards name is client-side problem.

Here's one more screenshot.

aaa copy.jpg
aaa copy.jpg (86.99 KiB) Viewed 2660 times
User avatar
whria78
Contributor
 
Posts: 128
Joined: Fri Sep 21, 2007 11:27 pm

Re: Korean language file

Postby Leodagan » Sat Nov 29, 2014 4:32 am

You're doing a great research job here :D

I wasn't aware that DOL could already do that much !

I think NPC's and Player's Name display aren't using a compatible "font", did you find any "switch" or "parameter" where you can change the "in-game font" and not only the "UI Font", like you did to enable chat and ragdoll Korean display ?

I think appart from this you pretty much solved everything :)

The Guard display shows the same number of "letters" than in the UI Display, so it's a good guess that client is using the correct character set (or multi-byte should have garbage'd the guard name...), but the font is clearly missing these special Korean character !
User avatar
Leodagan
Developer
 
Posts: 1350
Joined: Tue May 01, 2012 9:30 am
Location: Lyon

Next

Return to DOL Code Contributions

Who is online

Users browsing this forum: No registered users and 1 guest