Korean language file

[whria78]

1)
Korean(KR) language file (UTF-8 text file) / About 15% are translated.
Google Docs, work in progress
https://docs.google.com/spreadsheets/d/ ... p=sharing
2)
If I set to use SQL language (MYSQL utf8_general_ci charset), The lastest SVN does not show non-ASCII letters , only show "??". It works correct if I set to use Text language file (KR, UTF-8 text file). I think there may be a problem to process the UTF-8 SQL query results. DE language also does not show non-ASCII letters(umlaut , Ä, Ö, Ü) when I use SQL language DB.



ps)
I added one screenshot.

Environment:
SVN 3375 Dolserver (Use Text Language File ; use_dblanguage=FALSE )
(language TXT(UTF-8) file : GamePlayer.Sit.YouSitDown: DE(äü) 앉았습니다. 일어나려면 '/stand' 치거나 움직이세요.)
Client 1.109 , chat font 굴림(korean font) , main.dat ([international], mbcs=1, codepage=949)


1) It shows "Do you want to be teleported to [CLASHED LETTER].
Actually [CRASHED LETTER] is "돌토피아", UTF-8 encoded in script folder.
It crashed but it's because client does not support Korean font. It's a client-side problem.
I will work OK, if I change the client font which can support Korean.

2) It shows "DE(au) 앉았습니다. 일어나려면 '/stand' 치거나... ".
Dolserver can read UTF-8 language text database (/release/language folder) . All Korean correct.
In this time, because I set chatting font to 굴림-font(Korean font), it show correctly.

"DE(au)" is not correct. actually it is DE(äü) . The reason it show false , is the korean font does not support German umlaut.



ps) I added one more screenshot. in some case, UTF-8 SQL DB works.

SQL (edited by phpmyadmin)
column dolcharacters.name --> "피부미인" (UTF8)

In screenshot (chracter selection), you can see correct Korean character name.

[Leodagan]

Is it some translation file for DOL Server ?

[whria78]

Yes! Excel file is the newest and 20% are translated.

[Graveen]

You can search on theses forums iirc there was similar problems for russian, i wonder if UTF-16 was not the key.

[whria78]

Server script and language file works perfect if I set to use Text(UTF8) language file (use_dblanguage=FALSE) , But, when I set to use DB (use_dblanguage=TRUE), it does not works. it only show "??? ?? ???..".

In addition, even though I see "??? ???" when I set use_dblanguage=TRUE , I can chat with Korean, with 1.109 client (Korean font, main.dat ([international], mbcs=1, codepage=949)) . My chat does not changed to "??? ??". It works.


I guess it is locale (UTF8 <-> UTF16LE) problem. Windows use UTF16LE unicode, while MySQL use UTF8 unicode.

I think there a missing of translation ( UTF-8 (SQL DB)<---> Local locale (UTF-16LE (windows unicode) or ASCII (windows non-unicode, codepage) ).

UTF-8 (TEXT DB) ---> UTF16LE (dol sever) ==> works fine
UTF-8 (SQL DB) ---> UTF16LE (dol server) ==> not works (legacy ASCII letter[A-Z, 0-9] works, but foreign letters are changed to "??" )

I guess, dolserver is reading and writing SQL DB in non-unicode.

[whria78]

I added one screenshot, that shows Korean works,
But If I changed use_dblanguage=TRUE, All Korean is changed to "????"

[Leodagan]

You changed some parameters in the client to support multi byte ?

I can chat with Korean, with 1.109 client (Korean font, main.dat ([international], mbcs=1, codepage=949))

I was pretty sure DAOC only supported Codepage 1252

DOL shouldn't be able to handle these strings, I think it works because of a "raw copy/send" from the text you provided

When Using database, the C# SQL driver will convert the table character set (most of time UTF-8 or Latin-1) to the Software "Locale" (and here DOL should request CP1252 !)

So here the trouble, you should store a UTF-8 (or latin-1) string in database that is "garbage" but that should give expected CP949 conversion when using an UTF-8 > CP1252 conversion... (mostly impossible... even with tool because of multi byte...)

You can look around how to change the "Culture Info" when launching the Server, I know I can do this on Linux with Mono using the "LANG" environnement variable...

here is some "Codepage" : http://msdn.microsoft.com/en-us/library ... 85%29.aspx

[whria78]

1) DAOC Client supports 2 byte unicode
In default, daoc support only CP1252, but One Korean user analysis Korean client and English client and find that
main.dat have some clue to support Unicode.

If main.dat is changed to as,

[international]
mbcs=1
codepage=949

In memory analysis, the client saves data as 2 bytes unicode.
If mbcs is not set to "1", the client saves data as 1 byte ASCII code, as you know.

To support Korean language, True type Korean font is also required.
The default font does not have Korean letters, so there's a little modification to change the font to Korean font.

2) Unicode works. But in some case, it does not works in recent SVN(3375).

*** \language\EN\language.txt (UTF-8 text file) --> ok
SQL (all, UTF-8 charset)
*** column dolcharacters.name --> ok ( I added one more screenshot )
*** column language.EN --> !!!! only shows "??? ?? ???" !!!!

3) Unicode problem

<Windows>
local - 1 byte ASCII
unicode - 2 byte UTF-16LE
<linux> <MySQL>
local - 1 byte ASCII
unicode - multibyte UTF-8

I have no experience in C# programming, but as I know, C# all support 2 byte UTF16LE by default.
In windows, almost all function consider 1 byte string as ASCII string. In windows, UTF-8 string is also considered as 1 byte ASCII string.

When importing SQL data(UTF-8), it must be traslated local ASCII pagecode first. and then to 2 byte UTF16LE.
In windows, UTF-8 string is considered as like 1 byte ASCII string, the translation error can occur.

[Leodagan]

Very interesting, so the main.dat could contain other Multi Byte Character Set (mbcs...) than CP 949 ? (which is not an UTF16-LE compliant codepage...)

By the way UTF-8 is a 1 byte length coded charset, so it's seems normal that 1 char == 1 byte, but UTF-8 like other charset can handle "multi byte", that mean a control character will be used to reference char that need more than one byte to be writed... (from what I know of UTF-8 can go up to 4 bytes / character)

[whria78]

1)
CP949 = Korean codepage (euc-kr)

2)
Yes! We succeeded to chat in Korean, by small modification of the client. No modification of game.DLL. The client saves data as 2 byte UTF16-LE.
It can be adopted to other language, german, russian ... etcs.

3)
Unicode support was long problem in windows.

In windows API, there are two version of API. one is for ASCII, another is for UTF16LE.
In case of MessageBox API, there's MessageBoxW (UTF16LE) , and MessageBoxA (ASCII) . The compiler choose the function in C++ programming.

Because UTF-8 is multi - 1 byte string, windows fuctions consider UTF-8 string as ASCII 1 byte string.
so, function looks works (it does not result error), but it fail to show correct if UTF-8 string is not to translated to ASCII first.

In Dolserver,
TXT UTF-8 string works. [ /language/ , /script/ ]
some SQL UTF-8 string works, [ dolchracter.name ]
some SQL UTF-8 string does not work. [ lanauage.** ]

I guess there 's some missing of translation, when the SQL data is imported.

[whria78]

Here is Korean Client Patch file.

1) unicode config ( main.dat )
2) korean truetype font setting ( /pregame , /ui )

"Gulim" is the name of Korean Truetype font.

[Graveen]

Thank you whria78 to troubleshoot theses issues. Kudos !

[whria78]

I attached screenshots related to language issue.

If it can display character name correctly and / packet/client/168/character***.cs allows non-english letter,
user can name their character in native language.

Interestingly, I succeeded to rename the character name by GM command (/player name)

/player name 피부미인

and then, I checked the SQL have the correct value(피부미인).

and I can see the right name(피부미인) in the character selection window, but I cannot login because it does not allow. (related to packet/client/168/character***)

1)
Server Language : KR
User Language : EN (/language set EN)

The names of Craftskill are translated in the language of the server(KR).
It's better to be as the user language setting.

Untitled-1 copy.jpg (118.58 KiB) Viewed 5641 times

2)
Server Language : KR
/language/KR/KR-language.txt (UTF-8) -->
SetGuardName.Armsman: 암스맨
SetGuardName.Armswoman: 암스우먼

The guard name are not displayed well.

Untitled-2 copy.jpg (122.59 KiB) Viewed 5641 times

3)
Server Language : KR
/script/custom/liveteleport.cs

The display windows shows correct Korean letters.
But, In the game character, the name of the teleporter is not correct.

Untitled-3 copy.jpg (164.79 KiB) Viewed 5641 times

[whria78]

Incorrect Guards name is client-side problem.

Here's one more screenshot.

aaa copy.jpg (86.99 KiB) Viewed 5634 times

[Leodagan]

You're doing a great research job here

I wasn't aware that DOL could already do that much !

I think NPC's and Player's Name display aren't using a compatible "font", did you find any "switch" or "parameter" where you can change the "in-game font" and not only the "UI Font", like you did to enable chat and ragdoll Korean display ?

I think appart from this you pretty much solved everything

The Guard display shows the same number of "letters" than in the UI Display, so it's a good guess that client is using the correct character set (or multi-byte should have garbage'd the guard name...), but the font is clearly missing these special Korean character !