DOL on Linux with Mono3

[Leodagan]

Hello.

I'm currently trying to install DOL on a Linux private server and achieving a stable server which could run off Mono and expand platform compatibility.

For the background I'm an unix system admin working with managed applications server (mostly J2EE/Oracle based application) and was formerly a PHP/Java/MySQL developer which make DOL C# code mostly readable

My purpose with DOL running on Linux is to use some Idle power on a new Linux Box (Core i3 3.4Ghz; 8Go RAM; 2x 1To), I installed as a replacement for an old server, to offer a stable DAOC server for testing purpose and finally try to make an easier Live-like environment to reach RvR in New Frontiers (but I'm more a technician than a game master)

As I don't have lot of time to spend in creating content (which is the weak point of most early Shards) I intend to rely on tools to create some automated content...

Here is the Setup I've made so far :

Prerequisites:

- Mono (> 3.0 or 3.2, try your distribution specific testing packages for those...)
- GTK+ (for gui)
- xauth (for remote display)
- MySQL (I personally use MariaDB as a drop in replacement)
- Packaged SQLite3 for your distribution (Later DOL improvements)
- SVN

Optional :
- Apache + PHP + PhpMyAdmin for easy access to database.

Locales :

DAOC Client use the charset : en_US.CP1252 (Windows Code Page 1252 ~= ISO8859-15), you need to have this locale installed on your system to make it available for Mono.

This setup highly depend on your distribution, search google for your distribution Name + locale + en_US.CP1252

Here is a result for Ubuntu : http://askubuntu.com/questions/76013/ho ... ntu-server

I'll just give the step for "Gentoo" Users (Maybe ArchLinux too...) Because that's part of my Setup

as "root" :

Code: Select all

# echo "en_US.CP1252 CP1252" >> /etc/locale.gen && locale-gen

Getting Sources and Building

First create a working directory and checkout the DOL SVN

Code: Select all

$ svn checkout http://svn.code.sf.net/p/dolserver/code/ dolserver

Then go into trunk and launch xbuild.

Code: Select all

$ cd dolserver/DOLSharp/trunk ### BUILD $ MONO_IOMAP=case xbuild Dawn\ of\ Light.sln

I've seen a lot of diff files, accross the forum, to patch the csproj files but at last it only needs an "xcopy" workaround, xbuild can handle case insensitiveness with MONO_IOMAP=case

If you want to build DOL Server as "Release" binary (without debug symbols) you can specify the expected configuration in command line

Code: Select all

$ MONO_IOMAP=case xbuild /p:Configuration=Release Dawn\ of\ Light.sln

Remember, it will be really hard for DOL Community to help you with any bugs if you don't have debug symbols !

Preparing Database

For the MySQL part I rely on the InnoDB Engine, I don't know if it's my default Engine (can't remember If I configured that) but DOLServer created the tables using this Engine so I focused on tuning InnoDB

my.cnf parameters, notice some of them could be tuned to your system and I encourage creating a separate MySQL instance for DOL purpose as I'm using unsecure parameters and maybe "MariaDB only" parameters. (there is enough how to on Google for making a separate MySQL instance)

generic tuning, thread pool set for 2 processors, some caching and the almighty "lower_case_table_names" to mimic Windows MySQL Server

Code: Select all

[mysqld] thread_handling=pool-of-threads thread_pool_size=2 lower_case_table_names=1 # Turn on disk-ordered reads optimizer_switch='mrr=on' optimizer_switch='mrr_cost_based=off' # Size limit for the whole join join_buffer_space_limit = 300M # Limit for each individual table join_buffer_size = 100M optimizer_switch='index_merge_sort_intersection=on' # Turn on both Hash Join and Batched Key Access join_cache_level = 8

InnoDB tuning

Code: Select all

# Total Memory dedicated to innoDB (will use more) innodb_buffer_pool_size = 1024M # use system memory alocation (faster on recent systems) innodb_use_sys_malloc = 1 # Number of instances for Memory pool, raise this if buffer_pool_size is set to 8G or more innodb_buffer_pool_instances = 2 # this is the default, increase it if you have lots of tables innodb_additional_mem_pool_size = 128M # the max is there to avoid run-away growth on your machine innodb_data_file_path = ibdata1:10M:autoextend:max:20G # we keep this at around 25% of of innodb_buffer_pool_size innodb_log_file_size = 256M # for large transactions innodb_log_buffer_size = 32M # see the innodb config docs, the other options are not always safe # This will delay disk sync prevent iowait, but data can be lost on crash, improves performances dramatically !! innodb_flush_log_at_trx_commit = 0

You should restart your MySQL server after changing any of those parameters, if it's not already done create a DOL dedicated user with all privileges on a dedicated database

Import Public DB:

Code: Select all

# Get in Work directory first $ svn co https://dolpubdb.svn.sourceforge.net/svnroot/dolpubdb/ dolpubdb $ mysql -u "DOLUSER" -p'DOLPASSWORD' -b "DOLDATABASE" < dolpubdb/collection/dolpubdbV3.0.sql

If you created a dedicated MySQL Instance (that means another process running a MySQL database) you need to set the right socket file (which you should have configured to make 2 servers run side by side) with the option "-S"

example :

Code: Select all

mysql -u "DOLUSER" -p'DOLPASSWORD' -b "DOLDATABASE" -S /var/run/mysqld/mysqld2.sock < dolpubdb/collection/dolpubdbV3.0.sql

Configure and Launch DOLServer

Now we need to launch the DOLServer first time to make remaining tables and try to connect.

First configure the XML file by hand (I tried to use to DOLConfig.exe on X remote display it wasn't working great...)
From trunk directory, config file should be in : ./debug/config/serverconfig.xml

Code: Select all

<?xml version="1.0" encoding="utf-8"?> <root> <Server> <Port>10300</Port> <IP>0.0.0.0</IP> <RegionIP>127.0.0.1</RegionIP> <RegionPort>10400</RegionPort> <UdpIP>0.0.0.0</UdpIP> <UdpPort>10400</UdpPort> <EnableUPnP>True</EnableUPnP> <DetectRegionIP>True</DetectRegionIP> <ServerName>Test Server</ServerName> <ServerNameShort>TESTSERVER</ServerNameShort> <LogConfigFile>./config/logconfig.xml</LogConfigFile> <RegionConfigFile>./config/regions.xml</RegionConfigFile> <ZoneConfigFile>./config/zones.xml</ZoneConfigFile> <ScriptCompilationTarget>./lib/GameServerScripts.dll</ScriptCompilationTarget> <ScriptAssemblies>DOLBase.dll,GameServer.dll,DOLDatabase.dll,System.dll,log4net.dll,System.Xml.dll</ScriptAssemblies> <AutoAccountCreation>True</AutoAccountCreation> <GameType>Normal</GameType> <CheatLoggerName>cheats</CheatLoggerName> <GMActionLoggerName>gmactions</GMActionLoggerName> <InvalidNamesFile>./config/invalidnames.txt</InvalidNamesFile> <DBConnectionString>Server=localhost;Port=3306;Database=DOLDATABASE;User ID=DOLUSER;Password=DOLPASSWORD;Treat Tiny As Boolean=False</DBConnectionString> <DBType>MYSQL</DBType> <DBAutosave>True</DBAutosave> <DBAutosaveInterval>10</DBAutosaveInterval> <CpuUse>4</CpuUse> </Server> </root>

You should change DBConnectionString to match your own database (and Port if you used an dedicated MySQL instance)
CpuUse is set to the number of threads of the CPU (Core i3 = 2Core+2HT)

Remove the ColoredConsole from Log4Net
file : ./debug/config/logconfig.xml

Code: Select all

<appender name="ColoredConsoleAppender" type="log4net.Appender.ConsoleAppender">

Just changed ColoredConsoleAppender type to ConsoleAppender to prevent other modifications of code.

Now we can launch DOLServer.exe

Code: Select all

$ cd ./debug $ LANG=en_US.CP1252 mono --gc=sgen --server DOLServer.exe

Here I use the Mono SGEN garbage collector and --server option, it is supposed to work better for long processes than with standard options...

Ignore PerformanceCounterCategory Errors... I tried to port some of the code to "Mono PerformanceCounter" but it doesn't give result anyway... It's tracked as a bug in Mono bugzilla ! (Mono PerformanceCounter always returning 0)

Server should run without errors and get available in about 30 sec to 1 min, well ignoring the huge amount of warnings

Tips & Tricks
- Verifying locale

For those having trouble checking their Locale on DOL + Mono (you really need to mimic Windows CP1252 to have a working server for fresh DAOC Client...)

Using this command you can check that your Mono Runtime is correctly obeying the LANG setting you use :

Code: Select all

LANG=en_US.CP1252 csharp -e 'Console.WriteLine("Current Culture : {0}, Current Charset : {1}", System.Globalization.CultureInfo.CurrentCulture, System.Text.Encoding.Default);'

Should return

Code: Select all

Current Culture : en-US, Current Charset : I18N.West.CP1252

Warning/Errors I'm trying to Solve

- Current SVN is NOT Compatible with Mono 3.12, Mono 3.12 is not between Mono 3.0 and Mono 3.2 it's in fact a more updated version than Mono 3.2 ( 12 > 2 and not 2 > 1.2 ), some fixes may come later to make it work with latest Mono !

- The process memory is slowly increasing until a point where "RelocateRegions()" is called which can take up to 1 sec (around 800ms) I read somewhere on the forum that could be the weak garbage collector in Mono which hangs up the server up to 1 sec (this happens every 3 or 4 hours and I never noticed a freeze while in game)
Is there a big Impact on server if this callback take so long ?

All the code I use are SVN DOLServer and SVN Public Database, didn't tried any tool for now...

[Tolakram]

Excellent, thanks for the effort.

[Graveen]

Thank you Leodagan. Just a quick note to help you :
- xcopy is a postbuild event, that could be replaced by a simple script doing the job. It is rather a convenience than a necessity
- most errors happened to be a case problem with MySQL on Linux
- others issues were: log4net colored lines, and perf counters
- effectively uPNP stuff never been tested under mono, it is relatively recent

Last thing, i'm surprized DOLConfig.exe GUI is messed.

Another thing: if you want to patch the code to be mono compliant (best would be a new target, or macro & preprocessing via #define) do not hesitate. We don't support Mono, but i don't have any troubles to include some *tested* patches to have a compliant build.

[Leodagan]

- xcopy is a postbuild event, that could be replaced by a simple script doing the job. It is rather a convenience than a necessity

Well the changes are easy anyway and can be done by hand (with a find -exec after inhibiting the events)

- most errors happened to be a case problem with MySQL on Linux

Here I use MySQL parameter to use all table names as lower case, it should solve those problem (or I missed an other problem ?)

- others issues were: log4net colored lines, and perf counters

As long as perf counters are only used to display stats in the Console it's not a big impact, it can be easily replace with system tool

- effectively uPNP stuff never been tested under mono, it is relatively recent

Disabling it in my DOL config should remove that error message then (I'll give it a try) anyway I have no use for UPnP NAT stuff as I'm not running the box behind a router.

Last thing, i'm surprized DOLConfig.exe GUI is messed.

I think it's mostly affected by the remote Display (TCP/IP XForwarding to a Windows Xming server) and the lack of all graphic environment libs on the server.

Another thing: if you want to patch the code to be mono compliant (best would be a new target, or macro & preprocessing via #define) do not hesitate. We don't support Mono, but i don't have any troubles to include some *tested* patches to have a compliant build.

Most of the Code seems to be Mono compliant already, during my research I found that it's possible to query the VM to know if it's standard .NET or Mono. This could make it easier to write small Code variation where it's really needed

[Leodagan]

Here is the small test to check for Mono Runtime

Code: Select all

public static bool IsRunningOnMono () { return Type.GetType ("Mono.Runtime") != null; }

This could enable workarounds only where it's needed, without creating a new Mono Branch...

[Dinberg]

This is a great effort, good work so far. I'm afraid I have no Linux experience, but wish you all the best!

[Leodagan]

This is a great effort, good work so far. I'm afraid I have no Linux experience, but wish you all the best!

The Linux part is not really important, the main work is around Mono-compatibility or Mono-workarounds

I'll have to build a Mono release of DOL for Windows anyway, I can't use the server for all testing purpose and my workstation is running windows.

By the way disabling the UPNP parameter from DOLServer did the work about the networking error message.

[HunabKu]

- more info about xcopy replacment on Linux here : http://us.generation-nt.com/answer/xcop ... 20231.html
- activate mysql case sensitive on linux : http://forums.mysql.com/read.php?10,211 ... msg-211093
- lognet colored lines and perf counter not really necessary for dol works corectly i think
- upnp not necessary for all, but yes recently added, but i do not personally tested

[Leodagan]

"rsync" should be the best replacement for xcopy (rsync can work in local) but it's not in the base setup of most distributions...

"cp" even with a lot of options can't handle the job that is done with xcopy (recursively copy files filtered by extensions...) and have troubles to copy a "collection" of files to a non-existing directory. (needs mkdir -p before cp...)

the last options could be "find"/"awk" scripts (find is very powerful with options like -exec that can run a command or a complex awk script on each files) but it will be some work to test it and still not portable.


Getting back to the topic, I registered at Intel Software to grab their Intel C Compiler (icc) this compiler is said to produce binaries that run better than GCC (Gnu Compiler) or even MSVC (Microsoft Compiler), I'll try to build the Mono Runtime with it to run the DOLServer (and of course expecting improvements, but without any test protocol...)

I Don't have much time in week day to work on the tests...

[Leodagan]

Proposed as an Article.

[Graveen]

A sticky post ? Or an entry in the wiki ? Good idea.

[Leodagan]

In the Wiki, I went through http://www.dolserver.net/articles/ and added an article

[HunabKu]

Realy good idea, thanks.
Just waiting Graveen validate it

[Graveen]

Done Leodagan, thank you for your help. /bow

Hunab, i added a brand new mob trigger. Oh i see you written a guid for this !

[HunabKu]

Yes here