Project
DOL Server
Forum
- With Game Server Scripts Super Powers !
It should be Safe to delete for any user after a "first start" (kickstart) or against an existing Database, Official Release will only include Regions and Zones, other "Data" Package will be offered through external modules to prevent any damage to long time users
Whole script directory ./scripts/dbupdater is about file editing your database, if you don't want them delete them
(Mostly Properties Fixer, and Language Fix that were in Core up to now...)Inside this dbupdater Module are some special directories "insert", "replace", "unload" that can contains XML file describing Data to be added, replaced or distributed, these are Auto XML Package Scripts.
Once these scripts have been executed on your database, the AutoXMLUpdate Table will track any change to them and import them on next start if you modify anything inside ./scripts/dbupdater/insert or ./script/dbupdater/replace, if no changes are detected the startup will continue without any actions !
If you want to share XML Data script between users it's easily done by unloading your Table to the ./scripts/dbupdater/unload directory, there you can edit anything, rename or copy for backup purpose, prune some data from your own file before distributing, then use these script as "Replace" or "Insert" Data and copy them to the AutoXMLUpdate folder accordingly (or an other user directory Idealy...) for an auto loading on next Start !
XML file should be way easier to handle in a Versionning Control System, The Export/Unload command given with this module is able to order Records by all unique or indexed fields, it will provide easy way to distribute packages like full "Live Like" Data Career or DataQuest converted from Hardcoded Quests, it can be some way of distributing the basic "Epic Weapon" Package needed for the Live Quest...
The only constraint is to be careful with Keys, and to be EXTREMELY careful with Table not having Explicit Unique or Primary Key, if the Auto Updater can't match records with Unique Fields, duplicate could be created ! If too much keys are colliding (like itemtemplate Id_nb...) using replace scripts will be tricky, preventing from using "patch" script (insert scripts can only add new object not change anything...)
So this was only the Intro !
Unload XML DB
Let's try to do this in the right order, first you'll want to create some XML file to work on, maybe to build your own "kickstart" scripts, or maybe to share some table with co-workers, or even just to try this wonderful tool
a new command is available : /unloadxmldb [FULL|TableName]
This will unload your actual data accessed through DOL database connexion to XML file in directory ./scripts/dbupdater/unload
file names are "TableName".xml
The command can handle Table Real Name, or DataObject Class Name, entering the keyword "FULL" will unload all your table one after another !
You should really be careful with the "FULL" Keyword, it's a multi-threaded process that will eat most of your CPU / Memory / IOs / Database resources ! Use it when you have light population !
So you can try something like : /unloadxmldb ability
Then you can work on the resulting "Ability.xml" and edit or distribute it...
Configuration:
- ServerProperty "xml_unload_db_directory" - Default "dbupdater/unload"
Directory ./scripts/dbupdater/insert and ./scripts/dbupdater/replace are the keys to the process
They will be scanned on each server start for any xml file in any subdirectory, their name have no relevance !
Once they are scanned the XMLParser will try to guess Table and Object Type to insert data (or replace) accordingly.
After the process finished the xml package name (with subdir) and the file hash is stored with the Parsing result in a Database Table "AutoXMLUpdate", the date should be set accordingly, as long as the file name and hash match, the package will never be executed again !
You can delete row in the table, rename/edit files to force a reload
You just need to drop XML files in these folders and you'll have your data in-game !
Configuration:
- ServerProperty "xml_autoload_update_enable" - Default "true"
- ServerProperty "xml_load_insert_directory" - Default "dbupdater/insert"
- ServerProperty "xml_load_replace_directory" - Default "dbupdater/replace"
The Database API of DOL don't allow for easy object comparison using RDBMS powerful mechanisms, anyway a lot of DOL objects implementation should not rely on primary keys when comparing object but other "unique" fields, this mean we have to search them in Runtime... (Or we could use a query with "Where" Statement but we'll need a LOT of queries...)
As I'm willing to keep a wide range of compatibility with DOL available Database including SQLite, I used pure LINQ comparison with object Reflection and DataElement attribute to achieve to some kind of "Unique Field" Collision detection, allowing the Insert Script to eject the data, and Replace Scripts to rewrite ANY match !
This can be SLOW ! The good part is that it was easily parallelized
So if you have enough Memory and available "Cores" this won't be too much trouble, But you'll feel it anyway if you try to import data such as Mob Table or ItemTemplate Table......And with this my last Warning : Don't Try ANYTHING with InventoryTable
(It shouldn't be used anywhere in the field of "Packaging Data")
