OpenDrakan ~ A Drakan engine recreation

Anything to do with Drakan level editing and modifications, post it here! Also this is the place to tell us about your new levels and get player feedback.
Zalasus
Whelp
Posts: 18
Joined: Mon Jan 29, 2018 6:50 pm
Location: Germany

OpenDrakan ~ A Drakan engine recreation

Post by Zalasus »

OpenDrakan

Greetings fellow Drakan fans!

I'd like to present to you a project I have been working on over the past two months or so. As one could probably guess from the title, OpenDrakan is going to be an open-source recreation of Drakan's Riot Engine.

What that means is: I am rewriting the software part of Drakan completely from scratch in such a way that it can load the original game's files (which the user has to provide from a copy of the game) while keeping the source code open to everyone. In rewriting the engine, the original game's flaws and bugs can be fixed, as well as allowing the game to be ported to modern OSs without having to rely on patches or hacks.

My ultimate goal is to get Drakan: Order of The Flame to run smoothly on a mondern GNU/Linux system whilst keeping the original look and feel of the game. I am focusing on the singleplayer campaign, but multiplayer support is definitely something that could be implemented once the engine makes it to a point where everything else works fine.

Of course I want OpenDrakan to also run on modern Windows OSs, so I try to keep my code as portable as possible. Even porting the engine to Android or other more exotic systems shouldn't be too far from reality.

As I already mentioned above, this recreation won't let users play the game without owning a copy of it. No copyrighted material by Surreal Software/Psygnosis/Warner Bros. will be distributed along with OpenDrakan. I know the poor availability of the game (not on Steam/GOG etc.) is a problem in that regards, but it is one we currently have to live with either way.

This is what I've got so far:
  • Basic IO stuff (reading the engines data files)
  • Ability to load models, textures and levels
  • A simple, static level viewer
Here's a screenshot of OpenDrakan rendering the game's first level:

Image

As you can see it's a bit "rough around the edges" :D There's still plenty to be done until this will look similar to Drakan in graphics and gameplay. The major problem is that I am not very experienced in the field of 3D programming, so getting stuff on the screen is way harder for me than actually loading the game's files.

Since this is merely a funtime project of mine, I can't guarantee it will eventually see a release. But that also depends on whether the project will gain support by others in the future or not. In the meantime it's just me hacking around whenever I feel like it.

Source code
I'm writing the engine in C++, using OpenSceneGraph as a more convenient way of interfacing with OpenGL.
As OpenDrakan is going to be free software, the source code is publicly available. Right now it can be found on my GitHub account, here.

Thank you for your interest in my project. I hope to keep you updated regularly with any milestones I archive. Otherwise you can always check my GitHub for the latest project state.

HeckFluff
Whelp
Posts: 24
Joined: Sat Oct 14, 2017 12:47 am
Location: England

Re: OpenDrakan ~ A Drakan engine recreation

Post by HeckFluff »

Poking around in your reverse engineering notes...
followed by that, 16 bytes that indicate texture orientation, but in what way i can not tell yet
They're normalised uint16_t texture coordinates that apply to both textures.
*.adb Model/Character animations (?)
The Header struct is this:
const struct Header
{
float duration = 0;
uint32_t originalNumFrames = 0;
uint32_t numFrames = 0;
uint32_t unk0 = 0;
uint32_t modelNumNodes = 0;
uint32_t modelNumChannels = 0;
uint32_t unk1 = 0; // Non-Looping flag?
uint32_t refCount = 0;
float rotationThreshold = 0;
float scaleThreshold = 0;
float translationThreshold = 0; // Probably.
} header = read<Header>(bin);
I still don't know why the animation stores the number of channels.

The Frame struct is this:
struct Frame
{
float t = 0;
fmat3x4 xform = identity;
};
To attain enlightenment, invert the matrix. It was probably inverted for skeletal vertex morphing animations. Convert it to translation+quaternion for better interpolation, and maybe check for scale too.
*.ssd Animations ("STOMP"-Sequences)
It's probably best to call these "sequences" or "actor scenes" to avoid confusion with model animations.
Sequence (0x0310 - 0x0312)
If you haven't got this yet, gleam my source code, or fire up Sysinternal's Procmon to catch the file reads.

float box_x_length;
uint8[12] ??? // all zeroes. the fuck is this?
float box_y_length;
uint8[12] ??? // all zeroes
float box_z_length;
3x3 xform matrix of OBB?
Model (0x0200 - 0x0209)
The types of data are:
// 00 02 model info
// 03 02 vertex data
// 04 02 polygon data
// 06 02 texture refs
// 07 02 Bounding spheres
// 08 02 lod info/scene nodes/animation refs/channels
// 09 02 uint16, uint32. Haven't found a non-zero occurence yet.
There are a few things I don't know yet. I haven't looked into bounding spheres, or what the two transformations in each of a model's channels mean, but it's probably something to do with limb dismemberment (look at spiders).

And I don't know what the unknown below means, in the extracted model polygons:
const struct PolyHeader
{
uint16_t unk;
uint16_t n;
uint16_t texId;
} polyHeader = read<PolyHeader>(bin);
I suspect it's either something to do with graphics/physics materials or it's used in normals calculation. Set it to zero for all polygons and see what happens.

I also don't know how physics bodies are derived from models, but it's probably OBBs or spheres. Easy to test for, though. Just use a hex editor to make them massive and see what happens.

Class (0x0100 - 0x0102)
These are actually objects or maybe "entity templates" or "entity prefabs". They are instances of dragon.rfl classes with editor-specified property values. They are then cloned or used in levels as entities. This terminology doesn't match the editor's, but it makes more sense, because otherwise, you've got two kinds of classes: those in dragon.rfl and those in SRSC files.

For dragon.rfl, you might need to get the class definitions out of that as they contain the default values of properties and the inheritance tree. The data is stored as Win32 resources. Alternatively, just hard-code it all.

Zalasus
Whelp
Posts: 18
Joined: Mon Jan 29, 2018 6:50 pm
Location: Germany

Re: OpenDrakan ~ A Drakan engine recreation

Post by Zalasus »

HeckFluff wrote: Sun Feb 11, 2018 5:46 pm They're normalised uint16_t texture coordinates that apply to both textures.
Yep, I figured that out recently and started to experiment with how the UVs map
to the cell's vertices. I found that using the order bottom-left, top-right, top-left, bottom-right gives the right results in most cases, but some textures are still wrong, and not in a predictable way. Some are rotated 90 degrees, some are flipped. Might need to look into that a bit more.

HeckFluff wrote: Sun Feb 11, 2018 5:46 pm The Header struct is this: (...)
I hadn't looked into animations or sequences at all yet, but this will definitely help.

HeckFluff wrote: Sun Feb 11, 2018 5:46 pm 3x3 xform matrix of OBB?
I had thought of that, as a 3x3 float matrix would fit in there, but in that case I'd expect there to be at least one non-zero value in each row.

HeckFluff wrote: Sun Feb 11, 2018 5:46 pm These are actually objects or maybe "entity templates" or "entity prefabs".
I called them "object templates" in my code until I decided keeping the terminology consistent with the editor would be less confusing xD
In my code I distinct between the two types of classes by calling one "Class" and the other "RflClass", but I agree that I should change the first one to something else.

Thanks for sharing that info with me. Especially all the stuff about animations will be useful. I'll take a look at your sources.

HeckFluff
Whelp
Posts: 24
Joined: Sat Oct 14, 2017 12:47 am
Location: England

Re: OpenDrakan ~ A Drakan engine recreation

Post by HeckFluff »

I had thought of that, as a 3x3 float matrix would fit in there, but in that case I'd expect there to be at least one non-zero value in each row.
It is a 3x3 matrix. I've looked at System.mod at offset 208,778, and there's a float 3x3 matrix. It is read as a 48-byte struct, with the box position at the start. I'm not sure if the zero entries will ever be non-zero, though. You might want to assert that.

EDIT: Nope, it's not always a scale matrix. Anyway, here's an updated riot_database_format.txt, because I can't figure out GitHub. Oh, and it wasn't animation xforms that were inverted, it's character model node xforms.
Attachments
riot_database_format.zip
(7.75 KiB) Downloaded 553 times

Zalasus
Whelp
Posts: 18
Joined: Mon Jan 29, 2018 6:50 pm
Location: Germany

Re: OpenDrakan ~ A Drakan engine recreation

Post by Zalasus »

HeckFluff wrote: Tue Feb 13, 2018 2:38 pm It is a 3x3 matrix. I've looked at System.mod at offset 208,778, and there's a float 3x3 matrix. It is read as a 48-byte struct, with the box position at the start. I'm not sure if the zero entries will ever be non-zero, though. You might want to assert that.

EDIT: Nope, it's not always a scale matrix. Anyway, here's an updated riot_database_format.txt, because I can't figure out GitHub. Oh, and it wasn't animation xforms that were inverted, it's character model node xforms.
Thank you so much for updating my documentation :) I took the freedom of pushing it for you.

Someday I'll have to rework the documentation to incorporate what I have found out about the level structure (found here in case you haven't found it yet), and also to make the formatting a bit more consistent.

User avatar
yangez93
Dragon
Posts: 103
Joined: Wed Apr 05, 2017 9:10 am
Location: Poland
Contact:

Re: OpenDrakan ~ A Drakan engine recreation

Post by yangez93 »

Hi Zelasus.

Very good news and I'm very glad to see you started. There are a lot of such initatives as OpenMW, KaM Remake and many others. I wish you that you'll finish project.

As I said in another topic, you can make such features as:
  • Portability-Android version, maybe Linux as well Cross-platform. Drakan in smartphone or tablet? Sounds good.
  • Advanced Multiplayer feature- upgraded and free of bugs. Build-in tournaments, or even popular survive mode?
  • Add Bots- so the Drakan won't be little bit boring after finish game :)
  • Improve Edditing features such as Drakan Level Editor or make completly new Drakan 3D Level Editor? We will be able to make levels in easy way. Or Editor build-in? Not separate .exe file :)
  • Better AI,Improve AI system
  • Extra modes. Maybe face,skin and character creator.Capture the Dragon or domination mode.
  • Improved GUI and interface. Customisable GUI. Statistics of killed Wartoks, Dragons.
  • Improve game mechanics. Maybe new animations for Rynn, she hasn't a lot of animation of sword Attack, she cannot even melee attack.
  • Add plugin/mods functionality, there are a lot of game based mainly on mods or addons.
  • The Ancient Gates feature in Drakan:OOTF engine. Sounds really nice, I always wanted to play in PC not using emulator :)
  • Improved OST:add fx such as compressor,equalisation and mastering.
So that are my ideas, maybe some of that will be implemented :)

This project reminds me Drakan 10th Anniversary by Shelim. He rewrite engine in DirectX9 -> so we will be able to add extra features, fx and fixing bugs. You can also make features as are in Drakan 10th Anniversary such as shaders,normal maps and post-processing effects :)

For me it is very good news because, it makes positive competition to Unity's Drakan Remake. I hope it really helps build it because it can make much more interest :) I'am not very talented programmer so I wish you luck. Of course I'll make special section for this project in Drakan Remake WordPress if you agree :)

UCyborg
Dragon
Posts: 433
Joined: Sun Jul 07, 2013 7:24 pm
Location: Slovenia

Re: OpenDrakan ~ A Drakan engine recreation

Post by UCyborg »

You said you wanted to do new level editor at one point. I wonder, do you think you'd be able to come up with better algorithm for calculating layer visibility? I'm clueless about how 3D stuff works in general, but hopefully, the current issue with the editor is just that it's optimized for speed rather than accuracy, which might be fine while you're designing a level, but the end result can require ton of manual corrections.

I was hoping to upgrade all stock levels to support larger drawing distance, but Ruined Village even in its unmodified state has tons of layer visibility bugs. I've corrected most of them, but my dedication and patience with this game aren't great enough for me to be willing to do it all over again plus fix tons of other bugs that are created when you initiate layer visibility calculation in the editor. Volcano level is also quite problematic and hasn't been upgraded, just some small visibility bugs fixed. The rest of the levels turned out pretty well, though Alwarren took some doing.

Visibility bugs are also very apparent in the Grotto, eg. without manual optimization, when you head into the cave guarded by a Giant where you encounter a bunch of Scavengers, the engine renders big chunk of the level that lies beyond that cave wall that you see if you noclip through with "floy" command.

But maybe these things have to do with using possibly incorrect layer types or something and calculations could be done more accurately if layers were setup differently? Though I doubt it, there's a document out there that says as level gets more complex, more work is required to be done in the Layer Visibility window.

Any thoughts about how sound will be handled in the cross-platform manner? OpenAL isn't widely used AFAIK, but it does let you do 3D sound positioning and gives you a library of special effects via EFX extension, which could be useful to re-implement EAX functionality in the original engine.

Then there's the thing with music, if I remember correctly, there are some ties to DirectMusic. There was some talk about extracting it from Music.rrc and playing it back in DirectMusic Producer. Another interesting thing to tackle I suppose.

An idea that I guess Surreal didn't have time to implement, judging from greyed out option in the editor; objects/NPCs in the levels spawning only at certain difficulties. So you could have more monsters in a level at higher difficulties. First thing that comes when it comes to extending current modding capabilities.

Multiplayer component in the Drakan is also very insecure and prone to exploits. It appears even collision detection is handled on the client-side, so you can noclip through objects on the level. And of course the ability to download levels from the server or external URL supplied by the server is pretty much a standard feature which Drakan lacks.

Just throwing these thoughts out there before they slip out of my mind, I realize it's too early for some of these things.
"When a human being takes his life in depression, this is a natural death of spiritual causes. The modern barbarity of 'saving' the suicidal is based on a hair-raising misapprehension of the nature of existence." - Peter Wessel Zapffe

HeckFluff
Whelp
Posts: 24
Joined: Sat Oct 14, 2017 12:47 am
Location: England

Re: OpenDrakan ~ A Drakan engine recreation

Post by HeckFluff »

calculating layer visibility
This might not be needed for performance. But if it's needed for correctness, a simple way would be to see if a layer's bounding box is visible from any point where the player can be in another layer. If the layers are spatially disjoint, it might only be necessary to test rays coming from one of more planes of a layer's bounding box.
sound
There's also FMod, which is used in AAA games, if nobody's fussy about third-party licences.
music
I think MSDN has the file formats. And because it seems the instruments are supplied by Drakan, this should be doable on Linux too. A minimal reimplementation of DirectMusic. Or maybe Wine has something.
more monsters
This can be a separate setting. "Increased spawns". This would be something I would turn up to 11. Just plop down more NPCs at the same position.

UCyborg
Dragon
Posts: 433
Joined: Sun Jul 07, 2013 7:24 pm
Location: Slovenia

Re: OpenDrakan ~ A Drakan engine recreation

Post by UCyborg »

This might not be needed for performance.
Perhaps not on the new engine, it would be beneficial for the old one though. Running Drakan through dgVoodoo2 certainly gives some speedup, especially beneficial when you increase the distance. At least with newer cards, it was a bit slower on my old Radeon 4890.

Also, I suppose the old engine does more things on the CPU than it could and pushing those to the GPU would speed things up?
There's also FMod, which is used in AAA games, if nobody's fussy about third-party licences.
(G)ZDoom projects have dropped FMOD at one point.
A minimal reimplementation of DirectMusic. Or maybe Wine has something.
Microsoft's DLLs are still needed for the music to play. And nobody tells you which DLLs exactly you should override. Installing DirectMusic with winetricks also overrides dsound.dll, which is not optimal. AFAIK Microsoft's dsound.dll falls back to waveOut functions on WINE. Which are as bad as the ones on Windows. :mrgreen:

I remember someone saying for the ancient version of WINE on WineHQ that Drakan worked smoothly on it. It was always choppy for me.The only things that I've tried and worked reasonably on WINE were certain OpenGL games. And even there are some odd cases; when I tried Call of Duty 1 about 3 years ago, I've had problems with mouse responsiveness. I saw the issue was mentioned back in 2007.
"When a human being takes his life in depression, this is a natural death of spiritual causes. The modern barbarity of 'saving' the suicidal is based on a hair-raising misapprehension of the nature of existence." - Peter Wessel Zapffe

HeckFluff
Whelp
Posts: 24
Joined: Sat Oct 14, 2017 12:47 am
Location: England

Re: OpenDrakan ~ A Drakan engine recreation

Post by HeckFluff »

By "reimplementation", I meant writing new code to deserialise the RIFF files so that DirectMusic would not be needed. MSDN has the file format at https://msdn.microsoft.com/en-us/librar ... s.85).aspx. I'm not sure if this would be difficult or not. It would be a fun thing to try out if I wasn't busy with anything else (I sense a side-project urge). The instruments are provided by music.rrc, so this should be viable. Or maybe Wine could be used somehow.

And I've looked into 3D analytic visibility. It should be fun if you think 4D/5D CSG is fun. I don't. As a fun exercise, 3D line-to-plane visibility is just 3D CSG: Give the plane a 3rd coordinate, which is the position on the line, making a large cube, and then subtract extruded occluders from the cube. The remaining parts of the cube determine where on the line the plane is visible, and which parts of the plane are visible. Extending to 3D plane-to-plane visibility gets you 4D CSG, I think. 3D box-to-plane visibility will reduce to that. Arbitrary 3D region-to-plane is probably 5D CSG. Alternatively, there's the shoot lots of rays from everywhere method.

UCyborg
Dragon
Posts: 433
Joined: Sun Jul 07, 2013 7:24 pm
Location: Slovenia

Re: OpenDrakan ~ A Drakan engine recreation

Post by UCyborg »

By "reimplementation", I meant writing new code to deserialise the RIFF files so that DirectMusic would not be needed.
I know, just wanted to say WINE's code probably wouldn't help, the relevant functions are still a stub.
"When a human being takes his life in depression, this is a natural death of spiritual causes. The modern barbarity of 'saving' the suicidal is based on a hair-raising misapprehension of the nature of existence." - Peter Wessel Zapffe

Zalasus
Whelp
Posts: 18
Joined: Mon Jan 29, 2018 6:50 pm
Location: Germany

Re: OpenDrakan ~ A Drakan engine recreation

Post by Zalasus »

UCyborg wrote: Sun Apr 01, 2018 6:31 pm I wonder, do you think you'd be able to come up with better algorithm for calculating layer visibility?
I already thought about that. The way the engine handles visibility is rather simplistic. For each layer it gives me a list of layers that are supposed to be visible from that layer. But that of course is highly dependent on where on that layer you stand and how far your rendering distance is. But since I want OpenDrakan to run with the original files and still fix graphics and gameplay bugs, the better route would be to ignore the visibility data from the level files completely and calculate it during runtime.
HeckFluff wrote: Mon Apr 02, 2018 11:58 pm And I've looked into 3D analytic visibility. It should be fun if you think 4D/5D CSG is fun.
That brings me to the next point. Drakan is a fairly old game, so back then precalculating the visibility certainly saved you a lot of loading time. But nowadays, on my fairly modern PC, I get away with rendering all the geometry in the level at once with more than acceptable FPS. When considering I don't really have an interest in supporting old platforms, I'm not sure I wan't to go through the hassle. What I'm probably going to do is render every layer that's within rendering distance. With the fog set at default distance, this should be easily handled even by older PCs.

But that highly depends on the amount of help I'm getting. Right now I'm still coding alone, so I have to consider what features are worth my time. But if somebody would volunteer to look into better methods for the whole visibility stuff, of course I would prefer those.

UCyborg wrote: Sun Apr 01, 2018 6:31 pm Any thoughts about how sound will be handled in the cross-platform manner? OpenAL isn't widely used AFAIK, but it does let you do 3D sound positioning and gives you a library of special effects via EFX extension, which could be useful to re-implement EAX functionality in the original engine.
I already looked into OpenAL and found it very easy to work with, so that's probably what I'm going to use. Unfortunately, FMOD is proprietary software. And given the fact that all audio in Drakan is either raw samples or DirectMusic it's probably overkill, too.

HeckFluff wrote: Mon Apr 02, 2018 8:50 pm A minimal reimplementation of DirectMusic. Or maybe Wine has something.
For the whole DirectMusic stuff, I thought I'd go with a reimplementation. A quick Google search already brought this up, so I think that's the go-to solution for now.

HeckFluff wrote: Mon Apr 02, 2018 8:50 pm This can be a separate setting. "Increased spawns". This would be something I would turn up to 11. Just plop down more NPCs at the same position.
That's a great idea. There's probably a ton of such enhancements that would add to the game without altering the authentic gameplay too much. I'll keep an eye on that :D


Right now my agenda is to implement collision detection and physics. I recently got animations running, so expect an update on that soon.

UCyborg
Dragon
Posts: 433
Joined: Sun Jul 07, 2013 7:24 pm
Location: Slovenia

Re: OpenDrakan ~ A Drakan engine recreation

Post by UCyborg »

Right now my agenda is to implement collision detection and physics.
One of the buggiest aspects of the original! :D
But nowadays, on my fairly modern PC, I get away with rendering all the geometry in the level at once with more than acceptable FPS.
One day, I might try and compile that thing on my own. Curious how my PC runs the scene you posted in the first post.

I personally find Drakan's draw distance the weakest spot for immersion. These days, you can do quite crazy things when it comes to graphics. And Vulkan is also a thing, though using that API requires quite hardcore understanding of 3D from what I've read due to letting developers manage all the gory details.

It'd be nice if you eventually got some help. Re-implementing a game like Drakan does seem like quite a challenge. Even folks that work on modern engines for Doom/Quake for which the source code is available keep finding things to fiddle with. And after how many years, Doom is from 1993.
"When a human being takes his life in depression, this is a natural death of spiritual causes. The modern barbarity of 'saving' the suicidal is based on a hair-raising misapprehension of the nature of existence." - Peter Wessel Zapffe

frabert
Hatchling
Posts: 1
Joined: Fri Apr 13, 2018 9:48 am

Re: OpenDrakan ~ A Drakan engine recreation

Post by frabert »

Hi everyone! I noticed an unusual spike of traffic from this topic on the GH page so I thought I'd pay a visit! :D

I'm really glad the library is being considered for use, if anyone can give additional insight, help or bug reports it'll most happily be accepted! It's currently a mostly one-person deal right now, any support would be of help. If you have any questions about how to integrate the library into the engine just ask!

Zalasus
Whelp
Posts: 18
Joined: Mon Jan 29, 2018 6:50 pm
Location: Germany

Re: OpenDrakan ~ A Drakan engine recreation

Post by Zalasus »

Hey everyone!

As promised, I'm gonna give you a quick update on the animation stuff I have been working on.

Image
Basically, all you can see here is Rynn in her idling pose. I'd post a video, but currently seeing it in motion is really not that impressive. The big thing to note is that Rynn has an actual pose coming from an animation in contrast to the Wartock to the left who is rendered just as stored in the model.

It took great effort to get it to this point, and I have learnt quite a bit about 3D programming in the process. I knew nothing about skinning, shaders and the like when I started, but writing the animation code tought me many skills I can put to good use when needed for the project.

The screenshot above is a bit older. Right now you only see Rynn from behind as I already implemented rudimentary controls for walking around. But the free-rotation mode of the original Engine will of course return at some point :D

frabert wrote: Fri Apr 13, 2018 9:58 am I'm really glad the library is being considered for use, if anyone can give additional insight, help or bug reports it'll most happily be accepted! It's currently a mostly one-person deal right now, any support would be of help. If you have any questions about how to integrate the library into the engine just ask!
I'm happy to have your support in that matter. Finding your library took a big load off my mind. I hope that it'll integrate nicely once I get to writing the sound and music code. I'll certainly give you feedback :)

Lastly I wanna share some thoughts on in-game physics. I'm planning to use Bullet (same library OpenMW uses for collision detection), which means I'll have a much more sophisticated array of features to pick from than the original Engine. But since I want to make OpenDrakan look and feel as close to the original as possible, I don't think I can take full advantage of that. Dynamics seem like a rarely used feature in the original game, mostly used for particles, debris from destroying buildings and the like. Most other times, like when killed Wartocks fling health potions in the air, in-game objects behave much simpler.
Yet I believe by using a proper physics library I can eliminate most of the collision-related bugs present in the original, so I think I'm rather gonna "dumb down" Bullet a bit to preserve the look-and-feel, even if it means I'm paying with performance for a seldom used feature.
Anyway, as I already said, this project doesn't target legacy platforms, so as long as you don't try to run OpenDrakan on your original Windows 98 machine, I don't think that choice really matters.

Post Reply