OpenDrakan ~ A Drakan engine recreation

[HeckFluff]

With Bullet, you'll probably have some "fun" using its built-in character controller. Unless they've improved it, you'll probably want to build your own. Such a character controller could also be used for other simple physics, like those health potions, if you don't want them to rotate or have much interaction with dynamic rigid bodies.

[UCyborg]

I compiled Windows build of opendrakan and got some really funky results. The engine crashed in Engine.cpp, line 56:

mViewer->getCamera()->getGraphicsContext()->getState()->setUseModelViewAndProjectionUniforms(true);

I had to disable my secondary monitor to get past that line (OSG bug?). Then the level loaded (went with Wartok Canyons for the screenshot) and things got super weird, solid colored polygons everywhere!



There's also this:

Error reading file C:\Windows\Fonts\arial.ttf: read error (Could not find plugin to read objects from file "C:\Windows\Fonts\arial.ttf".)

Got it when hitting one of the stats key (F1 or F2). Guess another dependency is needed?

I installed MSYS2, updated installation with pacman, installed needed software (CMake, MinGW, needed libraries) and instructed CMake to generate MinGW Makefiles. I used this guide as a reference to overcome some of the MSYS specific quirks, but other than that, it was only slightly more difficult than compiling it on fresh Ubuntu 18.04 installation.

Ubuntu repositories have the older OSG 3.4.0 while the newest is 3.6.0. I wasn't aware of the F1 and F2 keys when I tried it on Ubuntu, so didn't see that in action there.

Anyone has any thoughts regarding my observations?

BTW, shouldn't passing -DCMAKE_BUILD_TYPE=Release to CMake result in generated build files to pass the parameters telling compiler to optimize the final executable?

PS: Drakan now runs pretty smooth on Linux via WINE when using both dgVoodoo2 (with d3dcompiler_47.dll) and DXVK.

[Zalasus]

I compiled Windows build of opendrakan and got some really funky results. The engine crashed in Engine.cpp, line 56:

mViewer->getCamera()->getGraphicsContext()->getState()->setUseModelViewAndProjectionUniforms(true);

I had to disable my secondary monitor to get past that line (OSG bug?). Then the level loaded (went with Wartok Canyons for the screenshot) and things got super weird, solid colored polygons everywhere!

That's a really funky result indeed. The fact that things are colored at all and Rynn has a pose means that shaders are working. I think this requires a deeper investigation. Thanks for pointing that out
Also, I admire your patience to compile the engine on Windows. I had given up on that rather quickly, but I guess I should try again sometime. Up until now, I wasn't sure whether the engine would run on Windows at all.

UPDATE: I added a commit recently that might address the problems you faced. See the issue on Github.

Got it when hitting one of the stats key (F1 or F2). Guess another dependency is needed?

The rendering stats (F1) are broken on Linux, too, and have been for quite some time. OSG's stats handler doesn't mix well with shaders it seems. Another thing that needs fixing.

BTW, shouldn't passing -DCMAKE_BUILD_TYPE=Release to CMake result in generated build files to pass the parameters telling compiler to optimize the final executable?

It should. However, until recently we were overriding that variable to "Debug" no matter whether the user set it or not. That is fixed now

[UCyborg]

All is well now! The engine actually creates 2 windows, one for each monitor, with the primary one in exclusive mode. Which looks kinda weird if you don't have 2 identical monitors, but I guess that can be tackled later when settings are added.

Also, I admire your patience to compile the engine on Windows. I had given up on that rather quickly, but I guess I should try again sometime. Up until now, I wasn't sure whether the engine would run on Windows at all.

Thanks, it was surprisingly smooth process. I would say cross-platform nature of the engine helps a lot.

I forgot to ZIP the shaders with the engine binaries last time, so it wouldn't work properly if someone else downloaded it and gave it a go. They're included now.

[UCyborg]

Wunderbar! The performance is a non-issue in this engine, even when ignoring visibility data and having a level with too much crap in it. The second most problematic level in that regard (Paradise) runs at 119 FPS with uncapped frame-rate in worst case scenario on my end, while the original engine chokes at about 10 FPS. Just Atlantis runs only at 3 FPS, which should climb back to normal rates after after adding fog (since everything is rendered ATM).

And another unimplemented feature in the Riot Engine; demo recording. There's one string in Dragon.rfl that says "Finished Demo Playback". There were supposed to be two more key commands available; in Dragon.rrc, the strings with ID 0x4004 and 0x4005 say: "Record Demo" and "Play Demo". I knew about the string in Dragon.rfl, but the latter two are new to me.

Oh, and the GameSpy authentication key is also read from Dragon.rrc: string 0x2142.

[Zalasus]

Wunderbar! The performance is a non-issue in this engine, even when ignoring visibility data and having a level with too much crap in it. The second most problematic level in that regard (Paradise) runs at 119 FPS with uncapped frame-rate in worst case scenario on my end, while the original engine chokes at about 10 FPS. Just Atlantis runs only at 3 FPS, which should climb back to normal rates after after adding fog (since everything is rendered ATM).

Wow, that's incredible, given that OpenDrakan is only single-threaded yet. I'm sure the visibility calculations and multi-threading become more important when I add more intensive calculations like shadow mapping, but your results are nontheless great news

And another unimplemented feature in the Riot Engine; demo recording. There's one string in Dragon.rfl that says "Finished Demo Playback". There were supposed to be two more key commands available; in Dragon.rrc, the strings with ID 0x4004 and 0x4005 say: "Record Demo" and "Play Demo". I knew about the string in Dragon.rfl, but the latter two are new to me.

Oh, and the GameSpy authentication key is also read from Dragon.rrc: string 0x2142.

I'm sure those features are easily implemented. Quick googling brought up a few examples how to capture frames with OSG and encode them with ffmpeg.

That particular GameSpy string is not found in my Dragon.rrc (the german version), but there are others that look like a key for something.

May I ask you how you know the strings from the file? In my version, most strings are XOR encrypted with a key I could not determine in it's entirety yet. So either you know more than I do, or you version simply is not encrypted. Would be glad to know since some of those strings are important for the UI. I already created an issue on GitHub for that, so feel free to comment. I would be very grateful

[UCyborg]

Wow, that's incredible, given that OpenDrakan is only single-threaded yet. I'm sure the visibility calculations and multi-threading become more important when I add more intensive calculations like shadow mapping, but your results are nontheless great news

For the record, the CPU is AMD Phenom II X4 920, clocked at 3,00 GHz from stock 2,80 GHz, year 2009, not exactly the speed demon, but it's from the era when AMD was getting better.

As for GPU, NVIDIA GeForce GTX 750 Ti, year 2014, its selling point being power-efficiency rather than performance. Though GPU doesn't have much work to do when running OpenDrakan.

I'm sure those features are easily implemented. Quick googling brought up a few examples how to capture frames with OSG and encode them with ffmpeg.

They'd be nice indeed. But I think what Surreal actually meant with demo recording was Doom/Quake style demo recording, in which the engine just stores series of commands sent to the server from the client (in those engines, singleplayer mode runs a local server) in a very compact file. So you get "recording" which can be played-back in the engine in which it was created, or any engine that is compatible with the protocol. Quake Done Quickest speedrun compilation packed in Quake's PAK archive weights a little less than 14 MB and about a little less than 5 MB when zipped.

Of course, some developers of modern source ports then also add the ability to record the gameplay the "normal" way, that is recording the actual image, which is what you're referring to in your comment.

That particular GameSpy string is not found in my Dragon.rrc (the german version), but there are others that look like a key for something.

Oops, I made a typo, the actual string ID is 0x2412 and it says: zCt4De

May I ask you how you know the strings from the file?

Actually, for the GameSpy string, I suspected it must come from some file since the game seems to pull it out of nowhere. I first encountered it when debugging the server code that responds to queries.

In my version, most strings are XOR encrypted with a key I could not determine in it's entirety yet.

They're encrypted regardless of language it seems.

So either you know more than I do, or you version simply is not encrypted.

Just yesterday, I found the function in Drakan.exe in which the decryption takes place.

I already created an issue on GitHub for that, so feel free to comment. I would be very grateful

Already did!

[UCyborg]

Something that just came to mind, Drakan has some weird bugs tied to frame-rate. Ideally, the new engine should be frame-rate independent. You don't want effects like flame on the Sword of Flame to just die when run on the shiny 144 Hz capable monitor.

Then we come to physics. Rynn can jump farther at higher frame rates. I guess the way it was meant to be played is how she jumps at 60 FPS. I also recall some odd behaviors when shooting NPCs with a bow with regular arrows. I didn't have debug messages on at the time, but it looked like the game behaved as if NPC was hit multiple times when hit once at particular spot, eg. the empty area between body and arms. I remember being able to gib a Scavenger at one point using a single arrow (with 10 damage) when he was still alive.

The most interesting bug related to frame-rate would be how much NPCs bleed. Things are consistently gorier at 20 FPS, although the same amount of blood can be observed under certain circumstances at saner frame rates. I wonder if the behavior you get at 20 FPS is the way it was meant to be.

Oh, and UI scaling is a must feature. Drakan's UI is practically useless at high resolution on a dense display. Size of lens flares are also dependent on the resolution.

[Zalasus]

Something that just came to mind, Drakan has some weird bugs tied to frame-rate.

Ah yes, that old chestnut. Quite a few games had this issue back in the day. How you could program things to be dependent on FPS on purpose is beyond me, so this must have been due to the inexperienced Surreal programmers or simply due to them cutting corners.

At least that is something I don't have to worry about in OpenDrakan. Everything I have written up until now is synchronized with real time. The update hooks in the new engine provide absolute time and the time passed since the last update, so it should be no problem to write things properly. The gore and flame stuff is most likely due to some quirk in the particle system, but that's another thing I (or a contributor~ fingers crossed) will have to figure out someday.

Oh, and UI scaling is a must feature.

That's actually the thing I have been toying around with lately. What bothers me is the fact that the UI textures are relatively small. The main menu is a texture with a relevant area of maybe 380*460px in total, which will look either pixelated or soft when scaled up to fit a modern screen.

But even worse, the bitmap font used by Drakan has glyphs 17px high, which is definitely too small for any modern screen. The obvious solution is to ditch the bitmap font and use a vector font instead, but unless I can get my hands on the original font they used to make the bitmap, that would be really unauthentic.

What do you think? I thought I'd add an option to scale up the UI, but map it 1:1 to screen pixels by default. The latter would be called 100% scale and for anything above that you'd just have to live with a pixelated or fuzzy UI.

[UCyborg]

What do you think? I thought I'd add an option to scale up the UI, but map it 1:1 to screen pixels by default. The latter would be called 100% scale and for anything above that you'd just have to live with a pixelated or fuzzy UI.

Sounds good. You can't do much else from the engine's standpoint anyway. Ideally, someone would make higher resolution textures for UI.

If I use 1280x720 as a base resolution and set dgVoodoo to upscale it to my monitor's native resolution 1920x1080, things are already fuzzy. Upscaling to 3840x2160 and letting the driver downscale it (NVIDIA Dynamic Super Resolution) smooths it out and it does look relatively good considering low-res nature of the original resources.

No idea how it would look on an actual 4K screen...

The obvious solution is to ditch the bitmap font and use a vector font instead, but unless I can get my hands on the original font they used to make the bitmap, that would be really unauthentic.

You're right. So for now, there are only 2 options; use original bitmap font which is pain to read, or another vector based font that isn't very Drakanish.

[Mechanist]

Hmm... here's some intel I gathered as a side-effect of reverse-engineering the Level Editor.

The 16 bytes that control the texture orientation on the layer tiles appear to be some form of coordinate mapping scheme.
There's always 1 each of the following values: 000000, 0000FFFF, FFFF0000, FFFFFFFF - just arranged in a different order.

Although I didn't gather the values for ALL the possible combinations, this should be enough to see the pattern emerge. NOTE: This is NOT adjusted for any byte order! Exact same as you would see in a hex editor.

_________________________1__________2__________3_________4
All transforms deselected: 00 00 FF FF FF FF 00 00 00 00 00 00 FF FF FF FF
Rot. L: _________________ FF FF FF FF 00 00 00 00 00 00 FF FF FF FF 00 00
Rot. R: _________________ 00 00 00 00 FF FF FF FF FF FF 00 00 00 00 FF FF
Flip H: __________________ FF FF 00 00 00 00 FF FF 00 00 00 00 FF FF FF FF
Flip V: __________________ 00 00 FF FF FF FF 00 00 FF FF FF FF 00 00 00 00
Flip H+V: ________________ FF FF 00 00 00 00 FF FF FF FF FF FF 00 00 00 00

I tried making some sense of this, but I don't know anything about how the texturing is handled by the engine, so I was unable to reach any meaningful conclusion here.

[Zalasus]

I tried making some sense of this, but I don't know anything about how the texturing is handled by the engine, so I was unable to reach any meaningful conclusion here.

I already checked out all of these different combinations a while ago when writing the layer code, and back then I also couldn't get rid of all issues. I already guessed right that the engine reads each of those 16-bit integers as U and V coordinates, where 0x0000 means 0.0 and 0xffff means 1.0 in texture space. What I couldn't quite figure out was which coordinate mapped to which corner of a layer tile. After some fiddling I found that the order top-left, bottom-right, bottom-left, top-right worked most of the time, but some textures kept getting messed up.

Your post made me review that once again, and guess what: My error was not only the order of the corners. It was that I assumed the engine would read them as UVUVUVUV, when in fact they are stored as UUUUVVVV! One quick change and now layer textures seem fine everywhere. So thanks for the heads-up

I find it extremely amusing that using the erroneous coordinate scheme still worked in most places. I'm sure there's a mathematical explanation for that

[Mechanist]

I've been digging straight down into the math involved in calculating the angle of a slope (for the purposes of determining whether Rynn will walk or slide on any given terrain triangle), and I have to say this is most bizarre.

But first, let's take a loot at the math involved before jumping to any conclusions.
Also, a note about the axis labelling convention used: the math involved assumes an XZY axis order (where Y is the "3D vertical", or "up/down" axis - the one along which the gravity is acting), the same as used by Drakan.
No idea why they didn't use the much more obvious (and sane) XYZ axis order, though.

Let's start off with a triangle. This one, to be specific:

Triangle.png (2.6 KiB) Viewed 2848 times

Presumably, there are many ways to tackle this, mathematically speaking. But arguably the simplest way is to use the surface normal vector for this purpose: the Y component of the (normalized) normal vector is simply the cosine of the vector's angle from vertical.
Taking the arccos() of the Y component gives us the angle of the slope from horizontal (in radians).

So let's take a look at what is involved in calculating the normal vector:

Math1.PNG (21.21 KiB) Viewed 2848 times

Well, that's not good at all!

But wait! We don't need to calculate the normal vector for an arbitrary triangle in 3D space.
For this purpose, the XZ coordinates of the vertices always lie on a regular grid (1 lu = 2048 wu apart).
Also, we don't care about the vector's XZ orientation either, only its angle from vertical (the Y axis, in this case).

So let's substitute the XZ coordinates of the vertices with fixed values, and see if that helps:

Math2.PNG (11.54 KiB) Viewed 2848 times

Much better!

Note that obviously I've assumed the wrong order of the vertices, since the normal vector now points the wrong way. But this is easily fixed by just inverting the sign on the Y component - and we don't care about the XZ components anyway.
Also just for fun, let's see how it looks like when rescaled to wu instead of lu.
So let's take a look at the final form of this equation:

Math3.PNG (1.93 KiB) Viewed 2848 times

That's very much tractable now
Note that I've gotten rid of the arccos(), the square root and the constants - they have no bearing on the nature of the result, only acting as a "scale factor" which can be factored out.
This is possible, because we don't care about the value of the angle in degrees (or radians) - we just need a value that is a monotonic function of the normal vector's angle from vertical, so that this value can be compared against a fixed constant.

But actually, we can do better still... let's go back to the non-simplified form from before, and carve it down to the essential part:

(y1 - y2)² + (y1 - y3)²

This is much better, since it involves only 3 additions/subtractions and 2 squarings, instead of 4 additions, 2 multiplications and 3 squarings.
There's also another advantage to doing it this way... using heights in wu (which are integers) allows foregoing the use of floating point numbers entirely: the result will always be an integer if given integer arguments!

Really, it doesn't get any simpler than that

------------------------------------------------------------

And now, we get to the truly bizarre part...

Since we now know how to calculate the angle of the slope, let's see if we can determine the basis for the walk/slide cutoff.

We could make an a priori assumption that it was done by calculating (y1 - y2)² + (y1 - y3)² with wu heights as arguments (for performance reasons - fewest operations, and no floats), but that's probably giving the devs too much credit.
Instead, let's explore the various possibilities in turn.

First up, what's the limit of a walkable triangle?
Without beating around the bush, I'll just give the answer straight away.
Assuming a triangle with the vertices labelled as shown above (and heights in wu), the steepest possible walkable triangle has the values:

• y1 = 956
• y2 = 1896
• y3 = 0

Making it any steeper (eg. y1 = 958, or y2 = 1898) makes it unwalkable.

So this corresponds to an angle from horizontal of 33.21074 degrees (0.5796368 radians) for the steepest walkable triangle, and 33.21127 degrees (0.579646 radians) for the least steep unwalkable one.

Plainly speaking, these values are nuts. I can't even begin to imagine what the underlying value was that was placed in the source code. It's not related to any rational fraction involving pi - the first close match is 12 * pi / 65, but even then it falls just outside the range of interest.

If we consider the (y1 - y2)² + (y1 - y3)² form, then the cutoff lies somewhere between 1797536 and 1797608.
That's not even correspond to any sane rational fraction based on the powers of 2.
No idea how they came up with these values, it seems totally arbitrary.

I find it hard to believe that someone on the dev team just randomly decided these numbers, yet I can find no rational explanation for them

[Mechanist]

Here's some very preliminary intel on how the dedicated server stores player data in memory.

First, how to easily find these structs:

MOV EBX,DWORD PTR DS:[68885664]
MOV EBX,DWORD PTR DS:[EBX+1AC]
MOV EBX,DWORD PTR DS:[EBX+4E] ; In the struct that EBX now points to, the # of connected players is stored at [EBX+8]
MOV EBX,DWORD PTR DS:[EBX] ; EBX now contains the pointer to the server's struct (dedicated server = "fake" 1st player)

More specifically - the pointer obtained above points to the 1st entry of an array of the player structs.
The player structs are exactly 0x09DA (2522) bytes long.

So to locate the structs for the other players, add (2522 * 0-based player ordinal #) to the abovementioned pointer.

NOTE: the player ordinal # is not the same thing as the player ID... or maybe it is? This requires further investigation.


Meaning of known fields in the player struct:

• (+0x003A) Player ping value, in miliseconds (DWORD),
• (+0x079C) Pointer to a struct which contains the player's IP in ASCII form (at +0x4A) and in binary form (at +0x5A),
• (+0x07B8) Zero-terminated string containing the player's name. Max 27 characters allowed (not counting the null terminator), but IIRC 32 bytes are allocated for the name string; likely due to data alignment considerations,
• (+0x07D9) Player kills count (DWORD),
• (+0x07DD) Player deaths count (DWORD),
• (+0x07F9) Amount of time elapsed since the player has connected to the server; measured in seconds, stored as a 32-bit float.

[Zalasus]

Hey Everyone

Since I'm currently dealing with a severe case of coder's block, the least I can do to contribute to the project is to give you some updates on what I've done so far. Sorry for neglecting those who don't frequent the Discord server in regards to updates on OpenDrakan.

Lighting works to some extend. A few things still need polish here, but overall the look and feel of OpenDrakan's lighting gets close to the original Riot Engine one:



Also working: The way the Riot Engine blends lighting between adjacent layers:



Apart from that, I've implemented a somewhat working version of Drakan's main menu (not including any submenus yet):



The menu still needs a bit of work. The crystals are actually textured in grayscale with a color filter applied afterwards. I still haven't figured out the exact colors used by Drakan. Also the lighting of those crystals is a bit off.

What you can't see in the pictures: The basics of sound output works (not including music, but including sound emitter objects and menu sounds, pitch shifting etc.).

The project needs to undergo a few radical changes in the near future, like seperating rendering from game logic, introducing a client-server architecture etc. Unfortunately, I couldn't settle on a good way to implement the needed changes yet, so for now I'm being unproductive. I hope that changes soon.

Thanks for your (continued) interest in my project

Regards,
Zal