Page 6 of 24

Re: Widescreen hack and some other fixes aka AiO Patch

Posted: Sat Jul 02, 2016 1:38 am
by UCyborg
Figured ignoring the level property Max Fog Depth might not have been the best idea, given that this property can intentionally be set to a lower value to have a foggier level. Upper limit is still 60 while vanilla Drakan has it at 30 and will ignore higher values. Min Fog Depth that is allowed has been reduced from 12 to 5. Permanent 'foghack' is still an option in config file. Of course, when creating levels, fog depth is just one parameter, tweaking NPCs to detect player from farther distance is pretty important too. And there's the extra setting for changing FOV multiplier.

Re: Widescreen hack and some other fixes aka AiO Patch

Posted: Sat Jul 02, 2016 12:35 pm
by mage150
UCyborg wrote:Only added 2 minor fixes for the Level Editor 3D Viewer I forgot about last time, on modern Windows, there is a strange glitch with windowed mode, window sort of flashes every once in a while and when it loses focus, for some reason Windows considers it non-responsive after 5 seconds. Workaround is disabling window ghosting feature. Already took care of that for the game with Windows 8 update of the patch so it's only fair we do the same for 3D Viewer. The second fix is identical to the one that resolves crashing on difficulty screen in the game, probably doesn't do much for the 3D view, but that code is nevertheless executed and it might as well be there, if only for consistency.

Interestingly, the glitch related to the windowed mode (in which 3D view works as well) does not occur when we run the game through dgVoodoo. But dgVoodoo still needs some work done with its fast video memory access option, which, when the author succeeds and makes it work perfectly, should make Drakan run through it perfectly (no slower mouse response and degraded performance with debug messages enabled in developer options). It also doesn't work with Level Editor yet, probably because of mixed DirectDraw + GDI usage.

When I thought the game somehow corrupted its settings that time, it was actually dgVoodoo and enabled debug messages. No wonder resetting settings and NOT turning on those messages fixed it...

mage150, hope you managed to get your game running, couple of nights have passed by now. ;) I'm surprised myself about how many issues have been taken care of by randomly hacking inside game executables.
I've been distracted by Rocket Leauge. I will probably have some chance to try this weekend with the extra Holiday time! Thank you for your updates.

Re: Widescreen hack and some other fixes aka AiO Patch

Posted: Sun Jul 03, 2016 11:45 am
by UCyborg
I've addressed the small memory leak yesterday caused by original "patch" for 3D view crashing on Windows 2000 and above. The author simply removed the call to UnmapViewOfFile without addressing the actual issue, which was that Surreal needed the single variable from the memory block which was being freed prematurely. Somehow, access to that block was possible even after being released on Windows 9x, even though the documentation clearly says UnmapViewOfFile unmaps the mapped view from process' address space, which is exactly what happens on NT family of Windows.

With the original patch, the memory block remained visible on the memory map, even though no actual data except for that single variable was needed from it afterwards. In addition, Microsoft's docs say to completely free everything related to mapped file view, the view must be unmapped and CloseHandle called on the file mapping object. Only CloseHandle was called with the original patch.

The final ZIP has the number 2.19, in case anyone grabbed anything older and bugged, I've uploaded it 3 times yesterday. So that's it for now. Next time I'll be doing anything related to Drakan, I'll be PLAYING it! 8)

Re: Widescreen hack and some other fixes aka AiO Patch

Posted: Sun Jul 03, 2016 9:41 pm
by mage150
The game works! I was able to set my keybindings and start a game normally with a completely fresh install. I made a backup of this working version so I can tinker now without the risk of this breaking.

It's surprising how well this engine has scaled to modern hardware. The lighting is still really good looking, just low quality textures. I can't wait to play through the game again!

Re: Widescreen hack and some other fixes aka AiO Patch

Posted: Mon Jul 04, 2016 1:24 pm
by UCyborg
Great! It's really easy to break something when tinkering with it. When I was trying out various player-created levels, one level just refused to work in multiplayer mode (Apex). If you setup dedicated server and join it just crashes. There's one really beautiful level IMO (forgot the name...), which doesn't have any significant problems, but if you try hard to fly the dragon outside the map, crash-bang.

Agree about lighting and textures. AFAIK, the original textures even have only 16-bit color depth and the maximum allowed size iz 256x256 (didn't check every single texture, but they seem mostly 64x64 in size) and not really any support for any other real format besides plain bmp. Though I'm not sure if this affects anything else, besides the storage requirements. Someone made a texture pack for stock Hill of Fire level, in its current form only compatible with 10th Anniversary Mod.

One important issue about textures, this engine is supposed to have some form of mipmapping, which I might have accidentally broken with patch that resolves crashing on difficulty screen and black menu background on modern hardware. With quick glance at the stock texture databases, this feature doesn't seem to be utilized. But if mipmapping is the answer to performance problems when attempting to use bigger textures and it actually works, it might even be feasible to increase default texture size limit of 256x256. It's really easy to trick the engine to accept bigger textures, but then performance becomes subpar. Will investigate this further at some later point in time. Having native support for bigger textures would certainly be awesome, but who knows, there might be other obstacles in the way.

Re: Widescreen hack and some other fixes aka AiO Patch

Posted: Tue Jul 05, 2016 3:40 am
by mage150
If memory serves me correctly mipmapping was available for LOD textures on animated objects like NPCs Rynn and Arokh. Besides that it was never available. I think devs specifically mentioned it was practically a defunct engine feature.

Re: Widescreen hack and some other fixes aka AiO Patch

Posted: Fri Jul 08, 2016 12:48 am
by UCyborg
Here comes the texture update! I've increased maximum texture size to 1024x1024, imported Mauva Malhar's textures into the game databases and it worked! There doesn't seem to be any performance issues with 512x512 textures, even on low-end laptop. Check this thread. It's just that 10th Anniversary Mod is inefficient even when you don't add any extras on top of it.

I did actually break the mipmapping feature, it's implemented and seems to work if you create mipmaps in the editor, just wasn't used anywhere, otherwise, you'd see black textures in game. Fortunately, I figured out a better patch so both mipmapping and background in menus work. Didn't know about Surreal mentioning about the mipmapping, but one similar thing that really didn't work properly was the LOD feature that reduced poly count of models when away from the camera, which includes Rynn, Arokh, NPCs, trees etc. It's mentioned in the OP and is only noticeable when you increase FOV.

Anyway, here are the textures. Just update the game and they'll work. Perhaps a full retexture of the game with somewhat less restrictions might be possible, it'll just cost extra RAM. The preview of larger textures doesn't work in Level Editor's Databases window. And one more funny thing, normal truecolor textures are considered 16-bit for some reason. I thought that was kinda odd that they'd be 16-bit, I clearly remember back on Windows 7 and earlier, if you set 16-bit color mode, colors get worse.

Re: Widescreen hack and some other fixes aka AiO Patch

Posted: Fri Jul 08, 2016 1:24 pm
by mage150
Even if no one uses it, I think I might take a shot at an HD mod. I have an artist available. I still remember how to texture map in the modeler.

Dude you are fricking amazing.

Re: Widescreen hack and some other fixes aka AiO Patch

Posted: Fri Jul 08, 2016 7:07 pm
by UCyborg
Drakan in HD would be pure awesomeness! It seems there are some folks lurking on these forums, so who knows, some might be interested. I wonder if that guy stopped his retexturing efforts only because he saw things don't run very well with 10th Anniversary Mod.

I did only little experimenting, spamming whole Mountain World Common TD database with 1024x1024 textures and loading up Wartok Canyons. RAM usage increased to around 800 MB and frame-rate was chopped in half. Would be interesting to see the results if textures in remaining databases used were replaced like that. Difference in file size between 512x512 and 1024x1024 is 768 KB vs 3 MB, so it quickly adds up.

It seems the game keeps most of the textures in regular RAM instead of video RAM, I guess that made sense in the 90s when video RAM was scarce. We can only go so far by tweaking numbers in the game executable. But not everything is in size.

It's really a shame how Drakan was just abandoned like that. I haven't played Daikatana, but I've read about it some time ago. Overhyped, not well received, commercial failure. And couple of years later, its code was given to people knowing their way around these things, who fixed tons of bugs and made a Linux and Mac port.

Re: Widescreen hack and some other fixes aka AiO Patch

Posted: Tue Jul 12, 2016 3:44 pm
by UCyborg
I did some testing during the weekend and have some good news and some bad news about the editor. The bad news is that the editor is still fundamentally broken ever since Windows 2000. From what I could gather from disassembled code, it employs so called memory mapped files for data sharing between the editor application and the accompanying 3D view application. Sharing and manipulating data across processes requires some sort of synchronization mechanism, this is where the problem is. It utilizes standard window messages and in some places the blocking function SendMessageA to keep things synchronized, which appears to be the culprit.

Looking up MSDN documentation, this technique isn't mentioned for synchronization, but specific synchronization objects are. I'm assuming if one of those objects were employed, the application might survive the transition from from DOS to NT based Windows.

With the current design, some messages simply don't make it to the other process, resulting in glitches and crashes. Even process affinity affects the crash frequency. No compatibility fixes related to thread scheduling remedies the problem. Heck, running on single core makes it crash sooner. Either way, it's all pretty messed up and dependent on random factors, which can be observed by trying multiple Windows versions and loading different levels. And since communication between 3D view and the editor is the most important and basic function, you just can't afford to have buggy communication between them. Though from my messing around with it, crashes usually happen when it comes to sharing information about models between the two processes, so you usually observe missing models in the 3D view. Depending on the circumstances, you can crash as soon as you put the Camera Tool on the map, when you fly around the level etc. Even if it doesn't prevent you from creating working level, the glitches that shouldn't be there are there. I also noticed in Process Hacker a handle to file mapping object rexStompTime that appears and stays there each time you stop the STOMP sequence, I suppose it should also be released, unless that's a separate bug.

The good news is, since my previous patch for the 3D Viewer executable, the Level Editor is now usable for the first time in WINE! Which means you could put a Linux distribution like Ubuntu on the PC, install WINE and run the editor through that. Communication between the two processes works flawlessly there. So giving WINE a shot might be worth a try. Apart from a visual glitch with texture previews, they render below the window on the main surface, I think there's a very good chance that there aren't any function breaking bugs. It certainly behaves nicer at the first glance than it does on Windows, I can fly around the map, put the camera at random places, play STOMP sequences, all these in almost any order, and no crashes and missing models. I've added registry file with the correct settings for the game and the editor, since not every setting can be set in GUI, if anyone wants to experiment.

It would be ideal if it just worked properly on Windows, but the fact that it works better in WINE now shows we're certainly heading in the right direction. Unlike Windows 98, Linux can actually run on today's hardware.

Other than that, I've updated the editor Texture View Options dialog to allow preview sizes to be set to 1024 and resolved another obvious resource leak, handles to 3D Viewer's process and thread weren't closed when the viewer was closed.

So that's it, do as you wish with the updated binaries and information, I won't be digging into this game's mysterious code any time soon, if at all.

Re: Widescreen hack and some other fixes aka AiO Patch

Posted: Tue Jul 12, 2016 10:44 pm
by UCyborg
Right, forgot to mention, the issue with texture previews occurs since Windows 8. If their original size is greater than the preview square, scaling down doesn't work properly, ends up from barely recognizable to pitch black.

And did you know the actual max players limit on dedicated server is 64? Really, if you use -maxpayers parameter, it goes that high. The network programmer must have thought about the future. :mrgreen:

Another funny fact, before my patch for Session creation failed error, servers could be hosted by specifying -ip 0.0.0.0 on command line. Though there would still be problem with NAT.

Re: Widescreen hack and some other fixes aka AiO Patch

Posted: Wed Jul 13, 2016 12:28 am
by mage150
UCyborg wrote: And did you know the actual max players limit on dedicated server is 64? Really, if you use -maxpayers parameter, it goes that high. The network programmer must have thought about the future. :mrgreen:
Yes. There used to be a dedicated drakan server launcher. You could set it in the command line. It was pretty taxing on a machine to run drakan beyond 5-10 players.

Re: Widescreen hack and some other fixes aka AiO Patch

Posted: Mon Jul 18, 2016 12:20 am
by UCyborg
Woo! I took another look at editor crashing problem and as it turns out, there's really nothing wrong with keeping things in sync using standard window messages. The actual problem were RECURSIVE calls to SendMessage, which in practice, only served to fill up stack space in Windows kernel and when it filled, messages stopped coming through.

Stack space is very random variable which will differ from system to system, is not under our control and we don't care about it anyway. This is why it would crash at different points on different systems. Just one of those bugs that didn't show up immediately in its time I guess.

Knowing this, I was able to implement really elegant solution, replacing SendMessage with SendNotifyMessage (non-blocking SendMessage in this case) where code flow leads to recursion, which seems to be the case in parts dealing with models and characters, judging by spotted keywords.The final result is much better functioning editor that doesn't crash right and left and displays everything that's supposed to show up in 3D.

Re: Widescreen hack and some other fixes aka AiO Patch

Posted: Tue Jul 19, 2016 12:55 pm
by mage150
UCyborg wrote:Woo! I took another look at editor crashing problem and as it turns out, there's really nothing wrong with keeping things in sync using standard window messages. The actual problem were RECURSIVE calls to SendMessage, which in practice, only served to fill up stack space in Windows kernel and when it filled, messages stopped coming through.

Stack space is very random variable which will differ from system to system, is not under our control and we don't care about it anyway. This is why it would crash at different points on different systems. Just one of those bugs that didn't show up immediately in its time I guess.

Knowing this, I was able to implement really elegant solution, replacing SendMessage with SendNotifyMessage (non-blocking SendMessage in this case) where code flow leads to recursion, which seems to be the case in parts dealing with models and characters, judging by spotted keywords.The final result is much better functioning editor that doesn't crash right and left and displays everything that's supposed to show up in 3D.
So what you are saying is, I could probably make a new Drakan map with minimal editor issues? I might actually give it a shot.

Re: Widescreen hack and some other fixes aka AiO Patch

Posted: Tue Jul 19, 2016 5:05 pm
by UCyborg
I think so, before this update, sometimes, you were lucky if you could put camera on the level to view it in 3D without crashing. If that seemed to work, you could crash when you were flying around the level. The funniest result of this glitch I saw was watching the cutscene in Wartok Canyons when Rynn talks to the farmer and the house was missing xD

It might not be problematic at beginning when you have very few objects in the level, but as the level grows, these glitches are bound to show up eventually. And like I said, it's pretty random and dependent on Windows version and possibly other variables beyond our control. Either way, the way it was done originally was pretty silly when you take a closer look, so the new patch logically can't have any bad side effects, it just eradicates probably the worst bug that ever found its way in the editor.

I'm surprised that these specific crashes weren't mentioned on these forums. Plus the original patch that was made to address separate issue with the 3D Viewer crashing on Windows 2000 and above wasn't implemented properly, so if there was odd scenario that could make the program crash because of it, that's also the thing of past now.

A bit late, but better now than never! :D

Here's the example of that bug in action:
Image