MM 6/7/8 Total Remake

[Nomad]

Hello there.

Contacts:
skype: alex.g.parshin
facebook: https://www.facebook.com/mm7engine
code: https://github.com/gp-alex/world-of-might-and-magic

Current status:WE ARE SEARCHING FOR 2D ARTISTS TO DO TEXTURING WORK






Today I would like to present my project.
For about four month now, I am working for a project on a decompilation, reconstruction, upgrading and tuning of Might and Magic 6/7/8 engine. As a base, I've taken an original MM engine (7th) and decompiled it. But the result of decompilation is very unstable, all data structures are messed, so I analyze the code, find the dependencies, correct the errors, and then rebuild it. The difference is when rebuilding, I can add any features I like, even those not supported by the engine. This is a big work, so I reconstruct and add the features one-by-one. The latest stable version includes:

* all the animations, videos, sounds and music are played from original dataset (*.vid and *.snd files) using the engine.



* Decompression and parsing of LOD files
* Loading and rendering RGB, Transparent and palette textures.
* TGA and PNG textures, data override folders
* DirectDraw4 render (DirectDraw2 support was cut due to huge amount of excess code)
* Input using DirectInput8
* Font rendering (except little issues)
* First game state (MainMenuUI) is operational:




* Second game state (PlayerCreationUI) operational.




* Loading Outdoor maps (*.odm), actors (creatures), icon/texture frame tables
* First touch to events (*.evt)



Currently working on first in-game render and parsing world management/update code:



* 12.06.2010 Minimap and in-game GUI are fixed!


* 15.07.2010 3D Render on it's way, weird colors are because palettes are not set up correctly



*



* 29.09.2010 Actors (NPCs, monsters), objetcs, dropped items and other placeables are rendered and animated. Little problems with palette colors are being solved (getting rid of the palettes actually). Interactable actors are on the way.




* 30.09.2010 Colors are finally fixed.


* Walking, colliding, jumping, falling, drowning. Inventory.


* 05.12.2011 Terrain is finally rendered, many thanks to mkienenb and his mm8leveleditor sourcecode.
First playable demo will come shortly after.




My goals here:
(1) Fully reconstruct the original engine.
(2) Update render to modern Direct3D 10 graphics maintaining back-compatibility with mm6/7/8 resources.
(3) Create a base for the community which can be enchanted by everyone willing to contribute, e.g. keep the engine open-source for programmers and provide various flexible tools for artists and modders so anyone can create his own adventures in the world of Enroth.
(4) Additionally, there are some ideas for multiplayer.

Current project status:
stage (1) ~35% completed ( 70 000 / 193 000 lines of code parsed)

Expected feedback:
* Ideas, wishes, support.
* Strongly required C/C++ programmers with basic knowledge in assembler.
* 2D artists

[mkienenb]

Sounds like a great project! I can't help at this time directly, but but I might be able to answer questions on data structures once in a while if you get stuck on something.

-Mike
MM8LevelEditor / revised MMLevelEditor

[phamlongtuan]

Love to hear something new like this , i'm just normal player so i'm here to cheer you guy

[tolich]

@Nomad [Feaerenil] (:)): it's good to see your progress.

Why you still don't remove these blue/black borders from the icons? You're probably to generate alpha-channel for icons you load.

Also, welcome there.

[Nomad]

mkienenb
You're the author of MM8 Level editor? It wuld be cool then to take a look at headers. I know all the fields of the data structures, but many of them are of unknown use to me. In general, now it looks like this:

Code: Select all

struct ODM
{
 unsigned char  string_000000[32];
 unsigned char  pFileName[32];        // byte[32] @ 000020
 unsigned char  pFileDescription[32]; // byte[32] @ 000040
 unsigned char  string_000060[32];
 unsigned char  string_000080[32];
 short          array_0000A0[8];
 int            dword_0000B0;

 struct class51_stru0 struct_0000B4;

 unsigned char        *ptr_0000D4;
 struct class51_stru1 *ptr_0000D8;
 int                   dword_0000DC;
 unsigned char        *ptr_0000E0;
 unsigned char        *ptr_0000E4;
 unsigned int          uSkyTextureID; // dword @ 0000E8
 int                   dword_0000EC;  // main tile texture id?

 int            dword_0004C0;
 unsigned char *ptr_0004C4;

 int array_0004C8[10];
 int array_0004F0[14];

 unsigned char array_000528[0x3C8];
 unsigned char array_0008F0[0x3C8];

 char some_additional_fields[0x20000];
};

If I knew at least some of vatiables' names, then things would've been much more clearer.


phamlongtuan
Thanks, that's much appreciated too


tolich
Teal/black colors in texture are for alpha channel. At first, I've started to edit this in photoshop, but it takes too much time (for me) and draws my attention from the immediate task.

[tolich]

Teal/black colors in texture are for alpha channel.

Much easier. Palette entry 0 is transparent.

[mkienenb]

We can do better than variable names. We can tell you exactly how 99% of the data is used.

Grayface (Sergey), Angel Death, and I are writing a new MM Level Editor -- this one can actually create levels instead of just decode level formats or view maps, and we have a functional editor for most of outdoor MM6 map elements now (sprites, 3d models, height, tiles, but not monster spawn points or individual item/monster placements yet).

Grayface, Angel Death, and a few others created this wiki page documenting the .BLV (indoor map) format.

http://rewiki.regengedanken.de/wiki/.BLV

I will look into creating these pages for you.

http://rewiki.regengedanken.de/wiki/.ODM
http://rewiki.regengedanken.de/wiki/.DDM

In the mean time, you can look directly at the revised MMLevelEditor source code for parsing LOD data here:

http://mm8leveleditor.cvs.sourceforge.n ... ideattic=1

Specifically, for ODM, you will want to start at this class:

http://mm8leveleditor.cvs.sourceforge.n ... arser.java

For DDM, you will want

http://mm8leveleditor.cvs.sourceforge.n ... arser.java

-Mike

[mkienenb]

Teal/black colors in texture are for alpha channel.

Much easier. Palette entry 0 is transparent.

What tolich is saying is that if you want to remain backward-compatible with the MM engine, the first entry in the palette for an image always indicates a "color" where the pixels are 100% transparent. A pixel is either completely transparent or completely opaque.

For a modern graphics engine, you'll need to convert that "color" into a transparency channel or alpha map or alpha component for your image.

I haven't bothered to fix this in the new MM Level Editor yet either, but it's not all that important for an editor

[mkienenb]

mkienenb
It wuld be cool then to take a look at headers. I know all the fields of the data structures, but many of them are of unknown use to me. In general, now it looks like this:

Code: Select all

struct ODM
{
[...]
};

If I knew at least some of vatiables' names, then things would've been much more clearer.

Is this from MM6, MM7, or MM8? The formats are slightly different for each one. Each version tends to add a few more fields, although sometimes they change the meaning of a field.

[/quote]

[GrayFace]

Grayface, Angel Death, and a few others created this wiki page documenting the .BLV (indoor map) format.

It's mostly by Angel Death. I've changed just one structure and abandoned the idea.

[mkienenb]

http://rewiki.regengedanken.de/wiki/.ODM

I have committed a start for the ODM page, covering the top-most data structures.

Note that I have concentrated on MM6 ODMs for the last few months, and there's a good chance that my information for MM7 and MM8 is no longer entirely accurate.

Actually, there's a good chance that the entire thing has mistakes, as I don't process the data sequentially when I parse it. I read the entire file contents into a buffer and access it by offsets.

However, I can guarantee that the currently source code to which I posted a link earlier can both read and write MM6 odm files, so my source code should always be considered the authoritative source of information.

Feel free to post or email more questions, or to update the wiki page directly as you find mistakes or determine something new. Let's see if we can use your project to finally document the odm file in a human-readable format.

Note that there is a lot of information in the odm files that is irrelevent. This is because the odm files were created by a 3DO editing tool. Most of the binary data files in the LODs are direct C structure dumps, so sometimes there's real information that is only useful to the editor (like resource management names, editor versions, modeling tool defaults) and sometimes there are data dumps that are completely useless (like pointer structures for local variables which have no meaningful value once the program terminates).

[Nomad]

tolich

Much easier. Palette entry 0 is transparent.

After thinking a while, I have rejected the paletted textures since this technology is totally obsolete

mkienenb
First of all, you were asking somewhere.. well, I am using MM7 engine.
Next,

http://rewiki.regengedanken.de/wiki/.ODM

This description conflicts a bit with your java ODM loading code.
By the way, my decompiled and your java loading functions are almost identical
Thatks for the other links and MM8LevelEditor! I never used java, but this is a minor problems, this source will be very useful!

What tolich is saying is that if you want to remain backward-compatible with the MM engine, the first entry in the palette for an image always indicates a "color" where the pixels are 100% transparent. A pixel is either completely transparent or completely opaque.

The reduce the amount of work, I have abandoned standard MM textures (pcx, tga, indexed). Writing a 32bit tga loader took me about 20 minutes, much faster than diggin in a lots of scary code. Other formats are added on-the-fly using libpng, libjpeg and such.
For backward-compatibility, in the release version I have decided to ship a tool that will repack existing LODs into a new one, just before game starts, changin old data formats into new ones. That will reduce the amount of code in engine and help to eject all the unnecessary and legacy functions that support totally outdated formats into a stand-alone tool.
Since I look forward to UPGRADE the engine, the refactoring needs to be done on the fly to avoid double work.

For a modern graphics engine, you'll need to convert that "color" into a transparency channel or alpha map or alpha component for your image.

That is how it's currently done.

[Nomad]

Currently working on something that looks like Frustrum Culling or Rasterization... God give me strength

[mkienenb]

http://rewiki.regengedanken.de/wiki/.ODM

This description conflicts a bit with your java ODM loading code.
By the way, my decompiled and your java loading functions are almost identical

Make notes on the page where you find conflicts. I don't have any time to look at it today, but if you note the problems, I'll correct them.

If you don't hear from me in a week, remind me via email

[GrayFace]

For backward-compatibility, in the release version I have decided to ship a tool that will repack existing LODs into a new one, just before game starts, changin old data formats into new ones.

I'll give you Delphi classes for working with LODs.

[Nomad]

In-game UI is almost parsed, now it's mininmap's turn.
Post #1 updated.

[Ck-NoSFeRaTU]

stage (1) ~20% completed (35 000 / 193 000 lines of code parsed)

On parsing are you mean rewriting code from ugly hex-rays listing to easy human readable form or just fixing errors? If first then it is equal or even more than writing engine from scratch. If second it can be useless waste of time because who can undestand that ugly code can modify original exe directly or by hooks.
But I'm very interested in that project anyway.

SVN with the latest version resides here:
http://www.assembla.com/code/bb7/subversion/nodes

Open up svn anonymous read access for checkout.

[Nomad]

Ck-NoSFeRaTU

On parsing are you mean rewriting code from ugly hex-rays listing to easy human readable form or just fixing errors?

It's rewriting along with refactoring, extracting classes and structures, member functions, naming variables, etc.

If first then it is equal or even more than writing engine from scratch.

It's even more, since I try to refactor on the fly where it is possible. But this was I have a (almost) stable exe at any time.
When you're writing from a scratch, it makes a long way to have at least a demo. And on the "technical demo" stage most of projects die.
So there are the benefits in my approach.

But I'm very interested in that project anyway.

This is very a good news!

Open up svn anonymous read access for checkout.

I guess it's done.. try again.
I'm not a power user with svn :\

[tolich]

http://www.castlegobs.nl/forums/index.p ... topic=1791
Do you both work together?

[Ck-NoSFeRaTU]

http://www.castlegobs.nl/forums/index.p ... topic=1791
Do you both work together?

Who both? Feaerenil and Nomad? IMHO It's the same person. :-)