Strike Commander with Oculus Rift (Virtual Reality Headset) project.

[fabiensanglard]

Back in the 90s, my dream was to be able to play Strike Commander with a Virtual Reality headset.

Now in 2013, I have the Oculus Dev and the C++/OpenGL skills but not the source code of SC. It seems to have been lost...so I cannot add that feature !

1. I wonder if anybody have managed to reverse engineer SC.EXE. I found the Pentacle project (Ultima8 engine) and Exult project (Ultima 7 engine). Google returned nothing for SC so I assume no work has been done on SC but it cost nothing to ask here. I know that some people have the source of WC3 which uses the same RealSpace 3D engine as Strike Commadner: Maybe they did some work on SC based on that ?

2. I was thinking of an other approach to achieve my goal: Reverse engineer the assets and program my own flight simulator. That would not be the real deal but that would be close: I found Mario Brito amazing file format description for TRE and PAK files. I am in the process of writing something to extract the bitmaps to I will have the textures...but will still have to recover the 3D models from the IFFs. I wonder if anybody managed to reverse engineer the content of OBJECTS.TRE (../../DATA/OBJECTS) or it is documented anywhere ?

 

[fabiensanglard]

Well, it took me a week in my spare time but I have managed to extract the palette, 3D models, the texture and all the voices from the intro (.VOC Creative Labs format). It was very helpful that Origin uses IFF file structures:

Object Viewer:

Textures viewer :

Those were the low hanging fruits. Before starting up IDA I may try to see how the cinematic are encoded. Probably frame delta encoded with LRE compression. Problem is that they are stored in .PAK which were a little bit documented by Mario....but it still kinda looks like mashed potatoes.

 

[wcnut]

That is awesome! Occulus Rift aside that is quite an achievement, congrats! I recognize XCode. Do you know if the OR will be mac compatible? What about DOS compatible? DOS doesn't have OpenGL. Wont that be problem? Unless you plan of attack requires reconstructing the entire code and then redoing the entire game from scratch?

 

[Bandit LOAF]

You might want to look at Wings of Glory, which uses the same engine and supports a pair of contemporary VR helmets including the mouse look /head tracking.. (One is the Forte VFX-1, which works well!)

 

[fabiensanglard]

@wcnut.

Programming for Oculus involves many steps. Among them, rendering twice at half the horizontal resolution (for each eyes) and more importantly perform a barrel distortion to counter the Oculus binocular distortion: I won't even mention that Strike Commander was using 24:8 fixed point math so I may not even have enough precision to move the POV slightly to right eye and left eye. It is not rocket science if you have a game engine based on float and if you have the source code. But in the case of Strike Commander, that makes patching the DOS executable impossible.

In terms of reverse engineering, I have done some work with IDA on SC.EXE (which is just a launched for MKTERR.EXE (the terrain map generator that took hours to run), MKGAME.EXE (the installer ?) and STRIKE.EXE (the actual RealSpace engine))...It was 8KB and quite easy to figure out....but STRIKE.EXE is 746KB: It is a COLOSSAL amount of work. Just as a reference, it took Gregory Montoir (an expert with impressive accomplishments) 2 months to revert ANOTHER WORLD from assembly to C...and ANOTHER.EXE is only 20KB. So if I don't manage to reach a critical mass and have other people interested like for Pentacle/Exult I will probably end up giving up.

A more realistic approach is to attempt to understand the MAP format....and then write my own engine (which is actually not that hard) using the MAP, MODELS and TEXTURES from Strike Commander. I may have to dive into MKTERR.EXE but it is "only" 204KB so it may be doable with a gross amount of work.

@Bandit LOAF

Wings of Glory is an excellent suggestion. Unfortunately, it is SC that i am dreaming to play with: For the longest time I only had 2MB of RAM and couldn't even run the game. Then when I sold a kidney and reached 4MB, my 386 could only run the game at 10fps....with minimal details (it was slightly less ugly that Stunt Island or Flight Simulator 4)....and I still played it days and nights. It is a bit like a childhood dream that I am pursuing.

Unless you meant that the source code has been somehow "released"...which would help me tremendously to understand the MAP format.

 

[wcnut]

Well whatever you do, just the fact that you are digging into the files is of tremendous interest to the people here, I'm sure, and I wish you the best!

 

[HCl]

Hey here you are , I didn't notice you started a thread on this topic until now.

I sent you a small e-mail regarding the PAK format. It really is standard WC1/WC2 packaging, with RLE data. Once you become familiar with the WC1 RLE format you'll be able to decode it. As a test, i ran it through one of the Matlab scripts i put together for research and decoding WC data and it decodes the various blocks perfectly. (small screenshot below)

Anyway, great project, keep up the good work!

 

[fabiensanglard]

Thanks Mario, your hard work and generous explanation have been incredibly helpful. I can now read the PAK archives....at least partially !

I was hoping the RLE format was used to stored full animation but it looks like it is made of layers.....which means there is more work to be done in order to understand the full animation format:

There are still some data at the bottom of the PAK that I cannot made sense of....my worse fear is that it is some kind of bytecode to reconstruct the animation. It doesn't look like there is enough data to be it but how knows.

Looking at those format is a surprise of every moment: The PAK archive is made of blocks. If a block contains images, you must search for an IFF file to see it if contains a palette. If you find one, use it...but the color channels are on 8 bits each. If you don't find an IFF that contains a palette then use the default one....BUT there, color channels are encoded in the standard VGA way: 6 bits. OMFG !!!!!!

I wonder if I can look toward the Pentagram project since they seem to be able to read U8 animations. It may not help: U8 was a Lord British project and SC was a Chris Robert project. I don't know if the team collaborated/shared technologies. I don't even know if they were working from the same offices.


Anyway, back to work....

 

[delMar]

Great thread, fabiensanglard.

I am currently dealing with the Origin FX engine over at www.wcworkshop.net.
My project's not there yet when it comes to graphics but maybe you'll be able to benefit from it as well some day.

 

[fabiensanglard]

@delMar

Sweet, so the Origin FX system may have been used for the midgames (intro, cut-scenes) in Strike Commander ? Do you have any source code or idea of how the codec worked ? I am thinking of maybe bytecode that allows to program the animation and compose all the layers into one.

I have made some great progress in the last 24 hours:

- I have parser for the voices in the game (VOC) and XMIDI musics. I don't have a player but it is peanuts since I can leverage the players from the great people of Exodus project.

- I am now able to locate and extract the map texture !!!

I have even found assets that I never saw in the game, like crop circles:

I also have a good idea of how maps are organized (they seem to features precalculated gouraud color or gradient)....but so far it is still mashed potatoes.

The two BIG things that I am missing are the ability to understand MAPS and ANIMATION. I hope some people with reverse engineering skills will join the project, I know from experience that a one man effort is usually very hard to carry on for a while....

 

[delMar]

Nice job, fabiensanglard.
I never become tired of viewing images which were extracted right out of a game's resource files

Currently, I am busy with getting some UI and basic editing done. Once that's working, I may try to work on the images.

Did you already browse the various mission editing tools available here?
Also, you might want to check out this thread.
The last post on the first page contains a tool called WCNAV which also allows to view midgame animations.
Not sure, how well that fits SC.
According to HCl, there should be some serious relationship between SC and WC.

It really is standard WC1/WC2 packaging, with RLE data.

 

[HCl]

Hey great work here!

For everyone's benefit, i'll elaborate a bit on WC1/2 cutscenes, which have a lot in common with cutscenes from Armada, Academy and Strike Commander:

Animations are stored exactly the same way, with frames coded in "blocks". Each block, as you noticed, share the same palette (this strategy in fact is not so different from the encoding of some Origin FMV movie codecs, for example WC3/4 and Privateer 2). I've been able to rip to AVI, with a Matlab script, sequences from WC2, Armada, even Academy. Needless to say, I want to import new animation sequences into WC1 / 2 at some point in the near future, it's part of what I have in mind for WC2 modding

In *all* these games, from what i've seen, animations are encoded with typically has three possible modes:

- Fully differential encoding, where the first frame has the entire image and subsequent frames only have the differences (like the Niven Landing or the pic below: figure 1 is the full image, figure 2 is just the current frame difference).
- Frame + background: in each frame, draw the background, then the animation frame in order.
- Full frames: simple wipe and draw.

Optionally, the screen can be "wiped" at points. On my scripts, i cheat a bit, since this is extra information, not present in these files, and explicitly tell it at which points it happens. But it ends up working rather well.

Now, where are this additional information stored? (sound effect triggers, background triggers, etc) It depends on the game, and not all is good news:

- On WC2, this info is apparently on INCIDENT.Sxx (although i haven't broken it down yet)
- On Armada... this seems to be hardcoded on the EXE(!), at least for the Armada intro sequence. But, by cueing the triggers manully, the intro can be ripped. For example, from one of my Matlab scripts:
Intro
- On Academy, whatever triggers seem to be hardcoded on the EXE as well... but there's only 2 sequences in there, so I can understand why they are not configurable. For example:
Academy Logo
- On Strike Commander? Good question, and i don't have my CD with me to check. But i'd bet there's a file in there similar to INCIDENT.S00 that allows configuring the triggers of the animation sequences.

My plan for importing new cutscenes:
- For WC2: use my current RLE encoder to convert an AVI to RLE frames, then add the necessary information to INCIDENT.S02 (this approach can most likely be reused for Strike Commander and Privateer)
- For WC1... well, let's say i hope to achieve something with some COM-magic when the time comes

 

[Light]

I just thought I would point out what you called "crop circles" are probibly Geoglyphs.

http://archaeology.about.com/od/gterms/g/geoglyphs.htm

https://www.google.com/search?q=anc...A2gWlnYHgBA&ved=0CAkQ_AUoAQ&biw=1920&bih=1075

 

[fabiensanglard]

Here is the PAK format that contains the map data:

Strike Commander TERRAN format :

Example MAURITAN,PAK

Triangles are probably within MAURITAN.TRI
MAURITAN.TRI seems to have indexes at the bottom of the file (0x00 to 0x75)

MAURITAN.PAK contains 7 unique entries (0 duplicates):

    Entry[0] size: 540 bytes.
        Block[0] size: 540 bytes:    IFF  block which describe textures in MAPS chunk
                                                    2 bytes unknown
                                                   8 bytes textureName
                                                   2 bytes unknown
                                                   Textures are references to TXM_LIST.IFF
                                                   TXM_LIST.IFF is referencing TXMPACK.PAK (which is full reverse-engineered).
                                                    
    Entry[1] size: 778900 bytes.
        Block[0]    size: 2400 bytes (IMAGES block ?).
        Block[1]     size: 2400 bytes (IMAGES block ?).
        Block[2]     size: 2400 bytes (IMAGES block ?).
        Block[3]     size: 2400 bytes (IMAGES block ?).
        .....
        Block[323] size: 2400 bytes (IMAGES block ?).
                                                            My guess is that it is the pre-gouraud ground MipMap level 0.
    Entry[2] size: 195700 bytes.        
        Block[0]    size: 600 bytes (IMAGES block ?).
        Block[1]     size: 600 bytes (IMAGES block ?).
        Block[2]     size: 600 bytes (IMAGES block ?).
        Block[3]     size: 600 bytes (IMAGES block ?).
        ......
        Block[323] size: 600 bytes (IMAGES block ?).
                                                            My guess is that it is the pre-gouraud ground MipMap level 1.
    Entry[3] size: 49900 bytes.
        Block[0]    size: 150 bytes (IMAGES block ?).
        Block[1]     size: 150 bytes (IMAGES block ?).
        Block[2]     size: 150 bytes (IMAGES block ?).
        Block[3]     size: 150 bytes (IMAGES block ?).
        ....
        Block[323] size: 150 bytes (IMAGES block ?).
                                                            My guess is that it is the pre-gouraud ground MipMap level 2.

                                                             Pretty sure the atomic unit in those
                                                            is 6 bytes. 6 can divide all of them
                                                            And I can see a nice pattern in the hex editor
    Entry[4] size: 15196 bytes.
        Block[0] size: 15196 bytes (TRIANGLES .TRI).        -> MAURITAN.TRI Map Triangles. Mashed potatoes for now.
    Entry[5] size: 3480 bytes.
        Block[0] size: 3480 bytes (OBJECTS .OBJ).        -> MAURITAN.OBJ (Object types and coordinates on the map)
    Entry[6] size: 648 bytes.
        Block[0] size: 648 bytes ( .AVG block).            -> MAURITAN.AVG (No idea what is this)

Intriguing that for each PAK (QUEBEC, SANFRAN, ...) there are 323 entries in the three "mipmap" parts (1,2 and 3). Even more interesting is that 150 x 4 = 600. And 600 x 4 = 2400. So clearly, entries 1, 2 and 3 are very connected: 3 is a mipmap of 2 and 2 is a mipmap of 1. I wonder if they are textures of the pre-calculated gouraud shading. Gouraud is diffuse lightning which does not depend on the Point of View like Specular does...so it is a viable theory.

 

[fabiensanglard]

I have finished the 3D models exporter and added a Renderer with texture and gouraud shading:'

F-22:

F-15:

A:10

F-16:

YF-23 :

 

[fabiensanglard]

I am releasing the source code of the tools I have build for Origin Systems RealSpace engine:

LibRealSpace :

https://github.com/fabiensanglard/libRealSpace

RealSpace was used for many other Origin games :

Strike Commander
Pacific Strike
Wings of Glory
Wing Commander III
Wing Commander IV

I hope this library will benefit anybody working on those games.

A list of what libRealSpace can do so far:

• Full TRE Support.
• Full IFF Support.
• Full PAK Support.
  
  
• Render Textures 3D Jets for all Level Of Details
  
  
• Render animation layers (individually).
• Render Map Textures.
  
  
• Guess PAK contents:
  • VOC sound effects.
  • XMidi music
  • Animations.
  • IFF
  • Images
  • Map Textures
  • Sub-pack structure

The thing I would really want to do next are extracting the 3D maps and full animations.

 

[fabiensanglard]

A few more pretty pictures I came across in the PAKS :

 

[ginger_tigra]

Wing Commander III
Wing Commander IV
{...}
Full TRE Support

I'm sorry but your format of .tre don't look like that in WC3/WC4. For example, take look at HCl's XTRE format description. Or, take my .tre extractor&maker (written in C).

 

[delMar]

I am releasing the source code of the tools I have build for Origin Systems RealSpace engine:

This awesome.
I'm really looking forward to finding the time and dig through

 

[fabiensanglard]

I am still horribly stuck on the maps and animation. So in order to cheer myself up, I rebuilt the Object Viewer, DogFight and MainMenu: