Wing Commander Toolbox

Discussion in 'Fan Projects & Editing' started by UnnamedCharacter, Feb 26, 2015.

  1. UnnamedCharacter

    UnnamedCharacter Ensign

    Messages:
    121
    Likes Received:
    106
    Trophy Points:
    2,235
    Wow, it has been a long time since an update. Work on the toolbox has continued and this release has substantial changes.

    There are major API/XML changes, resulting in breaking backward compatibility. The code base has been refactored making it easier and quicker to add support for new file types.

    The ability to edit strings in the WC1 executable has finally been added; only a few strings are supported at the moment. For this release I was more concerned with getting the functionality in place and will tackle the tedious work of finding additional string pointers in the future.

    This release also begins support for WC2 and WCAcademy; only a few file types at the moment: COMM, DIFFLEVL, INTELSHP, PALETTE, SPEECH, WAVEINFO, etc.

    See first post for the file attachment.

    This will most likely be the only update for this year; Merry Christmas everyone.
     
  2. UnnamedCharacter

    UnnamedCharacter Ensign

    Messages:
    121
    Likes Received:
    106
    Trophy Points:
    2,235
    WC1 EXE String Editing

    Editing strings in an executable is tricky; there are several constraints. The total amount of space available for strings cannot change; to make one string longer, another has to be made shorter. Strings can be grouped together into a block, with many blocks and individual strings spread across different locations within the executable.

    For example, imagine the following scenario of 4 strings divided into 2 blocks.

    Original layout of strings: 5, 5, 5, 10 = 25 characters
    [DATA] [ string5, string5, string5 ] [DATA] [ string10 ]

    Modified strings: 3, 3, 3, 16 = 25 characters
    Failure: not enough continuous string area for a string with a length of 16

    Modified strings: 8, 3, 3, 11 = 25 characters
    Failure: cannot fit all strings
    [DATA] [ string11, string3, (empty1) ] [DATA] [ string8, (empty2) ]​

    The good news, there are several strings within the executable which can easily be sacrificed with no real impact: the error messages, the information to purchase Secret Missions, and even the credits if necessary. I do not recommend making the sacrificial strings empty, in case the game needs to display a message, but instead to make them as short as possible. At the moment, the algorithm used to position strings is very simple; it does not try to maximize available space.

    Use the following XML to produce the result in the images bellow.

    Code:
    <ExeFile>
      <ExeFile.Strings>
        <ExeFileStrings>
          <ExeFileStrings.Bar>
            <ExeStringItem Value="Talk to SHOTGLASS." />
            <ExeStringItem Value="A longer talk to ??????????????" />
            <ExeStringItem Value="Talk to ??????????????" />
            <ExeStringItem Value="Check pilot scores" />
            <ExeStringItem Value="Enter barracks" />
            <ExeStringItem Value="Fly training mission" />
            <ExeStringItem Value="A longer talk to %s." />
            <ExeStringItem Value="" />
            <ExeStringItem Value="Talk to %s." />
            <ExeStringItem Value="" />
          </ExeFileStrings.Bar>
          <ExeFileStrings.ErrorMessage>
            <ExeStringItem Value="Sorry, an error has occured..." />
            <ExeStringItem Value="You do not have enough memory." />
          </ExeFileStrings.ErrorMessage>
    ...
    
    wc-left.png wc-right.png
     
  3. UnnamedCharacter

    UnnamedCharacter Ensign

    Messages:
    121
    Likes Received:
    106
    Trophy Points:
    2,235
    Since there has been a few post recently about medal scores, I thought I would update the WC1LiveViewer. It has the same restrictions as before, only Kilrathi Saga is supported (WC1.EXE and wing1.exe).

    To use it, start the game, then this program (WC1LiveViewer). If it recognizes the running process, it will populate the table with game data. Since the game data is continuously updating, I find the best approach is to keep pausing the game during gameplay.

    wc1liveviewer-medalscore.png
     

    Attached Files:

  4. UnnamedCharacter

    UnnamedCharacter Ensign

    Messages:
    121
    Likes Received:
    106
    Trophy Points:
    2,235
    Another update with more support for WC2/Academy file types: ALTINT, ALTSHIPS, INTEL, MISSILE, MODULE, PCSHIP, SHIP, and XSHIP. Also updated WC1 EXE file to support editing of closing dialog, comm message, insert disk, landing comment, medal, mission type, rank, and stranded strings.

    See first post for the file attachment.
     
    Last edited: Feb 8, 2018
  5. YPersonified

    YPersonified Chief Petty Officer

    Messages:
    11
    Likes Received:
    5
    Trophy Points:
    1,185
    Thanks again. Good to see you back with more updates.

    I have a problem while trying to pack WC.EXE.XML with the latest 4.1.0. I chose random destination, clicked Pack, then it says it can't find WC.EXE.

    So I tried the folder with WC.EXE instead, but this 'Unable to fit all executable strings' message pops up and fail.

    Any suggestions?
     
  6. UnnamedCharacter

    UnnamedCharacter Ensign

    Messages:
    121
    Likes Received:
    106
    Trophy Points:
    2,235
    That is the intended behaviour, although perhaps not necessarily obvious as to why. When packing a data file from XML source, a new data file is created; therefore any destination location can be chosen. But, when packing an executable file, a new file cannot be created and must instead be merged with an existing executable file; hence the message of not being able to find WC.EXE.

    The total amount of space available for strings cannot change; to make one string longer, another has to be made shorter. There are several strings within the executable which can easily be sacrificed with no real impact, such as: error messages, information to purchase Secret Missions, etc. I provided more detail in a previous post.
     
    ChrisReid likes this.
  7. UnnamedCharacter

    UnnamedCharacter Ensign

    Messages:
    121
    Likes Received:
    106
    Trophy Points:
    2,235
    Another update for WC2, added support for the CAMPAIGN, GAMEFLOW, INCIDENT, and SERIES files; all these files are structured the same way. But this time around, there is a complication: these files have bytecode embedded in them.

    This means the flow of the game, the cinematic scenes, including all of the dialog, was done using a custom scripting language. Which was then compiled into a custom set of instructions to be executed by the game itself using a custom interpreter. I have never seen any examples of the scripting language used for the game or even any mentions of it. As far as I know, this bytecode is completely unknown, undefined, and undocumented.

    The only way to tackle this is to completely reverse engineer it; I don't imagine that to be a small undertaking. In any case, for the time being, I simply export these bytecode chunks as binary files.

    See first post for the file attachment.
     
    Last edited: Mar 16, 2018
  8. Quarto

    Quarto Unknown Enemy

    Messages:
    11,881
    Likes Received:
    275
    Trophy Points:
    69,385
    Location:
    Poland
    That sounds about right. As I've said before (possibly in this thread...? Dunno), WC2 is remarkably modern in its design - it shares a lot of design principles with the sequels all the way to WCP. While WC1 has an engine built for one single game, WC2 is designed to allow continued expansion to new products. They didn't make much use of this, ultimately, but the necessary design philosophy was there.
     
  9. delMar

    delMar Rear Admiral

    Messages:
    326
    Likes Received:
    57
    Trophy Points:
    22,935
    Gender:
    Male
    Location:
    Austria
    Awesome progress!

    Can't wait to find the time and start tinkering with the byte values.
     
    Shades2585 and ChrisReid like this.
  10. delMar

    delMar Rear Admiral

    Messages:
    326
    Likes Received:
    57
    Trophy Points:
    22,935
    Gender:
    Male
    Location:
    Austria
    Hey @UnnamedCharacter, I got a question for you: when I do an xmlunpack of series.s00, I get an .xml file and some .bin files.

    The .xml does not contain any bytes from the .bin files, right? The content of the .bin file would be just referenced, like shown below, right?

    HTML:
    <OffsetChunk file="series.S00-ContainerBlock000-ContainerGroup-SpriteGroup-ScriptGroup-OffsetChunk.bin" />
     
  11. UnnamedCharacter

    UnnamedCharacter Ensign

    Messages:
    121
    Likes Received:
    106
    Trophy Points:
    2,235
    Correct, no binary data is in the XML file. The OFFSET chunk is exported and then referenced using the "file" attribute.

    Here is some additional information on the structure of the binary content.
    • All the data is in little endian.
    • The first 2-byte (16-bit integer) represents the number of entries (always the same as the number of symbols).
    • Then it is a series of 4-byte (32-bit integer) offsets (matching the number of entries). To calculate the file position, use the current position of the offset, plus the value of the offset.
    Using the example code bellow: there are 11 entries, with the first 8 being empty (-1, -1, -1, -1, -1, -1, -1, -1, 12, 281, 384). For the remaining 3, their file positions are: 46 (offset position 34 + value 12), 319 (38 + 281), and 426 (42 + 384).

    Code:
    <ScriptGroup>
      <OffsetChunk file="SERIES.S00-ContainerBlock002-ContainerGroup-SpriteGroup-ScriptGroup-OffsetChunk.bin" />
      <SymbolChunk>
        <SymbolChunk.Items>
          <SymbolItem Text="__slot0" />
          <SymbolItem Text="__slot1" />
          <SymbolItem Text="__slot2" />
          <SymbolItem Text="__slot3" />
          <SymbolItem Text="__slot4" />
          <SymbolItem Text="__slot5" />
          <SymbolItem Text="__slot6" />
          <SymbolItem Text="__slot7" />
          <SymbolItem Text="PALADIN_DRINKING" />
          <SymbolItem Text="STINGRAY_DRINKING" />
          <SymbolItem Text="SPARKS_WORKING" />
        </SymbolChunk.Items>
      </SymbolChunk>
    </ScriptGroup>
    

    Code:
    0B 00 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
    FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
    FF FF 0C 00 00 00 19 01 00 00 80 01 00 00 77 01
    23 26 FF 83 0F 10 00 9E 00 78 00 00 09 00 24 01
    4F 00 0E 02 00 26 FF 92 27 89 01 24 00 1B 0F 47
    00 32 07 0F 3F 00 30 24 07 1B 0F 0C 00 24 03 44
    ...
    
     
  12. delMar

    delMar Rear Admiral

    Messages:
    326
    Likes Received:
    57
    Trophy Points:
    22,935
    Gender:
    Male
    Location:
    Austria
    Thanks for the clarification.
    Man, that's a l o t of data...
     
  13. Quarto

    Quarto Unknown Enemy

    Messages:
    11,881
    Likes Received:
    275
    Trophy Points:
    69,385
    Location:
    Poland
    I'm sure this has been said before, but it is so great to see the continuing progress with this project. When I first took interest in Wing Commander, one of the things I tried to do while biding my time for an opportunity to play WC3 or WC4, is to attempt to mod WC2 using HCl's mission editing tools. But at the time, all you could do is edit ships and missions. Seeing this stuff gives me hope that one of these days, it might be possible to produce a complete mod with a campaign structure, dialogues, and everything. Even if the original art assets still had to be used (I don't think there's anybody around who could attempt to produce WC2-like talking heads... unless... well, come to think of it, there is Denis Loubet :p), it would be so nifty.
     
    ChrisReid and Shades2585 like this.
  14. delMar

    delMar Rear Admiral

    Messages:
    326
    Likes Received:
    57
    Trophy Points:
    22,935
    Gender:
    Male
    Location:
    Austria
    Hm, I think the Privateer talking heads allow for all kinds of permutations of eyes, hairstyle, skulls, etc.
    Might require some conversion of file formats, but still easier than drawing some (at least for me :) )
     
  15. Quarto

    Quarto Unknown Enemy

    Messages:
    11,881
    Likes Received:
    275
    Trophy Points:
    69,385
    Location:
    Poland
    That is true, although Privateer talking heads do look very different to WC2 heads because they're straight-on instead of three-quarters view. On the bright side, though, if Privateer heads were used, one could just as easily recycle Strike Commander and Pacific Strike heads, since they're all the same style (file formats might slightly differ, though - I don't think anybody ever even tried extracting anything from Pacific Strike anyway).

    However, I must say, the more I think about it, the more it's obvious that the solution would indeed be Denis Loubet. I mean, with his Patreon account, he's happy to draw stuff on request - so really, if the tools were there, and somebody wanted to produce a WC2 mod with new faces, it would be a matter of raising some funds within the community and then going to him. It would be a fascinating project, for sure. Imagine the possibilities, what with the option of importing the Jrathek and Wraith into WC2... not to mention the Hellcat, Dralthi, and Arrow WC2 imports that HCl did with Howard Day a few years back.

    Well, I guess for the time being, I doesn't matter - we'll have to wait for @UnnamedCharacter to develop his toolbox further. But it's exciting stuff to think about!
     
  16. delMar

    delMar Rear Admiral

    Messages:
    326
    Likes Received:
    57
    Trophy Points:
    22,935
    Gender:
    Male
    Location:
    Austria
    It is.
    I'd love to do SM 1.5 in WC2, as all the unique scenes (like briefings in the Colonel's Office) don't work in WC1.
     
    Shades2585 likes this.
  17. Darkmage

    Darkmage Vice Admiral

    Messages:
    913
    Likes Received:
    20
    Trophy Points:
    31,435
    Location:
    Australia
    We've also got the audio file format for WC2/Privateer decoded if you wanted to make a speech pack for WC2 mods...
     
    Shades2585 likes this.
  18. Bandit LOAF

    Bandit LOAF Long Live the Confederation!

    Messages:
    28,635
    Likes Received:
    552
    Trophy Points:
    69,485
    Gender:
    Male
    Location:
    Ashton, MD, USA
    The character heads were Glen Johnson; Denis did things like cockpits, ship close-ups and sets.
     
  19. delMar

    delMar Rear Admiral

    Messages:
    326
    Likes Received:
    57
    Trophy Points:
    22,935
    Gender:
    Male
    Location:
    Austria
    New, unique settings would be awesome :)
     
  20. delMar

    delMar Rear Admiral

    Messages:
    326
    Likes Received:
    57
    Trophy Points:
    22,935
    Gender:
    Male
    Location:
    Austria
    Does anyone know, which file contains the sound effects of WC2?
     

Share This Page