Privateer 2 editing and extracting

Discussion in 'General Wing Commander Chat' started by Lin Kuei Ominae, Feb 16, 2011.

  1. Lin Kuei Ominae

    Lin Kuei Ominae Spaceman

    Messages:
    65
    Likes Received:
    0
    Trophy Points:
    0
    You don't have to jump to the nearest planet. When you load a savegame you are already on that planet that has these set/modified bbs cards. ;) So just enter the booth, pick the mission and start from the planet to play it.
    However, i still haven't figured out the corresponding mission to a certain mission number.
    e.g. there are Missions with the index number 368, while the missions 369 and 367 don't exist. This seems to be more like a pointer to a certain mission rather than an enumeration of one mission after another (e.g. like read mission at line 368 in the mission*.iff file, where only line 368 has an entry point and the following line has already certain data to this mission and the previous line 367 data to the previous mission). So it can still take a bit time to find the really working mission indexes.
    Right now i'm working on a way to store the data user-friendly and allow the user to expand/modify this file (most probably an ini with sections like [368] Name="Ser Arl Shoyer's ass kicked" Reward="5.500Credits").

    I'm still searching the news hex address. Like the string "BBS_", which is followed by the section of the bbs cards or "PCOM", which is followed with the planet commodities, there is also a special "NEWS" string in the savegame file.
    However unlike the other sections, this NEWS section doesn't seem to contain the information about the currently shown news.
    But i haven't given up on that and will surely sooner or later find the place where these information are stored. A certain tab in the editor is already included, and this will then also allow to modify all related data to the news. ;)

    After that is done, there also isn't much left that needs to be deciphered from the savegame, as only the big section with the accepted missions is left and this one looks really complicated due to the many dynamic variables like ships involved in the mission, their nav point, relative position from the nav points centre, direction speed, interaction with other ships etc etc.
    I'm not sure if i ever implement this, but this will surely go in the last version of the editor. Though i also know it would be very useful to directly read all the corresponding data concerning a certain mission, so you could quickly compare the WCPedia mission page with the real mission data. Then it might be even possible to write complete new missions and hook them into the game via editing the savegame. But as i said, this is the very last point on my list.
    I'm much, much more interested to know how i can modify the ship speed, shield, hull, yaw/pitch/roll values as well as the weapon firepower range etc. Though this data doesn't seem to be stored in the dark.exe, but in one of the strangely compressed iff files (i assume gameflow.iff).
     
  2. AD

    AD Finder of things, Doer of stuff

    Messages:
    4,918
    Likes Received:
    102
    Trophy Points:
    36,885
    Location:
    BC, Canada
    Don't know if it's relevant or not but the WC prophecy dev cd stuff Cpt Johnny sent had a program for viewing IFF files that would organize all the data for you. Not sure that it works with P2 but it was interesting for WC3, 4, and even the PSX files. (called VIFF.exe IIRC). So as long as you can extract the archived IFFs off the P2 cds it might give you a good tool for hunting for stuff in the hex editor.
     
  3. Lin Kuei Ominae

    Lin Kuei Ominae Spaceman

    Messages:
    65
    Likes Received:
    0
    Trophy Points:
    0
    Can you give me a link to this VIFF tool? Mario Brito wrote in the comments of his bigf privateer 2 extractor tool that the compression of privateer 2 and wingcommander is in some cases very similar but in others totally incompatible/different (e.g. zlib for some of the priv2 files).
    Maybe the tool works on the right files and i can uncompress them to finally read/write the ship data.
     
  4. AD

    AD Finder of things, Doer of stuff

    Messages:
    4,918
    Likes Received:
    102
    Trophy Points:
    36,885
    Location:
    BC, Canada
    Unless someone beats me to it I'll have look for the link when I get back from work. You probably need to extract the IFFs first for sure though.
     
  5. AD

    AD Finder of things, Doer of stuff

    Messages:
    4,918
    Likes Received:
    102
    Trophy Points:
    36,885
    Location:
    BC, Canada
    This should be a link to the dev CD downloads: http://www.wcnews.com/news/update/8858

    It's a bit of a bigger download but I can probably just post the VIFF program later if there's a need. I recommend not setting compatabilty on VIFF95.exe and dragging the IFFs into it instead of doubleclicking the iffs to open. Not sure how useful it will be but here's a shot of what you see when you load P2s booth.iff:

    [​IMG]

    When you load an IFF you get the breakdown list on the left that gives you headers and also the location. doubleclicking a chunck in the list gives you the Hexedit-like box containing the code for that specific chunk.
     

    Attached Files:

    • viff.jpg
      viff.jpg
      File size:
      360.7 KB
      Views:
      1,971
  6. Lin Kuei Ominae

    Lin Kuei Ominae Spaceman

    Messages:
    65
    Likes Received:
    0
    Trophy Points:
    0
    I checked out the VIFF.exe (thanks for the download link btw), but unfortunately it didn't help me further. Because it can't uncompress the chunks (neither extracting them) and i also haven't found a converter to convert the chunks into a common picture or readable text format.
     
  7. AD

    AD Finder of things, Doer of stuff

    Messages:
    4,918
    Likes Received:
    102
    Trophy Points:
    36,885
    Location:
    BC, Canada
    Yeah, it's not so much a decompression tool as it organizes the IFF file (and other files too) by chunk and gives you the addresses of each individual chunck. As far as I can tell the only files where the BIGF tool is useful are SETS.IFF, ISETS.IFF and PILOTS.BIG and SPEECH.BIG and DANI.BIG. When I compiled HCLs tool it was useful to extract a bunch of *.shp files *.FAT and *.fat files and a bunch of IFFs for planets and stuff. Presumably since they were in speech.big many of the *fat files are audio.

    As far as what compression is used beyond that, I wouldn't be the one to ask. Oddly though SAMPLE.OPL contains a list of all the files in DANI.BIG.
     
  8. Lin Kuei Ominae

    Lin Kuei Ominae Spaceman

    Messages:
    65
    Likes Received:
    0
    Trophy Points:
    0
    Yes, the fat files are definitely the priv2 audio format.
    In one of my posts above, i did something similar and found a quite decent solution to convert the fat audio files.
    let me quote myself.
     
  9. AD

    AD Finder of things, Doer of stuff

    Messages:
    4,918
    Likes Received:
    102
    Trophy Points:
    36,885
    Location:
    BC, Canada
    It probably doesn't tell you anything you don't already know but further looking into the files spacegeo.iff and spacetex.iff it appears that these contain geometry and textures respectively. The IFFs within spacegeo all start with "FORM BR3D" and have "MAT" sections that each have a defined texture listed. Although the listed textures say *.pix (which it seems is an Alias Wavefront format) they perfectly match the names of all the IFFs in spacetex.iff

    In the geometry files the textures are listed in the last chunk (MCMP) and it would seem the palette is defined in the second to last chunk of each MAT section (MSNM) which matches the assumptions Gremule made from looking at some of the source files a while back (http://www.wcnews.com/news/update/10376;http://www.wcnews.com/news/update/9629).
    In the IFFs for the... meshes I guess we could call it rather than geometry, the palate is listed as a .tab which corresponds to that Tab.gif Gremule mentions. So for example can1_d1.iff uses the game.tab palette for each of it's five texture files. Comm_d1 however lists 11 mat sections (therefore eleven textures) and 'cv.bl.br.tab' as the palette for each texture.
     
  10. Lin Kuei Ominae

    Lin Kuei Ominae Spaceman

    Messages:
    65
    Likes Received:
    0
    Trophy Points:
    0
    yep. They even have the same filenames as in the P2_Archive.zip, which has the geometry in 3ds and the textures as gif.
    So an experience decoder should be actually able to compare the files and find the correct compression algorithm, to uncompress/decode the used iff files.

    Damn, it would be awesome to be able to change some 3d-models. The first thing i would do is giving the stream laser MKII a small dual laser beam and the Kraven MKIV a quad laser beam (or big dual laser beam), so the laser beams match with the weapon 3d-model shown in the booth. (i always wondered how the Kraven MKIV can fire a triple laser beam while the gun has only 2 big and 2 small barrels)
    As a next step i would make all the laser a lot more different, so they aren't only simple lines with the same length but different texture, but also a sphere (e.g. for the volt laser) and different in size and length. (This was definitely a great advantage of Privateer 1 where each weapon also had a different unique looking projectile)
     
  11. Bob McDob

    Bob McDob Better Health Through Less Flavor

    Messages:
    3,390
    Likes Received:
    1
    Trophy Points:
    36,885
    Location:
    Grassy Knizzle
    I forgot to post this.

    http://reckoning.solsector.net/Privateer2Ever/P2Tex.zip

    Using these palettes and a copy of Adobe Photoshop CS, I manually shifted the colors on the textures. Put the models from the archive into one of the folders and it should show up in the proper colors, assuming that the texture exists - I didn't do every color for every GIF.
     
  12. AD

    AD Finder of things, Doer of stuff

    Messages:
    4,918
    Likes Received:
    102
    Trophy Points:
    36,885
    Location:
    BC, Canada
    Ok, I've split off the Privateer 2 hacking stuff into it's own thread out of the WCpedia discussion since this particular tangent has been taking it's own course and gettinng interesting in it's own right.

    Lin Kuei, I did email HCl about his bigf extractor tool. near as I can tell he never actually compiled a version for others to use that included zlib. I've been attempting to add zlib into it myself but honestly - as you may have guessed - I pretty much have no idea what I'm doing. I'm fumbling in the dark. Anyway he may have some more stuff stored away but I got the impression that it would take a while for him to come up with anything, but if you have a specific question I can always forward that to him.

    Anyway, here's the basics of what HCl said he could remember about his exploration in to the P2 formats:

    He're's first HCls text out of the bigf source code itself and then a follow-up discussion we had:

    Now, I might continue to try and figure out how to include the Zlib stuff but I'm certain someone with actual programming skills will have a lot more success than me. (They'll definitely get it done a lot faster than me at least) However of use I also managed to fine a command line tool for un-zlibing raw code chunks. That might be useful if I can figure out how it works.
     
  13. Lin Kuei Ominae

    Lin Kuei Ominae Spaceman

    Messages:
    65
    Likes Received:
    0
    Trophy Points:
    0
    I would like to know how he found out the algorithm just out of interest (a simple trial and error would surely have taken ages).
    Though beside of that a code example to extract every file format would be really great.
    I would then enhance my Priv2Editor with a File-modding section that gives the normal user the chance to edit/compress/uncompress the p2d files with a graphical interface and a simple file browser.

    If that is too much work, it would be awesome if he could point me to the file with all the ship data like shield strength, speed, armor, gun/module/missile slots etc as this doesn't seem to be stored inside the dark.exe.

    Other than that every tiny bit of information he can provide is useful.

    Maybe also point him to this topic so we can communicate directly.
     
  14. AD

    AD Finder of things, Doer of stuff

    Messages:
    4,918
    Likes Received:
    102
    Trophy Points:
    36,885
    Location:
    BC, Canada
    I'll definitely be emailing him again, and he did mention that he might have some older resources stored somewhere so I'll see what he says and definitely mention that I started this thread ( I had pointed him previously to the other thread).

    With regards to samples etc, all I can say is don't get your hopes too high. HCl has had a lot on his plate personally and hasn't been able to dedicate much time to this sort of thing lately. I'm not sure much will change in that regard but it's definitely something he's interested in and that's near to his heart.
     
  15. Lin Kuei Ominae

    Lin Kuei Ominae Spaceman

    Messages:
    65
    Likes Received:
    0
    Trophy Points:
    0
    Well if he can point me in the right direction (code snippet), i think i can come up with the rest. It's just that i have absolutely no clue how he found out the right hex values for e.g. table1 in his Priv2 TGV audio decompressor or some of the other algorithms operating principles.
     
  16. Bob McDob

    Bob McDob Better Health Through Less Flavor

    Messages:
    3,390
    Likes Received:
    1
    Trophy Points:
    36,885
    Location:
    Grassy Knizzle
    It'd be great if we could rip models from the game - I think I pointed out in the other thread that the Chirichan Demon model from Paul Hughes' pack is corrupt.
     
  17. AD

    AD Finder of things, Doer of stuff

    Messages:
    4,918
    Likes Received:
    102
    Trophy Points:
    36,885
    Location:
    BC, Canada
    So I mentioned your question to HCl and here's the quick answer:

    Anyway he mentioned that he'll try and look through his old stuff and see if he can remember more and possibly pop in and let us know what he finds here.
     
  18. Lin Kuei Ominae

    Lin Kuei Ominae Spaceman

    Messages:
    65
    Likes Received:
    0
    Trophy Points:
    0
    I see, another disassembling expert :D Coming from the C&C modding community, i'm quite familiar with this kind of elite modders.

    Thanks AD for forwarding my question and thanks HCl for the answer.

    I'm eager so get more info on the various P2 filetypes. :)
     
  19. HCl

    HCl WC EXE archaeologist

    Messages:
    327
    Likes Received:
    52
    Trophy Points:
    22,935
    Hey all!

    Sorry for not keeping an eye on the Forums as often as i should, i'll make an effort to keep up...

    Lin Kuei, love your editing tool, great work! :)

    Due to past e-mail exchanges with AD i took another (very quick) look at the Def! chunks in P2. I made a quick program to test things out and it turns out most of what i recalled was correct: Def! chunks are indeed compressed with zlib's deflate, which means you only have to call the inflate function to get the data back. The compressed chunks have a "Def!" identifier, followed by the uncompressed length of the data block, compressed length, and finally the deflated compressed stream.

    This means we have all we need to decompress the stream right there! Good :)

    So i basically came up with this quick program. It decompresses the first Def! chunk on SPACE.IFF (lots of stuff hardcoded, this is not a complete tool)

    (quoting most relevant sections only.. still quite large, sorry about that)

    Out of curiosity, the P2 Demo space.iff begins like this:
    and it decompresses into:

    Which is good, there is definitely some structure there! At first glance it makes sense.

    Without checking further, i suspect this particular chunk is a header that points to what may be sprite data. So you may want to parse the header and start dumping 8-bit bmps, to see if anything interesting comes up.

    Anyway, hope this is useful! If you need the small VS project i hacked together for this test, let me know.

    I'll see if I can find something on my old notes regarding the other algorithms...
     
  20. Lin Kuei Ominae

    Lin Kuei Ominae Spaceman

    Messages:
    65
    Likes Received:
    0
    Trophy Points:
    0
    This is great news HCl. Thank you. :D

    Yep, this would be great. I could then directly convert it into C#, add it to the priv2editor and enhance it this way with a file-editing section.

    Oh, this is great. I can't wait to finally have access to the raw data of priv2 and maybe be even able to change stuff.
     

Share This Page