WC3 3DO codec, compression, decoding and comparison discussion

AD

Finder of things, Doer of stuff
Since ODVS' AI enhancement thread is getting a bit long and somewhat fragmented with a few different discussions happening, I figured I would break out and try and document what we *do* know about the WC3 3DO video in one place. Over the years various people have made good progress on decoding the 3DO videos though no actual tools have ever really come about from it. What's more, not having access to the raw video files makes side by side comparisons with the PSX version's videos difficult to gain any real meaningful data from. The PSX and 3DO versions both have benefits and drawbacks and ideally it would be cool to find a way to use both versions as a kind of hybrid source for some upscale in the future. @HCl made some progress on creating a decode for the videos but didn't actually release any tools for converting them into a usable format.

I'll try and post a bit of what he did say about the format from the discussions I did have with him with the hopes that someone actually technically skilled in this stuff can actually pick up where he left off. Also note that the sample video mentioned *is* still on HCl's solsector site so if people like @ODVS want to experiment with it, it's there.

I've been making some progress lately on documenting the WC3 3DO VGA chunk. I still don't have it completely documented, but here's an update just to keep you up to speed:
>
> So far, i've been able to confirm it is indeed very close to the PC version: 22 opcodes for image operations, huffman compressed, along with a variable-byte length block, motion compensation block and encoded pixel block. The content of these smaller blocks within the VGA chunk seems to be pretty much the same between the PC and 3DO versions, with only minor differences, such as pixels in CEL format instead of 8-bit VGA palette (which the PC version further compresses with the same dictionary-based algorithm used by other WC games, such as Super Wing Commander) .
>
> As an aside, observing the evolution in cut-scene encoding between WC2-SWC-WC3 is really exciting! The principles behind WC3 movie decoding turn out not to be that much different from the WC2 take off / landing animations (although WC2 is much simpler... and usually decodes the movie over one or more still backgrounds, encoded separately and triggered at different points).
>
> Anyway, the main difference between the 3DO and PC versions seems to be the image operations used by the main algorithm. While both use 22 opcodes for encoding the image, these have different meanings on the PC and 3DO versions. Gone is an opcode which handles a 3-byte run-length (which makes sense, given the movie resolution, 2-byte lengths are always enough), while other opcodes simply do something else... so the challenge right now is to document the changes and adapt the decoder accordingly.
>
> I think i'm getting close! In between data-analysis and binary disassembly, i hope to be able to dig up this information and integrate it on my script. I'll keep you informed as I go...

________________________________________



Good news here, i finally have all the details figured out for the 3DO compression algorithm, and managed to complete my script for decoding the video. I posted a sample here:
>
> http://hcl.solsector.net/sc_13.mov.avi
>
> I'm still running a few tests to make sure no bugs remain although, given the output, if there's still any bugs remaining they're likely to be pretty minor.
>
> Overall, what we have on the 3DO is a variation of the PC algorithm, with a few changes in order to take into account the CEL hardware and 3DO frame buffer memory layout. That's basically it. You get increased color depth when compared to the PC version (and without those "sticky pixels"), and don't get the somewhat blurry artifacts of the PSX movies resulting from the JPEG-like compression.
>
> Movie resolution is actually slightly lower, since it's 300 pixels wide instead of the 320 pixels both on the PC and PSX. Interestingly, however, the VGA chunks still encode the frame changes expecting a 320-pixel wide frame buffer (probably in order to fit the 3DO frame buffer), which means that if the CEL chunks (keyframes) are smaller than that, they are rendered on the center of the buffer.
>
> You'll note the video I posted doesn't have any audio. I think it is compressed with some kind of ADPCM algorithm, i just have to figure out if the decompression is done by the CPU or the 3DO DSP, and then dig in and figure out what's going on. I expect audio quality to be fairly close to the PSX videos though.

Another barrier to 3DO video testing is that there's a few hoops to go through to actually get to the video files. Once a person breaks open the 3DO disks (use a tool like 'opera decompiler' or unCD-ROM) the videos are stored similarly to the PC version, in an archive file of sorts. Instead of a TRE file the 3DO has BUNs. The BUN format has previously been documented but as far as I know, nobody has actually released any tools to extract all the files therein. Here's some info on Bun files: http://wiki.xentax.com/index.php/Wing_Commander_3_BUN

Further, it might be helpful for anyone attempting to work with this stuff to take a look at some of the original developer documentation for hte 3DO and how the graphics processing on the system works:

Note that if you are looking for 3DO video files, the WC3 demo on the 3DO sampler CD is available for download on WCNews but it has a somewhat different video format from the Full game version.
 

AD

Finder of things, Doer of stuff
Some comparison shots were posted by @EmuMusicFan in the other thread so I figured I'd include them here. The 3DO images are screencaptures of a youtube of an analogue capture from the video-out on a 3DO



I've also taken the video sample HCl provided and put it side by side with the PSX version. Both of these are direct extracts however it's interesting is that the width difference mentioned by HCl is noticable in that it's not actually a different aspect ratio but is actually cropped slightly on the right of the picture. I don't know if this is just a quirk of the way HCls extraction script was written or if all the videos really are cut off like that. The 3DO is definitely cleaner looking in the reds and seems to have more color saturation but appears noisier elsewhere.
 
Last edited:

Pedro

Admiral
Also if anyone is interested in looking into video formats I was disappointed to learn we have *no* method for extracting WCIV comms from either platform.

On PC they seem to be very similar to the main movie files but missing the header and grayscale. On PSX they are not even in the standard libs, so we have no way of extracting them never mind decoding them.

Anyone who’d be willing into these formats would be a lifesaver.
 
Last edited:

ODVS

Petty Officer
Some bad news I'm afraid, folks.

At least for my remastering purposes, the 3DO videos are not an improvement over the PSX videos.

Since @AD managed to point us in the direction of a rip HCl managed to make in times past, I figured I might as well give it a shot and the results were... not great.

As HCl noted, " Overall, what we have on the 3DO is a variation of the PC algorithm... without those 'sticky pixels')". Unfortunately, the sticky pixels/trail are there - they're just really hard to notice because the frame size is so small and the colour depth is higher than the PC versions.

However, once the frames are upscaled, they're painfully noticeable - and they throw the AI model off something rotten.

3D01.jpg

3D02.jpg


If we compare these to the same scene ripped from the PSX discs, it's orders of magnitude cleaner:

PSX1.jpg
PSX2.jpg


It's not just the trail artefacts, either - on close inspection, the PSX videos appear to have higher colour depth, which leads to less dithering and noise in the image overall.

Sorry it's not better news - but from my end at least, I don't think there's much point undertaking the work to extract the 3DO FMVs.
 

AD

Finder of things, Doer of stuff
...

However, once the frames are upscaled, they're painfully noticeable - and they throw the AI model off something rotten.
...
If we compare these to the same scene ripped from the PSX discs, it's orders of magnitude cleaner:...


Sorry it's not better news - but from my end at least, I don't think there's much point undertaking the work to extract the 3DO FMVs.
As it currently works, I'd agree, but I'm also curious about whether there weren't still bugs in HCls script. The 3DO may very well have some levels of processing that weren't present in his output. In the comparison shots captured from a 3DO there doesn't seem to be the level of noise present in HCl's sample. Still, in general the PSX version does seem overall smoother, but some of that is also the system introducing smooshing (technical term ;) ) to the image which also removes some detail. The one area that the 3DO is night and day better is the definition in the reds. The edges also seem a lot more pixely in the PSX version. The cropping is still present in EmuMusicFan's screenshot comparisons though so that's kind of a bummer.

While I don't think it's a rush to work out a decoding process I still think it's something we should do. Also in the future the AI should be able to compare both versions, take the red channel detail out of the 3DO and use the rest from the PSX version. I'd love to see that. It's already possible to do a "stacking" version of videos, but were just waiting on a practical implementation.
 

ODVS

Petty Officer
While I don't think it's a rush to work out a decoding process I still think it's something we should do.
Fair point - it would be good to have them archived anyway. And as you say, it's a proto-process - there could well be cleaner rips to be mined from the source material. Sorry to come across so pessimistic! 😝
 

Pedro

Admiral
if only we could get the HD upscaled videos modded into WC3KS
This is definitely possible; I bet if HCl was still around he could do it in no time riding off his existing work for WC4/P DVD. Grim/PopsiclePete are the two other possibilites.

If the WC4 remake goes well I'd certainly add support for WC3 files, but that is many years off.
 

EmuMusicFan

Chief Petty Officer
The BUN format has previously been documented but as far as I know, nobody has actually released any tools to extract all the files therein. Here's some info on Bun files: http://wiki.xentax.com/index.php/Wing_Commander_3_BUN
I just finished writing a BUN unpacker for Python3.

It is Very simple. I tested it with Disc 4. This seems to be correct in terms of the size of the files, e.g. sc_178 is similar in size to the PSX version, very large.

Usage:

If you have Python3 support, just drag the BUN on the deBUN.py and drop.
If you do not like to install Python3, Just download this: https://www.python.org/ftp/python/3.8.4/python-3.8.4-embed-amd64.zip, unpack it to a folder. In CMD window cd to python-3.8.4-embed-amd64, then run command like this:

python.exe F:\YOUR_PATH_TO_THIS_TOOL\deBUN.py F:\YOUR_PATH_TO_BUN\commov.bun

EDIT:

v1.0 released! Please use the version of "deBUN-release_v1.0.zip".
 

Attachments

Last edited:
  • Like
Reactions: AD

AD

Finder of things, Doer of stuff
I just finished writing a BUN unpacker for Python3.

It is Very simple. I tested it with Disc 4. This seems to be correct in terms of the size of the files, e.g. sc_178 is similar in size to the PSX version, very large.

Usage:

If you have Python3 support, just drag the BUN on the deBUN.py and drop.
If you do not like to install Python3, Just download this: https://www.python.org/ftp/python/3.8.4/python-3.8.4-embed-amd64.zip, unpack it to a folder. In CMD window cd to python-3.8.4-embed-amd64, then run command like this:

python.exe F:\YOUR_PATH_TO_THIS_TOOL\deBUN.py F:\YOUR_PATH_TO_BUN\commov.bun
This is awesome. Thanks for tackling this. However I'm running into some errors trying to do the version where I don't install python. If I use that do I need to have an AMD chip? (edit - I just installed python... it seemed simpler and it works so yay!)

Traceback (most recent call last):
File "e:\wc3_3do\WC3movies\deBUN.py", line 26, in main
with open(sys.argv[1], "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'e:\\wc3_3do\\WC3movies\\disk1mov.bun'
On another note the STR files in the STREAMS directory are also archive files of some sort. There's some interesting info commented out at the start of the Gameflow and mission str files too. The files contain IFFs, sound effects, and other files. It would be interesting to see as there's ships in the 3DO version missions that are present in the PC game files but were cut from the finished game but that show up in gameplay in the 3DO version.

The commented out code describes a lot of variable that are then specified in the next line of code. It also seems to describe how the archive works to a degree and then lists the complete file structure of all the contained files.
Code:
__# Example STREAMER Script --

#

# This file is fed into 'streamer' which contructs a map file that then is fed into

# 'stream_build'.

#

# <rol> 9-17-94

# <pci> 10-18-94



# 'Root' is the base directory for all Script file references

# MPW variables may be accessed using standard {variable} notation



Root /fs/r3/wc3/streamer/



# 'Remote' is relative to 'Root' where all FILE data is located



Remote remote/



# System ticks per second (Hz)



system_ticks_second 225



# System seek rate, how many ticks does it take to seek to a file position



stream_seek_rate 120



# The transfer rate of the stream, what is the bandwidth?

# in bytes/second



#stream_transfer_rate 276480

stream_transfer_rate 165536



# The Audio Stream, this should be a sound designer II file



audio remote/wc3/Streams/audio/mission.sd2



# The Audio Channels, 1 for mono, 2 for stereo



audio_channels 2



# audio_byte_width 1 for 8-bit samples, 2 for 16-bit samples



audio_byte_width 2



# audio_rate, sample rate at which sound was recorded



audio_rate 22050



# audio_buffer_size, the maximum size of an audio chunk



audio_buffer_size 65536



# The pathfinder file: if no pathfinder file is given, then the audio file is considered an aiff

# that should be played straight through



pathfinder remote/wc3/Streams/audio/mission.path



# file chunk size: Files will be cut into chunks of this size



file_chunk_size 65536



# minimum_compression: What percentage gain must refpack get to bother compressing a file



minimum_compression 100



# block size: CD-ROM or whatever block size of target platform, files in stream are aligned to this



block_size 2048



# List of data files that are to be in the stream

#

# file path/filename.ext priority

#

# priority => bigger numbers have relatively higher priority



# the response file is included in the stream for debugging

file wc3/Streams/mission.scr 0



group sim_mission_1

{

file wc3/gamedat/SR00MIS0.BIN 0

}



group sim_mission_2

{

file wc3/gamedat/SR00MIS1.BIN 0

}



group sim_mission_3

{

file wc3/gamedat/SR00MIS2.BIN 0

}



group sim_mission_4

{

file wc3/gamedat/SR00MIS3.BIN 0

}



group mission_a1

{

file wc3/gamedat/SR01MIS0.BIN 0

}



group mission_a2

{
There's some pretty useful info in there for determining audio formats and such. For comparison that was mission.str and the next is the same block from the gameflowXXX.str file:

Code:
_____________________________________________________________________________________________________________________# gameflow stream script --
#
# This file is fed into 'stream_map' which contructs a map file that then is fed into
# 'stream_build'.
#
# <rol> 9-17-94

# 'Root' is the base directory where the files can be found, since different computers may be
# compiling this script {Storm} is a MPW variabile exported by <pci> UserStartup.Storm giving
# the path to the STORM directory.
#
# NOTE/ All file paths in this file are prefixed by the 'Root' specified below.
#

# 'Root' is the base directory for all Script file references
# MPW variables may be accessed using standard {variable} notation

Root                     /fs/r3/wc3/streamer/

# 'Remote' is relative to 'Root' where all FILE data is located

Remote                    remote/

# System ticks per second

system_ticks_second        225

# System seek rate, how many ticks does it take to seek to a file position

stream_seek_rate        120          

# The transfer rate of the stream, what is the bandwidth?
# in bytes/second

stream_transfer_rate    176480

# The Audio Stream, this should be a sound designer II file

audio                     remote/wc3/Streams/audio/gameflow_cd1.sd2

# The Audio Channels, 1 for mono, 2 for stereo

audio_channels            2

# audio_byte_width 1 for 8-bit samples, 2 for 16-bit samples

audio_byte_width        2

# audio_rate, sample rate at which sound was recorded

audio_rate                 22050

# audio_buffer_size, the maximum size of an audio chunk

audio_buffer_size        65536

# The pathfinder file, if no pathfinder file is given, then the audio file is considered an aiff
# that should be played straight through

pathfinder                remote/wc3/Streams/audio/gameflow_cd1.path

# Since RefPack is being used to compress the data in the stream, and refpack requires the entire
# data to be in memory for it to be decompressed, this requires that we break the data into chunks
# limited to the bytes specified below.

file_chunk_size            65536
minimum_compression        100
block_size            2048

# List of data files that are to be in the stream

# parameters ---
# filename    -     the filename of the data files
# priority    -     the priority level of the data, the lower the number, the higher the priority

#        filename        priority

# response file is included in stream for debugging
file    wc3/Streams/gameflow_cd1.scr       0  

# sprites
file    wc3/gamedat/warpcore.BIN           0

group bin_files
{
    file    wc3/gamedat/movienames.BIN     0
    file    wc3/gamedat/premix.BIN         0
    file    wc3/gamedat/movies.BIN         0
    file    wc3/gamedat/Spritenames.BIN    0
    file    wc3/gamedat/version.BIN           0
    file    wc3/gamedat/sstable.BIN           0
    file    wc3/gamedat/scenetable.BIN       0
}

#sound effects

group gameflow_sfx
{
    file     wc3/sound/fx52.IFF        0
    file     wc3/sound/fx60.IFF        0
    file     wc3/sound/fx63.IFF        0
    file     wc3/sound
 

EmuMusicFan

Chief Petty Officer
This is awesome. Thanks for tackling this. However I'm running into some errors trying to do the version where I don't install python. If I use that do I need to have an AMD chip? (edit - I just installed python... it seemed simpler and it works so yay!)
Sorry, is it working properly now?

It seems that “e:\\wc3_3do\\WC3movies\\disk1mov.bun” looks like a strange path. It should be something like “e:\wc3_3do\WC3movies\disk1mov.bun” How did you write the command line?
 

AD

Finder of things, Doer of stuff
I definitely wasn't putting two slashes in there when typing the line so I'm not sure why it was returning the error like that. Either way I just installed python instead which so far is working pretty good. However it is seeing videos in the disks 2 and 3 BUNs but doesn't seem to be actually extracting anything. It's worked well for the common file, disks 1 and 4 and all the inflight files.
 

AD

Finder of things, Doer of stuff
Would you please post the output log text of Disk 2 and 3?

I have no DIsc 2 and 3 files at hand for now.
Code:
The simple BUN unpacker by EmuMuiscFan.
BUN Loaded!
Processing DIR_ Trunk...
DIR_ Trunk payload length: 576
file_payload_begin: 0x25c
file_payload_begin: 0x25c
['compression_type=0\r', '\r', 'disc2mov.rsp\r', 'wc3-movies-sc_50.mov\r', 'wc3-movies-sc_51.mov\r', 'wc3-movies-sc_53.mov\r', 'wc3-movies-sc_54.mov\r', 'wc3-movies-sc_A54.mov\r', 'wc3-movies-sc_56.mov\r', 'wc3-movies-sc_58.mov\r', 'wc3-movies-sc_A58.mov\r', 'wc3-movies-sc_60.mov\r', 'wc3-movies-sc_61.mov\r', 'wc3-movies-sc_63.mov\r', 'wc3-movies-sc_A63.mov\r', 'wc3-movies-sc_B64.mov\r', 'wc3-movies-sc_B65.mov\r', 'wc3-movies-sc_67.mov\r', 'wc3-movies-sc_68.mov\r', 'wc3-movies-sc_69.mov\r', 'wc3-movies-sc_70.mov\r', 'wc3-movies-sc_71.mov\r', 'wc3-movies-sc_72.mov\r', 'wc3-movies-sc_73.mov\r', 'wc3-movies-sc_75.mov\r', 'wc3-movies-sc_76.mov\r', 'wc3-movies-sc_77.mov\r', 'wc3-movies-sc_A78.mov\r', 'wc3-movies-sc_80.mov\r', 'wc3-movies-sc_81.mov\r', 'wc3-movies-sc_82.mov\r', 'wc3-movies-sc_83.mov\r', 'wc3-movies-sc_84.mov\r', 'wc3-movies-sc_85.mov\r', 'wc3-movies-sc_87.mov\r', 'wc3-movies-sc_88.mov\r', 'wc3-movies-sc_A89.mov\r', 'wc3-movies-sc_91.mov\r', 'wc3-movies-sc_A91.mov\r', 'wc3-movies-sc_B92.mov\r', 'wc3-movies-sc_A93.mov\r', 'wc3-movies-sc_95.mov\r', 'wc3-movies-sc_a121.mov', 'wc3-movies-sc_209.mov\r', 'wc3-movies-sc_223.mov\r', 'wc3-movies-sc_502.mov\r', 'wc3-movies-sc_503.mov\r', 'wc3-movies-sc_504.mov\r', 'wc3-movies-sc_513.mov\r', 'wc3-movies-sc_515.mov\r', '']
file_payload_begin: 0x6c4
file_payload_begin: 0x6c4
No. 1wc3-movies-sc_50.mov
Writing MOV Error!
Traceback (most recent call last):
  File "E:\wc3_3do\WC3Movies\deBUN.py", line 70, in main
    with open (filepath + single_filename, "wb") as f:
OSError: [Errno 22] Invalid argument: 'E:\\wc3_3do\\WC3Movies/wc3-movies-sc_50.mov\r'
And disk 3


Code:
BUN Loaded!
Processing DIR_ Trunk...
DIR_ Trunk payload length: 516
file_payload_begin: 0x220
file_payload_begin: 0x220
['compression_type=0\r', '\r', 'disc3mov.rsp\r', 'wc3-movies-sc_95.mov', 'wc3-movies-sc_97.mov\r', 'wc3-movies-sc_98.mov', 'wc3-movies-sc_100.mov\r', 'wc3-movies-sc_101.mov\r', 'wc3-movies-sc_103.mov\r', 'wc3-movies-sc_104.mov\r', 'wc3-movies-sc_106.mov\r', 'wc3-movies-sc_109.mov\r', 'wc3-movies-sc_110.mov\r', 'wc3-movies-sc_111.mov\r', 'wc3-movies-sc_112.mov\r', 'wc3-movies-sc_b113.mov\r', 'wc3-movies-sc_114.mov\r', 'wc3-movies-sc_115.mov\r', 'wc3-movies-sc_116.mov\r', 'wc3-movies-sc_117.mov\r', 'wc3-movies-sc_118.mov\r', 'wc3-movies-sc_a118.mov\r', 'wc3-movies-sc_120.mov\r', 'wc3-movies-sc_a121.mov\r', 'wc3-movies-sc_121.mov\r', 'wc3-movies-sc_A122.mov\r', 'wc3-movies-sc_124.mov\r', 'wc3-movies-sc_125.mov\r', 'wc3-movies-sc_127.mov\r', 'wc3-movies-sc_128.mov\r', 'wc3-movies-sc_130.mov\r', 'wc3-movies-sc_132.mov\r', 'wc3-movies-sc_134.mov\r', 'wc3-movies-sc_135.mov\r', 'wc3-movies-sc_137.mov\r', 'wc3-movies-sc_139.mov\r', 'wc3-movies-sc_140.mov\r', 'wc3-movies-sc_209.mov\r', 'wc3-movies-sc_224.mov\r', 'wc3-movies-sc_227.mov\r', 'wc3-movies-sc_501.mov\r', 'wc3-movies-sc_503.mov\r', 'wc3-movies-sc_506.mov\r', 'wc3-movies-sc_507.mov\r', 'wc3-movies-sc_512.mov\r', '']
file_payload_begin: 0x660
file_payload_begin: 0x660
No. 1wc3-movies-sc_95.mov
file_payload_begin: 0xeea21c
file_payload_begin: 0xeea21c
No. 2wc3-movies-sc_97.mov
Writing MOV Error!
Traceback (most recent call last):
  File "E:\wc3_3do\WC3Movies\deBUN.py", line 70, in main
    with open (filepath + single_filename, "wb") as f:
OSError: [Errno 22] Invalid argument: 'E:\\wc3_3do\\WC3Movies/wc3-movies-sc_97.mov\r'
 

EmuMusicFan

Chief Petty Officer
Please try this new version and post the logs.

Actually I am confused why there are so many special characters, like "\t" (TAB) and "\r" (carriage return), in the file name list. I just remove them.
 

Attachments

AD

Finder of things, Doer of stuff
Please try this new version and post the logs.

Actually I am confused why there are so many special characters, like "\t" (TAB) and "\r" (carriage return), in the file name list. I just remove them.
Thanks, I'll have a look after work
 

AD

Finder of things, Doer of stuff
The test version seems to work:

Code:
E:\wc3_3do\WC3Movies\deBUNtest.py E:\wc3_3do\WC3Movies\disc2mov.bun
The simple BUN unpacker by EmuMuiscFan.
BUN Loaded!
Processing DIR_ Trunk...
DIR_ Trunk payload length: 576
file_payload_begin: 0x25c
file_payload_begin: 0x25c
['compression_type=0', '', 'disc2mov.rsp', 'wc3-movies-sc_50.mov', 'wc3-movies-sc_51.mov', 'wc3-movies-sc_53.mov', 'wc3-movies-sc_54.mov', 'wc3-movies-sc_A54.mov', 'wc3-movies-sc_56.mov', 'wc3-movies-sc_58.mov', 'wc3-movies-sc_A58.mov', 'wc3-movies-sc_60.mov', 'wc3-movies-sc_61.mov', 'wc3-movies-sc_63.mov', 'wc3-movies-sc_A63.mov', 'wc3-movies-sc_B64.mov', 'wc3-movies-sc_B65.mov', 'wc3-movies-sc_67.mov', 'wc3-movies-sc_68.mov', 'wc3-movies-sc_69.mov', 'wc3-movies-sc_70.mov', 'wc3-movies-sc_71.mov', 'wc3-movies-sc_72.mov', 'wc3-movies-sc_73.mov', 'wc3-movies-sc_75.mov', 'wc3-movies-sc_76.mov', 'wc3-movies-sc_77.mov', 'wc3-movies-sc_A78.mov', 'wc3-movies-sc_80.mov', 'wc3-movies-sc_81.mov', 'wc3-movies-sc_82.mov', 'wc3-movies-sc_83.mov', 'wc3-movies-sc_84.mov', 'wc3-movies-sc_85.mov', 'wc3-movies-sc_87.mov', 'wc3-movies-sc_88.mov', 'wc3-movies-sc_A89.mov', 'wc3-movies-sc_91.mov', 'wc3-movies-sc_A91.mov', 'wc3-movies-sc_B92.mov', 'wc3-movies-sc_A93.mov', 'wc3-movies-sc_95.mov', 'wc3-movies-sc_a121.mov', 'wc3-movies-sc_209.mov', 'wc3-movies-sc_223.mov', 'wc3-movies-sc_502.mov', 'wc3-movies-sc_503.mov', 'wc3-movies-sc_504.mov', 'wc3-movies-sc_513.mov', 'wc3-movies-sc_515.mov', '']
1
file_payload_begin: 0x6c4
file_payload_begin: 0x6c4
No. 1 wc3-movies-sc_50.mov
2
file_payload_begin: 0x148ea7c
file_payload_begin: 0x148ea7c
No. 2 wc3-movies-sc_51.mov
3
file_payload_begin: 0x1be7b14
file_payload_begin: 0x1be7b14
No. 3 wc3-movies-sc_53.mov
4
file_payload_begin: 0x22ebbf8
file_payload_begin: 0x22ebbf8
No. 4 wc3-movies-sc_54.mov
5
file_payload_begin: 0x29e1db8
file_payload_begin: 0x29e1db8
No. 5 wc3-movies-sc_A54.mov
6
file_payload_begin: 0x2d045cc
file_payload_begin: 0x2d045cc
No. 6 wc3-movies-sc_56.mov
7
file_payload_begin: 0x40027ec
file_payload_begin: 0x40027ec
No. 7 wc3-movies-sc_58.mov
8
file_payload_begin: 0x4617274
file_payload_begin: 0x4617274
No. 8 wc3-movies-sc_A58.mov
9
file_payload_begin: 0x4a625a8
file_payload_begin: 0x4a625a8
No. 9 wc3-movies-sc_60.mov
10
file_payload_begin: 0x5cb3e2c
file_payload_begin: 0x5cb3e2c
No. 10 wc3-movies-sc_61.mov
11
file_payload_begin: 0x63ef250
file_payload_begin: 0x63ef250
No. 11 wc3-movies-sc_63.mov
12
file_payload_begin: 0x68d449c
file_payload_begin: 0x68d449c
No. 12 wc3-movies-sc_A63.mov
13
file_payload_begin: 0x6baccdc
file_payload_begin: 0x6baccdc
No. 13 wc3-movies-sc_B64.mov
14
file_payload_begin: 0x7135630
file_payload_begin: 0x7135630
No. 14 wc3-movies-sc_B65.mov
15
file_payload_begin: 0x787f3b0
file_payload_begin: 0x787f3b0
No. 15 wc3-movies-sc_67.mov
16
file_payload_begin: 0x7a52c10
file_payload_begin: 0x7a52c10
No. 16 wc3-movies-sc_68.mov
17
file_payload_begin: 0x8137420
file_payload_begin: 0x8137420
No. 17 wc3-movies-sc_69.mov
18
file_payload_begin: 0x8f0bfc4
file_payload_begin: 0x8f0bfc4
No. 18 wc3-movies-sc_70.mov
19
file_payload_begin: 0x9517474
file_payload_begin: 0x9517474
No. 19 wc3-movies-sc_71.mov
20
file_payload_begin: 0x9d93f54
file_payload_begin: 0x9d93f54
No. 20 wc3-movies-sc_72.mov
21
file_payload_begin: 0xa62b0c8
file_payload_begin: 0xa62b0c8
No. 21 wc3-movies-sc_73.mov
22
file_payload_begin: 0xade9ac8
file_payload_begin: 0xade9ac8
No. 22 wc3-movies-sc_75.mov
23
file_payload_begin: 0xb5adce4
file_payload_begin: 0xb5adce4
No. 23 wc3-movies-sc_76.mov
24
file_payload_begin: 0xbc2d6a0
file_payload_begin: 0xbc2d6a0
No. 24 wc3-movies-sc_77.mov
25
file_payload_begin: 0xc3e12bc
file_payload_begin: 0xc3e12bc
No. 25 wc3-movies-sc_A78.mov
26
file_payload_begin: 0xc7df914
file_payload_begin: 0xc7df914
No. 26 wc3-movies-sc_80.mov
27
file_payload_begin: 0xd580de8
file_payload_begin: 0xd580de8
No. 27 wc3-movies-sc_81.mov
28
file_payload_begin: 0xe64b030
file_payload_begin: 0xe64b030
No. 28 wc3-movies-sc_82.mov
29
file_payload_begin: 0xea864b4
file_payload_begin: 0xea864b4
No. 29 wc3-movies-sc_83.mov
30
file_payload_begin: 0xf9916d0
file_payload_begin: 0xf9916d0
No. 30 wc3-movies-sc_84.mov
31
file_payload_begin: 0x11ab82d0
file_payload_begin: 0x11ab82d0
No. 31 wc3-movies-sc_85.mov
32
file_payload_begin: 0x124c08e4
file_payload_begin: 0x124c08e4
No. 32 wc3-movies-sc_87.mov
33
file_payload_begin: 0x12e63528
file_payload_begin: 0x12e63528
No. 33 wc3-movies-sc_88.mov
34
file_payload_begin: 0x131c5610
file_payload_begin: 0x131c5610
No. 34 wc3-movies-sc_A89.mov
35
file_payload_begin: 0x1351eaf0
file_payload_begin: 0x1351eaf0
No. 35 wc3-movies-sc_91.mov
36
file_payload_begin: 0x14679e20
file_payload_begin: 0x14679e20
No. 36 wc3-movies-sc_A91.mov
37
file_payload_begin: 0x149b6680
file_payload_begin: 0x149b6680
No. 37 wc3-movies-sc_B92.mov
38
file_payload_begin: 0x14d471b8
file_payload_begin: 0x14d471b8
No. 38 wc3-movies-sc_A93.mov
39
file_payload_begin: 0x14fec390
file_payload_begin: 0x14fec390
No. 39 wc3-movies-sc_95.mov
40
file_payload_begin: 0x15ed5f4c
file_payload_begin: 0x15ed5f4c
No. 40 wc3-movies-sc_a121.mov
41
file_payload_begin: 0x16135ba8
file_payload_begin: 0x16135ba8
No. 41 wc3-movies-sc_209.mov
42
file_payload_begin: 0x166fadb0
file_payload_begin: 0x166fadb0
No. 42 wc3-movies-sc_223.mov
43
file_payload_begin: 0x16f6bd10
file_payload_begin: 0x16f6bd10
No. 43 wc3-movies-sc_502.mov
44
file_payload_begin: 0x1721113c
file_payload_begin: 0x1721113c
No. 44 wc3-movies-sc_503.mov
45
file_payload_begin: 0x176192d8
file_payload_begin: 0x176192d8
No. 45 wc3-movies-sc_504.mov
46
file_payload_begin: 0x178b74c8
file_payload_begin: 0x178b74c8
No. 46 wc3-movies-sc_513.mov
47
file_payload_begin: 0x17a32240
file_payload_begin: 0x17a32240
No. 47 wc3-movies-sc_515.mov
 

AD

Finder of things, Doer of stuff
I've put together a sample pack of the videos extracted from WC3 3DO, the WC3 3DO deo, and SWC for the 3DO which all - to my knowledge - have differing video formats despite having a *.mov extension. I've also included comm video files as well. https://drive.google.com/file/d/15UZQV3oP2x_Ia0oNF3PgzHCLMf39pCRB/view?usp=sharing


HCl did previously work on deciphering the SWC videos and CEL image format, and released extracted and converted packages of the audio and video files. However I don't think he released any actual tools for this either. Though he did leave a number of notes about the file formats. For anyone interested in figuring out the WC3 stuff it's possible this stuff could be useful, or maybe not, but he ends the entry with some info and musings about WC3 3DO format. But anyway you can download the movie and audio packs and find the info here near the bottom of the page : http://hcl.solsector.net/

Discussion of SWC and CEL format
...when i was able to get a copy of the 3DO version of Super Wing Commander, I was pretty excited and I started to take a look at the game files. Sure enough, most of the game assets are coded in CEL format, in order to take advantage of the specialized CEL processor. The CEL format supports a number of pixel-formats, as well coded and uncoded modes, packed and unpacked... although, for all intents and purposes it can be thought of as an RLE-based format. The chunks used by these files pretty much match the ones created by the official 3DO SDK apps, so anyone familiar with these will feel pretty much at home when looking at SWC data.

With my hands on some reference documentation available on the net, i decided to make a script in order to extract SWC CELs. After a bit of hacking, i had what i needed to unpack CELs generated by the 3DO SDK as well as SWC data. Watching the first ship CELs show up was really exciting!

Then i turned my attention to the movies. Given the specs of the 3DO, it made sense that the game's FMV was also based on the CEL format so, since i already had a basic CEL extractor in place, i decided to go the extra mile and see if FMV sequences could also be extracted. As it turns out, they can! SWC movies basically use 16-bit CELs in order to code the differences between consecutive frames, each of them further compressed with an LZ77-based algorithm (with variable-bit encoding for lengths and offsets).

The FMV digital audio chunks are also compressed with this LZ77-based algorithm... which results in expansion more often than not, but works really well on silent areas of the audio track (all-zero values, which are really not as difficult to occur as all that, since the movies usually are played alongside with a MIDI track, triggered somewhere on the executable).

By the way, this is not very different from the way Wing Commander 2 sequences are encoded, with WC2's base RLE algorithm being used to encode image segments corresponding to frame differences. WC2 animation frames, by the way, are grouped together into ~64k-long blocks which are preferentially loaded into EMS memory, and optionally compressed with a form of LZW. As a curiosity, most WC2 assets are LZW-compressed, but can be decompressed at install-time optionally (the "Save Time or Save Space" option available on the floppy install).

This approach ended up being used by Academy, and all the way up to Armada, for coding the various cut-scenes.

Thoughts on WC3 video format:
As for the console ports of WC3, the PSX movie compression is pretty much hardware-based (MDEC) with a small software-based decompression step based on a set of variable-length codes. WC3 3DO, on the other hand was a clear adaptation of the WC3 PC algorithm, with a few changes here and there, mostly in order to take into account the 3DO CEL engine and frame-buffer memory layout. Interestingly, it supports some pixels coded in 16-bit precision and others in 24-bit precision (although the color components of these 24-bit values come from a limited set of 8-bit values).

I still need to look at the audio track of WC3 3DO videos before I start extracting those. As soon as the day job allows it, i hope to put some more time into it. I expect these to be compressed with a form of ADPCM, probably running on the 3DO DSP... but we'll see what shows up once i further disassemble the 3DO binary.
 

EmuMusicFan

Chief Petty Officer
My guess is that the PSX version might be better suited as a focus for future research into super-resolution... in sections with blue as the main color.
The PSX MDEC video, a MJPEG-like format, is a more modern format, although WC3 FMVs come with a very low bit rate and severe compression artifacts. There are a number of studies that are trying to work towards eliminating compression artifacts from images and videos, and some papers give very good results. I hope some will be available for practical use soon.

The 3DO version... It seems there is not enough blue colors in the palette. The reds are pretty good.
 
Last edited:

AD

Finder of things, Doer of stuff
The 3DO version... It seems there is not enough blue colors in the palette. The reds are pretty good.
Yeah, the blue space swirls in one of the previous screenshots definitely look better on the PSX. But in the same shot all the ships are a lot cleaner looking in the 3DO version. In general the "aliasing" on edges is overall better on the 3DO even on blue uniforms. I did up a few more straight frame comparisons with the video HCl sampled with the PSX version and did a simple 2x resize on them as well. Even on blue text the PSX version is just blockier. 3DO is the first image in each set:

1x
3DO1.jpg
PSX1.jpg

3DO2.jpg
PSX2.jpg

3DO3.jpg
PSX3.jpg

3DO4.jpg
PSX4.jpg

3DO5.jpg
PSX5.jpg
 
Top