Automatic Version Check File Patching

[Ribose]

I've been curious recently whether it would be feasable to make a C# bot download patch files and extract the version check files and replace the old ones using MBNCSutil's bnftp and mpq classes. I've already got it to successfully download the MPQ file using the name provided in the version check fail message, but I don't know how to find the hash files in there. (the only known files are exe and the (attributes), (listfile), and (signature) (according to the MPQ utility I'm using).

This is with the 1.15.3 patch for StarCraft: Broodwar.

[Barabajagal]

UL knows all about it. It's a well kept secret, apparently.

[Ribose]

UL knows all about it. It's a well kept secret, apparently.

I saw that topic, but yeah, all I know is its an MPQ and it has no files in the listfile so I can't tell which files are the hashes.

Actually, I looked at the WarCraft III patch (last one that occurred) and saw that it has a file called Patch_WAR3.mpq inside the mpq, and that has the patch files inside it... but for some reason StarCraft's patch doesn't have the names. :/

[Barabajagal]

They're different systems.

[Ribose]

I guess that is logical with the observed results, however that doesn't help much in learning about patching StarCraft with the MPQ.

I don't want to go and make an ugly big function that calls StarCraft to do it, like people did with Warden before solving* that. Besides, this isn't as hard as Warden, is it? :O Maybe it is, but I've seen people have already figured out how MPQs store stuff.

[Ringo]

You could just cheat and patch the game via you're bot by taking the mpq file, for example:
SEXP_IX86_1152_1153.mpq
Replace ".mpq" with ".exe"
Take "1153.exe", somthing like Tag = StrReverse(GetSTRING(StrReverse(File), "_"))
Connect to "ftp.blizzard.com"
Download "/pub/broodwar/patches/PC/BW-" & Tag
Execute the exe.
Wait awhile or track the patching progress.
Copy the binarys from you're starcraft folder to the bot's folder (if thats how you're bot works)
Take a guess at the version byte, based on the old version byte.
blah blah blah.

I had a little look into extracting the binarys once, but iirc, the files dont have a PE header so i'm not sure whats up with that -- might of been an error extracting them, it was along time ago.
But yeah, if I really wanted to add auto patching, I would just do the above, it's simple, it work's and to some degree, its fail safe(?)
The patch exe's have the same MPQ you would download from battle.net in them anyway -- it just appends the executable.
If you REALLY want to do it the hard way, I advise reverseing the patch exe to see what it does -- it should all be in there.

[Barabajagal]

Uh... except that requires the actual game anyway...

[Ribose]

You could just cheat and patch the game via you're bot by taking the mpq file, for example:
SEXP_IX86_1152_1153.mpq
Replace ".mpq" with ".exe"
Take "1153.exe", somthing like Tag = StrReverse(GetSTRING(StrReverse(File), "_"))
Connect to "ftp.blizzard.com"
Download "/pub/broodwar/patches/PC/BW-" & Tag
Execute the exe.
Wait awhile or track the patching progress.
Copy the binarys from you're starcraft folder to the bot's folder (if thats how you're bot works)
Take a guess at the version byte, based on the old version byte.
blah blah blah.

I had a little look into extracting the binarys once, but iirc, the files dont have a PE header so i'm not sure whats up with that -- might of been an error extracting them, it was along time ago.
But yeah, if I really wanted to add auto patching, I would just do the above, it's simple, it work's and to some degree, its fail safe(?)
The patch exe's have the same MPQ you would download from battle.net in them anyway -- it just appends the executable.
If you REALLY want to do it the hard way, I advise reverseing the patch exe to see what it does -- it should all be in there.

That's the kind of work-around I wanted to avoid because I have a hunch the public patches don't work if StarCraft isn't installed.

[tA-Kane]

I had a little look into extracting the binarys once, but iirc, the files dont have a PE header so i'm not sure whats up with that -- might of been an error extracting them, it was along time ago.
But yeah, if I really wanted to add auto patching, I would just do the above, it's simple, it work's and to some degree, its fail safe(?)
The patch exe's have the same MPQ you would download from battle.net in them anyway -- it just appends the executable.
If you REALLY want to do it the hard way, I advise reverseing the patch exe to see what it does -- it should all be in there.

Might I suggest the idea that if it's actually a patch file, there wouldn't be any complete binary file in it. If there were binary files in there ready to simply extract, it would simply be a compressed and/or encrypted storage file.

I suggest you look into actually reading the data in the MPQ and actually patching your binaries. I'm unfamiliar with whatever patching mechanism Blizzard uses. Some companies use RTPatch (JFGI) rather than making their own proprietary patching system, so that could be a good starting point for comparison of reverse-engineered code.

[Barabajagal]

If you're gonna reverse engineer anything, wouldn't BNUpdate.exe be the file to look at?

[Ringo]

That's the kind of work-around I wanted to avoid because I have a hunch the public patches don't work if StarCraft isn't installed.

True, it was just an option
I haven't really looked into this in detail, but i'm sure you only need a few starcraft files in a directory and maybe a registery value or 2 -- if stardat etc needs to be there, then yeah, to much hastle, I guess.

Might I suggest the idea that if it's actually a patch file, there wouldn't be any complete binary file in it. If there were binary files in there ready to simply extract, it would simply be a compressed and/or encrypted storage file.

I suggest you look into actually reading the data in the MPQ and actually patching your binaries. I'm unfamiliar with whatever patching mechanism Blizzard uses. Some companies use RTPatch (JFGI) rather than making their own proprietary patching system, so that could be a good starting point for comparison of reverse-engineered code.

Well, it was some years ago I tryed this, I  wunderd if the binarys would be extracted and ready to run, but after extracting them from the patch mpq, they didn't look like they had a PE header.
At that point, I didn't look into it in any more depth -- it was mainly just to answer a question.

If you're gonna reverse engineer anything, wouldn't BNUpdate.exe be the file to look at?

Yes, but the patch exe would be the better place to start I would have thought, since BNUpdate would require not only the game to be installed, but also running.
I haven't looked at the patch exe, but i'm pretty sure it creates a fake starcraft window, extracts the BNupdate.exe and runs it.
I'm sure theres alittle/alot more to it than that, but I persionaly would start with the patch exe it's self before poking around with BNUpdate.exe.
I think it would make more sence to try getting away with executing the BNUpdate.exe and let it do all the work, since BNUpdate comes with the patch, you never know for sure if it's going to follow a standard.

I might poke around with this if I get really bored some day starting with the patch exe -- I figger, if you know everything you need to run BNUpdate.exe then it's just a case of supplying it and running the update exe.
I think even doing it that way tho, blizzard could get bitchy and change BNUpdate to check for more install related stuff, making it harder to run.
If that was the case, you will run into bigger problems when trying to patch the binarys with you're own code, since I think BNUpdate only knows how and what to do for a said patch.
I really don't know a great deal about this, so don't quote me on this stuff.

[Barabajagal]

No, I mean... BNUpdate tends to be one of the only executable files in the patch MPQ, so I'm fairly certain it IS the patch exe.

[Ringo]

No, I mean... BNUpdate tends to be one of the only executable files in the patch MPQ, so I'm fairly certain it IS the patch exe.

It is, but, for example, BW-1153.exe contains the patch mpq and set's everything up, so BNUpdate.exe will run.
I just tested a few things, and the only things needed to patch brood war, are starcraft.exe, stardat.mpq and broodat.mpq.
BNUpdate.exe wont run, unless starcraft is running. So I figger that is what the patch exe does, creates a fake window.
From what I can see, the main reassion you need the stardat.mpq and broodat.mpq, is somthing to do with StarEdit.exe.
Every other file is created when patching (or extracted from the dat mpq's, then patched)
You could possibly get away with building smaller dat mpqs with just the needed files in them for bnupdate.exe to run, but i'm not sure how easy that would be.
Just by doing these few tests, it looks like the patch exe checks if starcraft's install folder exists, extracts the bnupdate.exe to it, creates a fake starcraft window, then executes the BNUpdate.exe.
Then what it looks like BNUpdate.exe is doing, is checks "starcraft.exe", "stardat.mpq" and "broodat.mpq" exists, before patching/doing it's thing.
If you can emulate the patch exe, in theory, you could just extract and run the bnupdate.exe -- that's still going to require you have the dat mpq's tho.
Emulating the whole BNUpdate.exe, in my eyes, could possibly be more hastle than it's worth, but at least, you wouldnt need 100mb of dat mpq's, since you would just be emulating the needed functions to patch the 3 checkrevision binarys.
Running BNUpdate.exe seems the better option for somthing like a BNLS server, where as emulating the BNUpdate.exe functions, seems a better option for a battle.net bot.
I still think, emulating bnupdate.exe is going to cause many problems, each time their is a patch -- that kind of defeats the object of automating patching, if it need's to be updated each patch, to patch
I'm guessing that's somthing skying and yoni's BNLS suffer's from, today?

That's what I would do anyway, if I was releaseing a battle.net bot, able to self patch it's binary's.
I would emulate pretty much what the patch exe does, and simply say, you must have the required dat mpq's inorder for it to patch, other wise it will fail.

I guess you have the 3 options:
A) Download the patch exe from ftp.blizzard.com and run it. (requires game to be installed, but most fail safe method)
B) Emulate the patch exe functions, and extract and run BNUpdate.exe from the mpq you download from battle.net. (requires the dat mpqs)
C) Emulate the bnupdate.exe enough to patch the 3 checkrevision binarys. (may require reworking, each patch or 3)

I persionaly would go for option B, but option A seems immune to any bnupdate changes, option C seems the most likely to suffer from changes to bnupdate, where as option B seems somwhere in the middle.
That's just me tho.

[UserLoser]

Bnupdate.exe is what you gotta look at.  It took me only two days to reverse and figure out to get it running correctly about two years ago, as I'm still the only person I know of who has running working code for this besides Skywing.  mpqs.lst, revert.lst, patch.cmd are some useful files that my patcher extract.  mpqs.lst will tell you if there are any MPQ files that have the version check files in them (only useful on War3 patches usually).  There's several types of patch types, 0, 1, 2, 4.  0 is a bit complex, 1 is the simplest as it is a direct copy, 2 and 4 (WoW/War3) are newer binary sort differential stuff.

patch.cmd is useful because it has things like "This patch upgrades Diablo II from version 1.00 or later to version 1.10." for example.

Also you need revert files if you plan on using update-from-any patches (PROD_PLAT_1xx_VERSION.mpq) instead of regular updates.  Revert files are the very first game version files (i.e. v 1.00).  On the update-from-any patches you actually have to copy over your existing game files with the revert files then patch those (that's how it allows you to update from any version)

[Hdx]

From 5 mins of looking at it, 1 is extramly simple. [as you said]
(word) header len 0x18
(byte) unknown 0x04
(byte) ver
For 0, theres 8 bytes always 0, [checked 4 diffrent patches]
then u16 length of file data
and after that I have no clue what the data is, but i presume it's a crc or something knowing that bnupdate complains if the files dont match the old one, even on ver 1.
Havednt looked at 0 yet, but in the last patch only battle.snp was 0. So meh.