New Local Hashing System - No Hashfiles Needed

[Joe[x86]]

Yesterday iago and I had a little chat about caching CheckRevision results, and it led to this. HdxBmx27 is gathering a list of ValueStrings with several bots which can be viewed here. I wrote a little utility to remove the duplicates which can be downloaded here.

Where you guys come in, is neither Hdx or I are talented enough in the carbon languages to write an effecient program to run the CheckRevisions on a list of ValueStrings. Could someone slap together a program to imput a list of valuestrings (CRLF delimited) from a text file, run the checkrevision on the hashfiles for all games, and output the results to a CHECKREVISION.DAT file? The format I've suggested is available below, but your welcome to modify it if you'd like.

Code Select Expand

<CHECKREVSION.DAT>
   <A=30356806 B=639656831 C=516206995 4 A=A-S B=B+C C=C^A A=A^B>
       <STAR>
           <! Insert STAR checkrevision value with above valuestring here !>
       </STAR>
       <SEXP>
           <! Insert SEXP checkrevision value with above valuestring here !>
       </SEXP>
       <! So on for the rest of the games>
   </A=30356806 B=639656831 C=516206995 4 A=A-S B=B+C C=C^A A=A^B>
   <A=812419633 B=984045719 C=833306109 4 A=A+S B=B-C C=C+A A=A^B>
       <! Game data for this valuestring !>
   </A=812419633 B=984045719 C=833306109 4 A=A+S B=B-C C=C+A A=A^B>
   <! The rest of the valuestrings and their data>
</CHECKREVISION.DAT>

[iago]

To clarify, this isn't possible.  Sorry to get your hopes up

There are more possible value strings than bytes on your harddrive.

But caching them for a couple minutes is good. Battle.net only changes their strings every couple minutes.  Doing that on a popular JBLS server, we got the stats:
1230 hits
904 misses

That means that 60% of the time, CheckRevision doesn't have to be calculated

[Joe[x86]]

I was under the influence that.. no, thats a bad way to state it.

I misunderstood something you said, and was thinking there was only about 20 valuestrings ever used.

-- Project Closed --

[UserLoser.]

Could just create a table & fill it with the same contents of each game file, then you can run the check the same way, just no other files involved.  I'm not saying this is a good idea, or I do it, just it's an idea for a localized, non-requiring method

[iago]

The file version information can be cached, which would atleast reduce the number of api calls required. The benefits would be more significant if you are multi-loading.

I might try that table idea. The checkrevision only reads the first 1/4 of the files (from my understanding atleast), which could reduce the distribution size of hash data.

Combine that with version info caching and you might have a more efficient solution.

Just had another thought, you could compress that 1/4 file data in memory and load with the application, that way no harddisk I/O access is required.

Actually, it rounds the file down to the nearest multiple of 1024.  So you could save a few bytes of harddrive space if you were into that.

Joe- what I meant was that if I unrolled the innermost loop, I'd have about 20 different conditions.  Which I was wrong about, but it's ok because I found a better way

[Joe[x86]]

UL, this whole idea was pointed at having a small file to get the results from instead of actually running it on the large (to 56ker's) hashfiles, making getting hashfiles more 56k friendly.

[UserLoser.]

UL, this whole idea was pointed at having a small file to get the results from instead of actually running it on the large (to 56ker's) hashfiles, making getting hashfiles more 56k friendly.

You realize that what you said makes absolutely no sense

[iago]

UL, this whole idea was pointed at having a small file to get the results from instead of actually running it on the large (to 56ker's) hashfiles, making getting hashfiles more 56k friendly.

You realize that what you said makes absolutely no sense

He said he wanted small files instead of funning it on large files, making it easier to get for people with slow connections.

What's the problem?

[DeTaiLs]

War3 hash files arent 56k friendly at all

[UserLoser.]

Nevermind, didn't see "getting".  If he didn't say getting my statement is correct

[iago]

Nevermind, didn't see "getting".  If he didn't say getting my statement is correct

If you didn't say "see", then your sentence didn't make any sense. 

[Eric]

Just archive the binary files... you only have to download them once every few months and even then, it's only a < 1-2 minute download on a dial-up connection.  Either deal with it or use a remote hashing server to pass the version check.

[MyndFyre]

This isn't hard to understand.  Instead of actually hashing the data, he wants to cache every possible version string and then the hashed data value for each product.  He doesn't want to have the files to hash into the other data.

However, there are a VERY LARGE number of potential version strings -- not just 20.  That is the problem that iago is trying to express to you guys -- as he said, there are more possible version strings than you have disk space.  You can't account for each one.