In-Game Packet Decompression

[tA-Kane]

Would someone Pin it along with the .BNI tut. Kain posted this way there both up at the top and noone asks + then theres also no need to search it out

I'm Kane by the way. Kain is someone else. There is a difference.

FYI, the file reference on BotDev has become invalid, as the file has moved to:
http://kbg3.ath.cx/misc/bni_file.shtml

Or, if you don't want to view the website's top and side panels, it's here:
http://kbg3.ath.cx/misc/bni_file.include

Additionally, BnetDocs has a document on the BNI file here:
http://www.valhallalegends.com/arta/bnetdocs/page.php?id=3
But, you should note that the BnetDocs one does not have the information on how to parse (render?) the embedded TGA image.

[Skywing]

Additionally, BnetDocs has a document on the BNI file here:
http://www.valhallalegends.com/arta/bnetdocs/page.php?id=3
But, you should note that the BnetDocs one does not have the information on how to parse (render?) the embedded TGA image.

That information is freely available on Truevision's FTP site.

[l)ragon]

Would someone Pin it along with the .BNI tut. Kain posted this way there both up at the top and noone asks + then theres also no need to search it out

I'm Kane by the way. Kain is someone else. There is a difference.

FYI, the file reference on BotDev has become invalid, as the file has moved to:
http://kbg3.ath.cx/misc/bni_file.shtml

Or, if you don't want to view the website's top and side panels, it's here:
http://kbg3.ath.cx/misc/bni_file.include

Additionally, BnetDocs has a document on the BNI file here:
http://www.valhallalegends.com/arta/bnetdocs/page.php?id=3
But, you should note that the BnetDocs one does not have the information on how to parse (render?) the embedded TGA image.

were all not perfect 8p
'typo' you all know who i ment.

i was just pointing out some topics to pin up, noticed your info up on botdev but forgot about it

[smoke]

[removed comment since it was mostly wrong]

-smoke

[smoke]

my mistake...

The original table is actually correct.  Blizzard treats those two values a little strangely to save 1 bit on each.  I'm not sure where the bug is in my compression table generator at the moment the inconsistancy to occur.  After fiddling with it for a while, I did manage to get my compression code to generate compressed data compatible with the original decompression code (tables).

-smoke

note: the 2 values talked about here were a couple of bytes that are slightly abnormal.  they caused some headaches in generating a valid compression table.  of course, that all means pretty much nil since I found the compression and decompression tables in the diablo ii memory space for 1.10s. (see below)

[Skywing]

my mistake...

The original table is actually correct.  Blizzard treats those two values a little strangely to save 1 bit on each.  I'm not sure where the bug is in my compression table generator at the moment the inconsistancy to occur.  After fiddling with it for a while, I did manage to get my compression code to generate compressed data compatible with the original decompression code (tables).

-smoke

There are indeed several bugs in the D2 packet decompressor code floating around - and you're right about them not being with the table itself.

You ought to be able to find them without too much trouble if you try to understand what all of the code does.

[smoke]

I just extracted all of the tables from memory of the 1.10s client.  They are all pretty much identical.  I also took the liberty of grabbing the huffman code table if you are interested in doing the compression.

Code Select Expand


HuffmanCodes:
01 07 17 1f 20 18 21 22 23 30 24 31 32 25 46 33  
34 35 26 19 1a 27 36 37 38 47 48 36 37 39 28 3a  
3b 38 39 3a 3b 49 3c 3d 4a 3e 3f 40 41 42 43 00  
4b 4c 44 45 46 47 48 49 4a 4b 4c 4d 4d 4e 4f 50  
4e 51 52 4f 53 50 54 55 56 57 58 59 51 52 5a 01  
53 3c 54 5b 02 55 56 5c 57 5d 5e 5f 03 60 61 62  
63 64 58 59 5a 65 66 29 2a 2b 67 5b 3d 2c 68 69  
6a 04 6b 6c 6d 6e 6f 70 71 72 73 05 74 75 76 77  
2d 78 79 7a 06 07 08 09 0a 7b 7c 0b 0c 0d 0e 7d  
5c 0f 10 7e 11 5d 5e 5f 7f 12 13 14 15 16 17 18  
80 19 81 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26  
82 27 28 29 2a 2b 2c 2d 83 2e 2f 30 31 32 33 34  
84 35 85 36 37 38 39 3a 3b 3c 3d 3e 3f 40 86 41  
42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f 50 51  
87 52 88 53 54 55 56 57 89 58 59 5a 5b 5c 5d 5e  
8a 5f 60 61 62 63 64 65 8b 66 67 68 69 6a 6b 1b

BitMasks:
00000000 00000001 00000003 00000007 0000000f 0000001f 0000003f 0000007f  
000000ff 000001ff 000003ff 000007ff 00000fff 00001fff 00003fff 00007fff

HuffmanLengths:
01 04 06 07 07 06 07 07 07 08 07 08 08 07 09 08  
08 08 07 06 06 07 08 08 08 09 09 0a 0a 08 07 08  
08 0a 0a 0a 0a 09 0a 0a 09 0a 0a 0a 0a 0a 0a 0b  
09 09 0a 0a 0a 0a 0a 0a 0a 0a 0a 09 0a 0a 0a 0a  
09 0a 0a 09 0a 09 0a 0a 0a 0a 0a 0a 09 09 0a 0b  
09 08 09 0a 0b 09 09 0a 09 0a 0a 0a 0b 0a 0a 0a  
0a 0a 09 09 09 0a 0a 07 07 07 0a 09 08 07 0a 0a  
0a 0b 0a 0a 0a 0a 0a 0a 0a 0a 0a 0b 0a 0a 0a 0a  
07 0a 0a 0a 0b 0b 0b 0b 0b 0a 0a 0b 0b 0b 0b 0a  
09 0b 0b 0a 0b 09 09 09 0a 0b 0b 0b 0b 0b 0b 0b  
0a 0b 0a 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b  
0a 0b 0b 0b 0b 0b 0b 0b 0a 0b 0b 0b 0b 0b 0b 0b  
0a 0b 0a 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0a 0b  
0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b  
0a 0b 0a 0b 0b 0b 0b 0b 0a 0b 0b 0b 0b 0b 0b 0b  
0a 0b 0b 0b 0b 0b 0b 0b 0a 0b 0b 0b 0b 0b 0b 06

CharTable (Base Address -> 0x6ff9f8b8):
00 00
00 01
00 ff
00 14
00 13
00 05
00 02
00 80
00 6d
00 69
00 68
00 67
00 1e
00 15
00 12
00 0d
00 0a  
00 08
00 07
00 06
00 04
00 03
00 6c
00 51
00 20
00 1f
00 1d
00 18
00 17
00 16
00 11
00 10
00 0f
00 0c
00 0b
00 09  
01 96 97
01 90 95
01 64 6b
01 62 63
01 56 58
01 52 55
01 4d 50
01 45 4c
01 40 43
01 31 3b
01 28 30
01 1a 25
01 0e 19
02 e2 e8 f0 f8
02 c0 c2 ce e0
02 a0 a2 b0 b8
02 8a 8f 93 98
02 81 82 83 89
02 7c 7d 7e 7f
02 77 78 79 7a
02 73 74 75 76
02 6e 6f 70 72
02 61 65 66 6a
02 5d 5e 5f 60
02 57 59 5a 5b
02 4a 4b 4e 53
02 46 47 48 49
02 3f 41 42 44  
02 3a 3c 3d 3e
02 36 37 38 39
02 32 33 34 35
02 2b 2c 2d 2e
02 26 27 29 2a
02 21 22 23 24
03 fb fc fd fe 1b 1b 1c 1c
03 f2 f3 f4 f5 f6 f7 f9 fa
03 e9 ea eb ec ed ee ef f1
03 de df e1 e3 e4 e5 e6 e7
03 d6 d7 d8 d9 da db dc dd
03 cd cf d0 d1 d2 d3 d4 d5
03 c5 c6 c7 c8 c9 ca cb cc
03 bb bc bd be bf c1 c3 c4
03 b2 b3 b4 b5 b6 b7 b9 ba
03 a9 aa ab ac ad ae af b1
03 9f a1 a3 a4 a5 a6 a7 a8
03 92 94 99 9a 9b 9c 9d 9e
03 86 87 88 8b 8c 8d 8e 91
03 2f 4f 54 5c 71 7b 84 85

CharIndex (Points to a row of the CharTable):
6ff9fa05 6ff9f9fc 6ff9f9f3 6ff9f9ea 6ff9f9e1 6ff9f9d8 6ff9f9cf 6ff9f9c6
6ff9f9bd 6ff9f9b4 6ff9f9ab 6ff9f9a2 6ff9f999 6ff9f990 6ff9f98b 6ff9f986
6ff9f981 6ff9f97c 6ff9f977 6ff9f972 6ff9f96d 6ff9f968 6ff9f963 6ff9f95e
6ff9f959 6ff9f954 6ff9f94f 6ff9f94a 6ff9f945 6ff9f940 6ff9f93b 6ff9f936
6ff9f931 6ff9f92c 6ff9f927 6ff9f924 6ff9f921 6ff9f91e 6ff9f91b 6ff9f918
6ff9f915 6ff9f912 6ff9f90f 6ff9f90c 6ff9f909 6ff9f906 6ff9f903 6ff9f900
6ff9f8fe 6ff9f8fc 6ff9f8fa 6ff9f8f8 6ff9f8f6 6ff9f8f4 6ff9f8f2 6ff9f8f0
6ff9f8ee 6ff9f8ec 6ff9f8ea 6ff9f8e8 6ff9f8e6 6ff9f8e4 6ff9f8e2 6ff9f8e2
6ff9f8e0 6ff9f8e0 6ff9f8de 6ff9f8de 6ff9f8dc 6ff9f8dc 6ff9f8da 6ff9f8da
6ff9f8d8 6ff9f8d8 6ff9f8d6 6ff9f8d6 6ff9f8d4 6ff9f8d4 6ff9f8d2 6ff9f8d2
6ff9f8d0 6ff9f8d0 6ff9f8ce 6ff9f8ce 6ff9f8cc 6ff9f8cc 6ff9f8ca 6ff9f8ca
6ff9f8c8 6ff9f8c8 6ff9f8c6 6ff9f8c6 6ff9f8c4 6ff9f8c4 6ff9f8c4 6ff9f8c4
6ff9f8c2 6ff9f8c2 6ff9f8c2 6ff9f8c2 6ff9f8c0 6ff9f8c0 6ff9f8c0 6ff9f8c0
6ff9f8be 6ff9f8be 6ff9f8be 6ff9f8be 6ff9f8bc 6ff9f8bc 6ff9f8bc 6ff9f8bc
6ff9f8ba 6ff9f8ba 6ff9f8ba 6ff9f8ba 6ff9f8ba 6ff9f8ba 6ff9f8ba 6ff9f8ba
6ff9f8ba 6ff9f8ba 6ff9f8ba 6ff9f8ba 6ff9f8ba 6ff9f8ba 6ff9f8ba 6ff9f8ba
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8
6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8 6ff9f8b8


Note:
The decompression code posted here merged the HuffmanSizes table and the CharTable.

[Lycaon]

Can I talk someone into doing a quick VB port of the decompression function for me? c++ is definitely not my strong suit.

[BinaryzL]

I will probably be porting it to vb later.

[UserLoser.]

Can I talk someone into doing a quick VB port of the decompression function for me? c++ is definitely not my strong suit.

This code is publically available via BnetAuth.dll - but beware, there's some some problems which may cause it to overflow or something and not work and crash your bot

[Lycaon]

I'll try the DLL til I either figure out why my ported VB function isn't working, or BinaryzL releases some code. I was hoping to get away from having to use extra DLLs.  

I was using the tables available from this thread (Brand.X's post).  Are those still valid or are there new/different ones with 1.10?

[Lycaon]

Nevermind, I got it converted.  Now I just have to redo my recv packet buffer :p